Acronyms and abbreviations
- CA – Constraints Analysis
- CBA – Cost Benefit Analysis
- CEA – Cost-Effectiveness Analysis
- EA – Economic Analysis
- EIF – Entry into Force
- EMIS – Education Management Information System
- ERR – Economic Rate of Return
- GDP – Gross Domestic Product
- IMC – Investment Management Committee
- LSMS – Living Standards Measurement Study
- MCA – Millennium Challenge Account
- MCA – Multi-Criteria Analysis
- MCC – Millennium Challenge Corporation
- M&E – Monitoring and Evaluation
- NPV – Net Present Value
- O&M – Operations and Maintenance
- PIR – Policy and Institutional Reform
- PV – Present Value
- TVET – Technical and Vocational Education and Training
- RBF – Results Based Financing
- RCA – Root Cause Analysis
- SD – Standard Deviation, of test scores
- SCDPs – Sector Consistent Design Patterns documents
- UA – Uncertainty Assessment
- USD – United States Dollars
- VA – Value Added
- WTP – Willingness to Pay
I. INTRODUCTION
A. Motivation
Between 2005 and late 2022, MCC invested over $628 million in education and workforce development interventions at almost every level of education (from primary through tertiary and adult continuing education), formal and non-formal, and in education that results in both academic and technical credentials. As with all MCC investments, those in education and workforce development aim to support the agency’s mission to reduce poverty through economic growth. Theory and evidence points to the importance of human capital —and especially the knowledge and skills attained through education, training, and work experience— as a key determinant of economic growth. In particular, greater levels of human capital facilitate the adoption of new technologies, and can increase efficiency and productivity, and thus enhance economic growth. At the firm level, the supply of skills is one of the criteria that businesses consider when deciding whether to invest, expand, upgrade technology, and hire more workers.
From the perspective of individuals, one of the most robust associations in research on economic development is that between earnings and level of education, bearing out the prediction from human capital theory that more educated people are, on average, more productive and therefore have higher incomes. Reviewing this relationship across countries and across time, Psacharopoulos and Patrinos (2018) find a global average of private (individual) returns to an additional year of schooling to be about 9 percent – a result that has remained stable across decades. Furthermore, disaggregated data show that women continue to experience higher average returns to schooling, and that returns to all are also higher in low-income countries. These findings highlight that targeted education and training investments can have strong implications for income distribution, poverty reduction and inclusion, given that education can equip less advantaged citizens with the means to better access income generating and welfare improving opportunities.
Despite a widespread understanding that education can provide significant benefits to individuals and to economies, notable market failures lead to underinvestment in human capital by private actors (both households and firms) and thus implies a role for public policy. A key market failure is the inability of poor families to borrow to finance their children’s schooling, despite the high potential benefits. In the absence of credit, opportunity costs loom large for poor families (and youth), since children’s work on farms or domestic work is often needed to sustain household consumption, while older children may need to be employed to support themselves or their families. Parents may also lack information about, or not fully value, these benefits. As a consequence, the benefits are not fully taken into account in their decisions regarding investment in their children’s education, implying an underinvestment from a social perspective. On the part of firms, while they would clearly benefit from having a more educated and highly skilled workforce, they do not have an incentive to invest heavily in education (or training) as they generally will be unable to capture these benefits, since those who receive the education are free to work for any firm, not just the one providing the education—i.e, there is an externality. For industry specific skills, firms often do provide on the job-training, though this tends to be less than socially optimal, for the same reason. Related to this externality is a collective action problem. For example, firms in a given industry could collectively provide relevant training and thus meet their skills needs, but enforcing this cooperation can be difficult.
These market failures imply an important role for the public sector to fund, and usually to directly provide, education services. Governments indeed devote substantial resources to education. In 2014 the global average indicated that countries spent about 5% of their GDP on education (World Bank DataBank), and many countries have experienced an increase in private sector provision at all levels of education, as well as a mix of public and private institutions. In many countries, private education provided by religious organizations, for example through Koranic and Catholic schools, is significant. Rising education expenditures have led to a dramatic increase in access to education over the past five decades in developing countries, leading to significant improvements in the quantity of schooling attained. Nevertheless, achieving universal secondary enrollment and completion remains an elusive goal, and the quality of education and learning has also lagged and is often extremely poor, which some recent research suggests explains the lack of association of education and growth at the country level. In significant part this is because governments lack the resources to further expand access to education and improve its quality. Other factors, including poor management of the education sector, are also important. Donor agencies such as MCC can play an important role by infusing resources as well as providing technical assistance for reforms in education, assisting with changes targeting the system/sector, school, teacher, and student levels.
While much of the foregoing has emphasized constraints on increasing the supply of human capital or skills, it should be obvious that this is not the whole picture as the skills provided to individuals need to be demanded in the labor market for there to be significant welfare improving, income and economic growth impacts. In a broad sense, this consideration motivates MCC’s starting point in the compact process, which is usually stated as the identification of binding constraints to private investment and economic growth–private investment being the main source of the demand for labor and for labor with specific skills. Within the scope of the present SCDP, labor market demand considerations play a crucial role in problem diagnosis and program design in education and training, particularly for technical and vocational education and training (TVET), which is—or should be—tightly linked with the labor market and the needs of employers.
B. MCC Experience
Across countries, one can recognize similar patterns and causes of poor education and training outcomes, yet the details of the problems and their potential solutions remain country specific. Evidence-based policy and data analysis are foundational to MCC’s model, a process that begins as soon as a country is selected as an eligible partner. MCC and country counterparts begin collaborating on the Constraints Analysis (CA) to identify the most significant and binding constraints to that country’s economic growth. Since MCC conducted its first CA in late 2007, by 2022, 41 analyses have been completed with MCC partner countries, of which 11 found inadequate education and skills to be one of the binding constraints. Building on the CA findings, the next step in the process is to conduct a root cause analysis (RCA) on prioritized binding constraints. The RCA is intended to drill down to the crux of why specific underlying problems exist. This leads the way to identify where public policy–and MCC as an international development funder–can serve to improve market outcomes and pave the way to greater economic growth and poverty reduction.
To address the country-specific challenges related to education and workforce development, MCC has dedicated its resources to a variety of investment categories including infrastructure and equipment, policy and institutional reforms, technical assistance, curriculum development and training of teachers and others employed within the education system. Interventions have targeted all levels of the education system, from the national government or education ministries to schools and training centers, to educators, to students and their households, and even to strengthening linkages between education and the labor market. Spanning 10 countries in Central America, Africa, Asia, and Eastern Europe, the interventions to date have also typically included efforts to improve education equity and inclusion of marginalized populations.
As project teams move from problem diagnosis to project identification, project logics are developed, and MCC economists carry out a cost-benefit analysis (CBA) of the projects. MCC’s CBA practice is a reflection of the Agency’s strong commitment to the use of rigorous evidence to inform investment decisions and is a requirement for all MCC compact investments. The CBA methodology requires the analyst to quantify all anticipated costs and benefits (private and social) of a potential investment, and then express those costs and benefits in monetary terms to estimate an economic rate of return (ERR) on the investment. This summary metric allows for comparison to other potential projects as well as to some minimum required rate of return. For MCC projects the typical ERR threshold that projects are expected to meet is 10%. The analysis is conducted at several points in time over the life of a compact investment. The most important is the CBA that informs the initial investment decision. During program implementation, a revised CBA may be needed to guide decision making when changes occur to the originally designed project. Additionally, MCC economists produce a ‘closeout’ CBA model within about a year after a compact’s closure, which will fully capture changes to project cost and scope. Finally, an independent entity is typically engaged to conduct an impact evaluation of the project, the findings from which are increasingly being utilized to conduct ‘evaluation-based’ CBAs. Note that for this final stage, the CBA uses actual information on realized project benefits from the evaluation, in contrast to predicted benefits in earlier versions of the CBA.
The closeout CBA and evaluation-based CBA are often linked to the last step in a project’s lifecycle: assessing the success of its implementation and its ability to achieve the intended objective. Evaluation is integral to MCC’s commitment to accountability, learning, transparency, and evidence-based decision making. MCC published its first general education evaluation findings in 2011. The Insights from General Education Evaluations Brief reviews and synthesizes MCC’s findings from its independent evaluations in general education, covering investments in both primary and secondary education. The evaluation results are supplemented with lessons learned developed by MCC staff. MCC plans to conduct a deeper analysis of lessons learned for general education, which will be published in a forthcoming Principles into Practice paper. The 2020 Principles into Practice: Training Service Delivery for Jobs and Productivity reviewed MCC’s lessons in technical and vocational education and training.
Lastly, it should be noted that CBA is not the only approach to assessing education projects. For example, cost-effectiveness analysis (CEA) is often used to compare education-related investments based on the costs to achieve a specific educational outcome, such as higher test scores or grade attainment, that is quantified but not assigned a monetary value. CEA could be more broadly considered at MCC, and particularly for education when multiple intervention options are on the table to support improvements in a specific educational outcome. Further, MCC’s own Beneficiary Analysis, discussed briefly in this paper, assesses projects on how well they target specific populations of interest, including the poor and women.
C. Purpose and Organization of the Guidance
This Sector Consistent Design Pattern (SCDP) guidance in Education follows similar guidance documents for other key sectors, namely Water Supply and Sanitation (Osborne, 2019), Land (Bowen and Ngeleza, 2019), Transport (Carter, 2020), Power (Epley, Mulangu, and Bowen, 2021), and Agriculture (Szott and Motamed, 2023). There is also a forthcoming SCDP in Health (Myers, Osborne, and Payaam, 2023). Each SCDP is built upon MCC’s general Guidelines for Economic and Beneficiary Analysis, which provides the overall principles that should be applied to projects in all sectors. The SCDPs are intended to serve as “living” resources that will be updated as necessary, as evidence and methods evolve and MCC’s approach shifts to reflect the latest findings in the sector.
The main purpose of this document is to provide guidance for MCC’s economic analysis of interventions in education. It will support analysts and economists who will conduct or review this work. These individuals will include MCC analysts, consultants to MCC, peer reviewers and counterparts in our partner countries. To reach this objective, this guidance seeks to provide a way of thinking about costs, benefits, uncertainty, and other topics, noting the strengths and weakness of different methodological approaches. It does not provide specific tools or spreadsheets that could serve as a paint-by-numbers approach to doing CBA in education. In fact, a main takeaway from the paper should be that the specifications of each model will be highly dependent on the location, context, problems being addressed, and detailed objectives of the proposed interventions.
This documentation of MCC practices and experience can also support several other purposes such as to improve consistency in MCC’s own work in education CBAs as carried out by different economists; facilitate peer reviews; and increase the transparency and accountability related to this work. By fostering dialogue on the strengths and limitations of CBA for different types of education interventions, this guide can help to advance the fields of education, monitoring and evaluation, and economic analysis.
This guidance is based on the broad field of education and training research but is grounded in MCC’s experience and the interventions that the Agency normally supports. Reflecting MCC’s investment portfolio, we focus on general education at the primary and secondary levels, with some reference to tertiary. Not covered here are pre-school/pre-kindergarten level interventions. While such early interventions have been shown to have significant long-term impacts, MCC has not yet invested in this space, so we do not consider such projects. In addition to general education, the paper also considers technical and vocational education and training (TVET) programs, which are intended to provide skills for the world of work, thus encompassing (in a broad definition) any education and training which provides knowledge and skills to enhance productivity and success at work, whether provided through formal or non-formal approaches and in school or work-based settings. Clearly, TVET is part of education in a broad sense (and the reverse, since all schooling ultimately prepares young people for the world of work in addition to other objectives), but given its particularly tight linkages to livelihoods, it is also distinct and therefore is given separate treatment in the paper.
Note should also be made of two topics tightly related to education and training that are touched upon frequently within this guidance: labor markets and health. With regard to the former, as mentioned above, the impacts of the education system play out most significantly in the labor market. Thus, labor market characteristics and conditions influence the returns to education and training investments, potentially reducing the expected impacts. MCC has occasionally made investments aimed to improve the operation of labor markets. Such interventions are not covered by this CBA guidance, but the discussion below will refer to labor market factors, especially when considering TVET programs. With regard to health, an individual’s health, not just education or skills, is encompassed within the concept of human capital. This will be further discussed below, but as with labor market interventions, this guide does not address CBA of health interventions, as a separate forthcoming guidance paper will be dedicated to economic analysis of MCC’s health and nutrition programs.
Organization of the Guidance
With this framing in mind, the remainder of this education and workforce development CBA guidance is organized into two main sections. Section II is focused on providing key background and programmatic information that is essential for designing a strong CBA model that captures the characteristics and objectives of the education investment. This section includes the following three components: the identification of problems; a typology of interventions or ‘projects’ to address those problems; and the use of these components to develop the project’s logic to get from inputs to outputs to outcomes and the longer-term impacts. Through Section II the reader should have a better understanding of the education sector and be more equipped to dive into the technical aspects of the economic analysis, which is the subject of Section III.
Section III begins by introducing the economic logic, which uses a well-defined counterfactual (i.e., a without project scenario) to capture the program logic’s outputs and outcomes as the CBA model’s intended benefit streams, with note on how these change across time. Section III. A, provides considerations for how to develop the counterfactual, time horizon, and discount rate for education and training projects. Section III. B follows with a focus on the work to identify, quantify, and monetize benefit streams. Three interventions based on MCC experience are included to provide concrete examples of applying these concepts to education and training projects. The discussion shifts in Section III. C to outline key cost considerations and MCC practices in defining and measuring all social costs require to reach the intervention’s intended benefits. All the topics in Section III are brought together in subsection D to determine the CBA results, with a description of useful sensitivity analysis that can inform the CBA metrics reported, as well as distributional analysis that can highlight results by important groups. Finally, Section III. E concludes by acknowledging the limitations of this guidance and provides recommendations on areas for further examination or coverage in a future version the SCDP in education and training.
II. EDUCATION AND TRAINING SECTOR: SETTING THE STAGE FOR CBA
A. What’s the Problem?
The field of education is complex, and interventions can be motivated by the desire to address numerous problems. This guidance does not attempt to go into detail on every type of problem that might be encountered in the education sphere, but rather to describe the problem diagnosis approach at MCC and highlight aspects specific to education and training. In particular, the section outlines three core problems identified in the literature and through MCC’s experience in education and training: – (1) inadequate quantity of education (encompassing low enrollment and completion rates), (2) poor quality of the education received, and (3) low relevance of skills obtained in the labor market (also referred to somewhat more narrowly as a skills mismatch). This section also provides insights into the potential root causes of these core problems—the next step in the process of designing interventions to address well-defined problems – and explores the linkages between education-related problems with other related topics, namely social and gender inclusion, labor markets and health.
For any sector, correctly identifying the problem, who is affected by it and how, is critical for the design and success of an investment in achieving the desired outcomes. Well-defined problems support the development of appropriate interventions to address those identified problems, as well as the proper selection and measurement of the related benefit streams, and ultimately the development of a more cost-effective project overall. Within the MCC context, after a country is selected as an eligible partner, joint work by MCC and the partner country team begins on the problem diagnosis through a Constraints Analysis (CA) to identify the greatest constraints to that country’s economic growth. Within the MCC team, the economist leads this work, which is built upon the foundations outlined in the Growth Diagnostic work by Hausmann, Rodrik, and Velasco (2005). Since MCC conducted its first CA in late 2007, 41 such analyses have been completed with MCC partner countries, of which 11 found human capital, particularly in education, to be one of the binding constraints to their economic growth (Annex I).
Given that human capital, broadly defined, is a key input into production, it is not surprising that constraints analysis for developing countries may find that inadequate human capital is constraining a country’s economic growth and its opportunities to reduce poverty through economic growth. Human capital can be a constraint either through low productivity due to poor current health or developmental deficiencies stemming from poor nutrition in childhood (as covered in the forthcoming health SCDP), or—the more commonly analyzed pathway and our focus here– through a deficient supply of skills for productive work. From the perspective of private firms, the availability or cost of human capital will affect decisions about investment, choice of technology, and exporting, thus strongly influencing productivity and economic growth. An education or skills-related human capital constraint occurs when the supply of skills does not meet the actual or potential demand of employers that would allow them to effectively manage, operate, and/or expand businesses overall or at a competitive cost. This is distinct from labor market problems on the demand or policy side that do not directly involve a human capital shortfall. For example, excessive labor regulations can make it difficult or costly to hire individuals with the right skills, or to dismiss those without the necessary skills. Finally, private investment and the associated demand for labor may be constrained by various factors, such as a poor macroeconomic policy environment, that lie outside the labor market (as well as the education sector) itself.
It is important to highlight as well that human capital constraints have especially strong implications for income distribution and poverty reduction, given that education can equip citizens with the means to access income generating and welfare improving opportunities. Targeted investments to tackle such constraints among marginalized subsets of the population can thus support greater inclusion and income distribution.
After the binding constraints to growth are identified, the next step in the process requires MCC country teams to conduct a root cause analysis (RCA) on each constraint. The RCA asks why a constraint exists. For example, root causes of low quantity of education might be found to be one or a combination of a lack of physical access to schools, high private costs of education (combined with lack of access to credit), or a belief that there are no or few benefits of schooling or schooling beyond some level ( especially for girls), or a perception that schools or teachers, hence the education provided, are of poor quality. This is detailed further below with additional examples of root causes for education and training constraints.
Annex II lists the 13 MCC country programs to date with education investments, noting whether a CA was conducted and, if so, then whether it found education as a constraint, or as a root cause to a different binding constraint. This provides a more complete picture of the connection between the CA, RCA and where MCC has invested in education. In summary, of the 13 country programs with education investments, 7 of them were from the period before CAs became an MCC practice. Of the remaining 6 that were based on CAs, in all but one case education was identified as a binding constraint. The remaining case was the CA for El Salvador’s second compact, for which education was found to be between a binding constraint and a root cause–The analysis framed the main constraint as low productivity in exports, with one of the three main barriers to this constraint found to be human capital deficiencies related to education. There are also 5 additional countries where education was found to be one of several binding constraints, but due to other MCC decision factors these did not result in education-related investments.
The shift from speaking about constraints to problems occurs when MCC begins to assess what aspects of the constraint could be addressed by an MCC-supported intervention – i.e., taking into consideration the other MCC decision criteria. The defined ‘core problem’ will be closely related to a constraint but is often narrower, and will become the focus for the RCA, which will be directed at identifying root causes of MCC-actionable problems. This process to further define the problem will result in a specific problem statement that can be used to inform project identification, project logic, and the economic logic that underpins the CBA. For the remainder of this subsection, the term ‘core problem’ is used, but as noted these are often similar to the identified constraints.
During the last 15 years of conducting CAs, three main education-related constraints, referred to simply as problems here, have emerged. Table 1 summarizes these problems, which are by no means mutually exclusive. The second column of the table indicates what kind of evidence would lead to identification of that problem, and the third column points to possible underlying causes of that problem that may emerge in the RCA.
In no particular order, the first problem is described as an inadequate quantity of education. In this situation, it would be difficult for employers to find enough workers with the requisite level of education to meet their needs. In some countries there could be a lack of graduates from primary or lower secondary school, resulting in low literacy and numeracy among those in the labor market. Note that ‘quantity’ could be indicated by graduation rates for different levels of schooling, by grade attainment, or simply enrollment overall or at different levels.
Second is the low quality of the education. When education quality is a problem, employers may observe adequate numbers of graduates at different levels of education (i.e., quantity of schooling), and in the right fields, but their skills are poorly developed. One might observe that many students graduate from secondary school without obtaining adequate skills, for example. For the analyst, assessing whether skills are up to a certain level may be easiest for general education in areas such as basic numeracy and literacy, where national, regional, or international exams may assess students’ competency against pre-defined standards for a given level of education. Note that the low education quality problem is distinct from, but potentially related to quantity of schooling. One can imagine a situation where low quality of education and resulting poor learning outcomes for students reduces the expected productivity and earnings benefit of additional years of schooling, which lead parents or students to choose not to proceed beyond some relatively low level of years of schooling.
The third problem is a skills mismatch, or more broadly, a lack of labor market ‘relevance’ of schooling obtained. In this situation, there may be plenty of graduates but not in key fields of study, or with the types of skills, needed by employers. The mismatch problem may be most relevant to TVET and higher education, where students tend to select a particular course of study and may select fields that do not match the specific needs of employers. However, it should be noted that the notion of skills mismatch is a broad one. Perhaps most obviously, it encompasses a lack of individuals with specific skills for particular industries, such as food processing technicians or aerospace engineers. Yet it could also refer to a lack of more general technical skills that can be used in many sectors, such as IT skills, accounting, or engineering. Even more broadly, it could describe a situation where students, with an eye toward ‘safe’ public sector jobs, tend to select a range of fields of study like law or humanities rather than STEM fields. The latter may be more valued by the private sector, but the jobs offered by the private sector may be viewed as less secure or having fewer benefits and thus be considered less desirable than employment in the public sector (a pattern observed in many developing economies). Clearly, the broader definitions of ‘mismatch’ may start to look more like ‘general skills deficit’ than a highly specific ‘skills mismatch’. Whatever the terminology applied, it is important to be clear about the type of mismatch or relevance problem that exists in a given situation, as the root cause of the problem and indicated interventions may differ. For example, the first case noted above could imply the need for industry specific TVET, while the third and likely also the second could imply that changes are needed in general education coursework or in incentives to entice students to enter or complete their studies for in-demand fields.
As just noted, TVET investments are typically thought of as a solution to the third problem, a skills mismatch. Defined more broadly, however, TVET also encompasses interventions to address aspects of both the first and second problems. This is particularly relevant for training programs that focus on general and remedial skill building, such as literacy training or soft skills training for individuals who are no longer in formal schooling, and who are currently unemployed or economically inactive, i.e., out of the labor force. This and other types of TVET interventions will be defined in the next section of the paper.
| Problem | Potential Evidence of the Problem | Potential areas to examine during Root Cause Analysis |
|---|---|---|
| There are not enough graduates to satisfy market demand. It could be that there are not enough graduates at all levels of education or at only certain levels. |
|
If there are not enough graduates, questions for the RCA would include:
|
| Problem | Potential Evidence of the Problem | Potential areas to examine during Root Cause Analysis |
|---|---|---|
| There may be enough graduates at the different levels of education, and they may be graduating in the right fields, but their skills are poorly developed. |
|
During the RCA the team will try to understand if this problem is due to:
|
| Problem | Potential Evidence of the Problem | Potential areas to examine during Root Cause Analysis |
|---|---|---|
| If there are plenty of graduates (e.g., of secondary school), but they have not acquired skills that the labor market demands then this could be an example of a “skills mismatch,” which may be most relevant to the TVET and higher education arenas. |
|
The RCA may question whether there are problems with the quality of instruction, quality or relevance of curricula, not enough programs provided in the market at the right level or in the right fields, market distortions, or other mismatched incentives/preferences (e.g., prejudice against technical vocational training). |
Examining Separate, but Related Problems: Considering their implications for education-related interventions
Thus far, we have focused on education and training-related problems, but there are considerations that require more attention to obtain a holistic perspective of the constraints and their root causes. This is particularly important as we move from problems to interventions that aim to address those problems. Here we briefly note the relationship between the key problems identified above, and three other factors, which are typically examined separately in the CA: inclusion and equity, labor markets, and health. Their relationship to education should be assessed, and as relevant, incorporated into the intervention’s logic and design, as well as the CBA model. The analyst should determine the potential for these factors to influence the education-related intervention’s costs and benefits overall and their distribution among subsets of the population. The specifics and significance of these issues will depend on the country context.
- Equity and Inclusion: For any of the three key problems outlined above, the questions asked during the CA and RCA consider how the problem may affect citizens differently based on income level, gender, location, ethnicity, native language, etc. If education or training opportunities are not available or accessible to certain groups, the ability of these groups to enjoy improved incomes and welfare will be hindered, and overall economic growth itself may be restricted. The context should be considered for each country to determine how the lack of access to education or training limits inclusion (and/or is caused by a lack of inclusion). Identifying disparities and related challenges during the problem diagnosis phase will support teams in designing programs that address these issues more wholistically and that integrate these aspects into the project’s design. For example, interventions to improve quality via curriculum changes may require special support for less advantaged groups for them to also benefit. Obviously, analyzing these issues requires information disaggregated by gender, income level, and other relevant categories.
- Labor Market-Related Problems: The CA framework examines this topic under the assessment of microeconomic risks, but it is important to emphasize here (as we did briefly in the Introduction) as the main economic value of education (for those who work for a wage or salary) is determined within the labor market. When examining education and labor market information the analyst needs to determine if the evidence is pointing to a constraint in education, the operation of the labor market itself, or perhaps a mix of the two. Correctly identifying the problem(s) will be critical in developing an appropriate program. In instances where obstacles exist in both the labor market and level of human capital, investments to improve education are likely to have limited impacts if no complementary investments are taken to address labor market constraints. For example, in many countries a highly regulated labor market reduces labor demand in the formal sector, forcing even many well-educated labor force entrants into less productive and less remunerative informal activities. Labor market inefficiencies are not limited to issues of regulation but can also encompass constraints on migration (e.g., rural to urban) to access opportunities, information asymmetries that constrain hiring ( especially of younger workers), and other factors. It should be noted that more broadly, a wide range of factors outside the labor market itself, including macroeconomic and trade policies, may inhibit firm growth and labor demand, and will be examined in the constraints analysis.
On the labor supply side, labor market policies, laws or regulations can incentivize or disincentivize investments in a given type or level of skills. For example, as noted earlier, governments may incentive training for careers in the public sector through generous benefits and job security that are not offered in most private sector jobs.
- Health: Poor health or nutrition may contribute to the first two education problems (low quantity and quality) outlined above. Children who experience poor nutrition or poor overall health are more likely to miss days of school, repeat grades, or dropout. Even when these students attend school, their health condition impacts their ability to learn, with the literature showing that malnourished children have difficulty concentrating and retaining information. Therefore, both education quantity (e.g., attainment) and ‘quality’ (defined here as individual ability to learn, not objective quality of teaching or infrastructure) are negatively affected. Further, there are important linkages in the other direction, from education to improved health and nutrition; in particular, a large literature shows the positive impacts of mother’s education (controlling for household resources) on children’s nutrition. Further, higher incomes as a result of having more education enables individuals and families to better afford food and health care. The education-health nexus is also related to equity and inclusion, given that access to sufficient, nutritious food or necessary medical care is disproportionately a problem for poorer households or certain subsets of the population. Health is examined separately in the CA and a future SCDP will be dedicated to this topic, as well as an MCC produced guidance and toolkit on how to appropriately examine nutrition. However, as health issues can clearly impinge upon education success, it is important for teams to be able to identify problems in both areas where they are relevant, and in turn develop appropriate solutions to address the root causes of those problems.
B. How MCC Country Programs tackle identified Problems
The previous section described MCC’s problem diagnosis process, moving from the Constraints Analysis to the Root Cause Analysis in order to develop detailed problem statements and from that to focused interventions. As noted, the problem statements inform the eventual project’s logic as well as the economic logic that underpins the CBA. This section describes education-related interventions, focusing on the potential inputs and outputs that they can produce. A broad set of potential interventions is outlined below with indications on where MCC has had experience. The sections that follow will dive into the overall project logics and economic logics for key interventions implemented by MCC and continue by detailing those interventions’ expected outcomes and benefit streams.
Donors and governments support a wide range of investments in education and training, yet a unified sector-level taxonomy has not been widely adopted. In an effort to organize the myriad of potential investments, this document separates interventions into the following four categories, based on the level at which the intervention’s implementation is intended to directly impact: the overall education system; the school or training center; the educator teacher; and the student or household. The first three focus on supply-side interventions, meaning that they aim to support improvements in the provision of education, whereas the fourth aims to increase demand for education by focusing on households and students. These categories are not necessarily mutually exclusive and indeed are often combined to holistically tackle complex problems. It should be noted that the level at which an intervention is implemented is not necessarily the level at which decisions about that interventions are made. Most importantly, changes in curricula, teacher training, school hours, etc., will often or even typically be determined at the system or ministry level, which has overall decision making and budgetary authority.
Further, within each level, interventions may be further distinguished by the focus or elements of the intervention, which may include policy and institutional reform (PIR), ‘hard investments’ like infrastructure and equipment, or non-infrastructure (‘soft’) investments such as training, technical assistance, and curriculum development. A further, distinct, element consists of measures to improve equity and inclusion of marginalized populations, which may feature as part of interventions at any of the levels.
The four categories of intervention levels are described in the bullets that follow, and Table 2 provides an illustrative list of potential investments for each level, and mentions as examples related MCC programs (note these do not comprise an exhaustive list of MCC education investments).
- System: At the highest level, these supply-side interventions are designed to impact the system that governs education service delivery, at a national or regional level. They often involve but are not limited to PIR (and PIR can be implemented at lower levels as well). For example, several projects noted below involve instituting systematic data collection about schools and students to facilitate data-driven decision making when allocating limited education resources. Such an intervention requires hard investments in computing hardware and software, as well as expertise to design a data collection system that can ultimately support PIR that can shift practices in using data to inform decision making. Interventions at this level could also include an overhaul or initial development of a national curriculum for a certain level of education or specific subjects, or a system for the operation and maintenance of schools or the use of public-private partnerships, particularly for TVET. At this level, decisions and implementation would both be expected to center around the system level.
- School/Training Facility: These supply-side interventions may include new, expanded, or rehabilitated buildings, new equipment, or improved teaching materials. They may also encompass school-level adjustments in practices, such as increasing the length of a school day, changes in class size, or adopting a school-based management approach. While these interventions are implemented at the school level, system-level decision making would likely play a role in defining school-level policies or determining the amount of funding available for a given purpose.
- Educators (teachers, principals, other school staff): Educators—above all the teachers–are a critical input to the education system. Interventions at this level employ various mechanisms to increase the quantity and/or improve the quality of teachers or other staff (existing and new), and potentially their supervisors as well. Teacher training is a common intervention that can include a variety of approaches such as pre-service training for those entering the profession, in-service training for existing teachers, and trainings themselves can differ by total hours, the mix between pedagogical and subject-matter instruction, mentoring, professional peer groups, etc. Additionally, teacher-level interventions will often require altering incentives and expectations of teachers, which may be categorized as PIR or coupled with larger system-level or school-level efforts. PIR-related activities at this level may also include, among other measures, a salary scheme based on qualifications and performance, or early retirement incentives that entice underqualified teachers to leave the system.
- Students/Households: These interventions seek primarily to stimulate the demand for or access to education, often focused on certain groups that may be considered disadvantaged or excluded from participating due to various reasons. Interventions targeting students or households may also be designed to impact the quality of learning, e.g., via school-based nutrition programming. Funding for such programs often comes from system-level decision making but could stem from grassroots or community-based efforts.
In addition to categorization by the level at which an intervention is implemented, investments are distinguished by whether they are in general education or TVET, and at what level of education – i.e., pre-school or early childhood education, primary, lower secondary, upper secondary, tertiary, and continuing education. The interventions outlined in the table are generally applicable to all education levels and types of schooling or training, but these further distinctions by school type or level will come into play during detailed project design and appraisal. This is particularly true for TVET investments, which in addition to basic elements such as infrastructure or teacher training, also have features reflecting orientation to and linkages with the labor market and employers. This leads to interventions that extend beyond the education system proper. The table below thus includes two additional categories to capture key activities within the labor market and with employers, which are often incorporated into TVET interventions.
Continuing briefly with the focus on TVET, several different forms of TVET can be noted, each reflecting specific objectives directed at specific beneficiary populations defined by their status in the labor market. The following three-way categorization is useful (Almeida, et al., 2012), though as noted below programs may combine elements of more than one type:
- Pre-employment training: Training in specific fields, usually as part of the formal education system and for youth who are still in school (i.e., who have not yet entered the labor market) or those who are looking to change careers.
- Training-related active labor market programs: Skills training – technical and industry focused or general and remedial, e.g., literacy training or soft skills training — for individuals who are no longer in formal schooling, and who are currently unemployed or out of the labor market. Typical participants include, but are not limited to, those who have dropped out of school, disadvantaged youth, and women who are not in the labor force.
- On-the-job or Continuous (or ‘In-Service’) training: This includes occupation-specific training for employed individuals, which may be provided by their employers privately or in cooperation with the public sector or a TVET center.
| Intervention’s Focus | Types of Potential Interventions (not exhaustive) | Examples: MCC Investment Experience |
|---|---|---|
| Education System [typically, at national or regional level] | Generally broader in scope and designed to impact the system and governance of education service delivery.
|
|
| School [all education service delivery facilities, including training centers] | Focus on altering the quantity and/or quality of education service delivery facilities. (Note: decisions on these interventions are typically made at a higher, i.e., system, level)
|
|
| Educator [Teachers, principals, other staff] | Interventions aim to address the quality and/or quality of educators and potentially of their supervisors as well, and often require altering incentives and expectations of teachers. (note: decision on these interventions are made at system or possibly school level)
|
|
| Students [& Households] | Student-level interventions seek to stimulate and impact the demand for education.
|
|
| Related Interventions, Outside of the Education System (Particularly Relevant for TVET) | ||
| Labor Market | A broad range of measures in the labor market can aim to improve prospects for TVET graduates, provide information to inform youths’ training and career decisions, and improve the overall efficiency of the labor market.
|
Morocco II (2016 – 2023): (1) Strengthen national Labor Market Observatory to improve policy coordination and promote joint activities among relevant Government agencies for labor market analysis; develop and rollout a related information dissemination platform. (2) Use results-based financing mechanism to replicate and expand proven programs providing integrated job placement services that targeted women and at-risk urban and peri-urban youth that were unemployed or outside the labor market. |
| Employers [public or private] | Interventions are designed to directly involve employers in TVET course design and implementation to ensure relevance to employer needs and improve training outcomes for both employers and trainees.
|
Industry input has been provided in several countries such as Morocco II, Georgia II, Mongolia and Côte d’Ivoire to select the fields and levels of training, develop curriculum, provide trainers and in-kind donations of old machinery and equipment or land, etc. |
MCC Experience
The far-right column in Table 2 provides examples of MCC education investments for each level of intervention described above, starting with the system level. Each entry provides the country name, years of the program, and key outputs that were completed or are in progress, depending on the stage of implementation. This is not an exhaustive list, but it does demonstrate where MCC’s efforts have been focused and where they may be headed. For example, earlier programs focused heavily on building new and rehabilitating existing school infrastructure. The recently published MCC assessment “Insights from General Education Evaluations” (September 2022) summarizes the agency’s earlier approach as follows:
Many of the early education projects for which MCC has evaluation results, tended to focus primarily on school infrastructure and textbooks as opposed to issues such as governance and policy reforms necessary to improve the learning curve. Additionally, MCC invested in wide ranging programs that tried to tackle numerous problems within the education system in a partner country, which spread limited resources over too many activities and made them difficult to implement.
Newer programs may still include ‘traditional’ investments like school infrastructure but typically will also have PIR components closely tied to the investments in infrastructure, with the aim of supporting sustainability. MCC continues to work towards finding the right balance between a significant investment that addresses identified constraints in order to initiate change or provide a demonstration effect, while also being implementable within five-years, cost-effective, evaluable, and having significant country ownership. MCC priorities also include designing interventions that support inclusion, climate change, and strengthen private sector engagement. These aspects and learning from our existing programs will continue to shape future MCC investments in education.
Experience with TVET
Between 2008 and 2014, MCC invested over $148 million in TVET programs. In the earlier ‘First Generation’ MCC TVET investments, such as El Salvador I, Namibia, and Morocco I, there was a predominant focus on programs to provide skills for out of school youth or disadvantaged populations, that is, the second of the three TVET types described above. However, this was not an exclusive focus, as some in-school TVET and in-service training were also included in these earlier compacts. As detailed in the ‘Principles into Practice’ report on MCC’s TVET experience, independent evaluations from El Salvador I, Morocco I, and Namibia revealed that the programs achieved many output targets but failed to achieve the intended labor market outcomes. In this respect, MCC’s results were similar to broader experiences with TVET. Weak TVET service provider accountability, especially to firms, and curricula that did not meet the needs of the private sector were identified as key factors behind the disappointing labor market outcomes. This also was in line with broader experience: thinking in the field was coalescing around the idea that failure to design programs that meet employer needs–and to work closely with employers to identify these needs and design appropriate curricula–lay behind the weak performance of TVET.
“Second Generation” MCC TVET investments have tried to address these shortcomings, by enhancing efforts to ensure provider accountability and tighter linkages to employers and the private sector — in other words, a model of TVET that is industry- or demand-driven. Compacts in Georgia, Morocco, and El Salvador (all second compacts) and Côte d’Ivoire include mechanisms to continuously involve the private sector or employers in TVET design and operation, internships or apprenticeships with employers to supplement classroom training, and various activities targeting TVET sector management, governance, or accountability. Given this industry focus, the newer programs tend to be more oriented to in-school, pre-labor force entry TVET programming (TVET Type 1) or to continuous training for current employees in targeted sectors or fields (Type 3), rather than remedial or general skills programs for say, low income out of school youth. Actually, the new programs tend to combine elements of the first and third TVET models, with the common ingredient being the strong employer or demand led orientation and employer participation. Thus, for Georgia II the selected centers are in STEM related fields and were designed for individuals with significant schooling (they were level IV or V out of V). Most participants have at least several years of work experience, so the interventions are not designed (only) for individuals who are in the pre-labor force entry phase. On the other hand, the employed participants may either be in the targeted fields or may enter from a different field, so the scope is broader than standard in service training provided by a firm to its employees.
Despite this new focus, some recent compacts do still feature Type 2 TVET interventions. For example, Morocco II’s Results‐based Financing (RBF) for Inclusive Employment sub-Activity aim to replicate and expand proven programs to provide integrated job placement services targeted at women, at-risk urban and peri-urban youth that are unemployed or not in the labor force.
Grant funding mechanism: Also reflecting a greater focus on private sector and industry demands, several recent MCC compacts—including Côte d’Ivoire, Georgia II, and Morocco II, and somewhat earlier, Namibia—have employed a competitive grants program to select who receives funding and technical assistance to support existing and/or new TVET centers. Explicit industry participation (e.g., sector committees/associations, consortium) is a requirement for potential grantees in these schemes, usually requiring them to provide a minimum financial contribution and include some form of a public private partnership (PPP) agreement. The grants are variably used for pre-labor force entry training for youth, training for those switching careers, or for in-service training for individuals already employed in the industry. This funding mechanism has several advantages, such as flexibility, responsiveness to industry demand, and a strong private sector stake as the industry shares the costs and can directly benefit from having a better skilled labor force. However, MCC has also experienced challenges during implementation, particularly where sector associations are weaker, and in conducting the requisite CBA. Estimating the TVET activity level ERR involves conducting separate CBAs for each of the awarded projects, and then appropriately aggregating. This requires cost and expected benefit data for various centers (and typically covers a variety of economic sectors), and timing of awards have not met MCC’s typical compact investment decision timelines. The latter would also be true for results-based financing or other innovative funding methods, but MCC has had limited experience with RBF to date (see Morocco II RBF example in Table 2). The issue of CBA for programs with grants facilities is discussed further below in Section III.
C. Project Logics
A common practice in project design, at MCC and elsewhere, is to develop a project’s logic or a theory of change that summarizes the problem(s) intending to be addressed, the targeted population(s), and the proposed logic chain of how project inputs will lead to outputs, then outcomes – often with a distinction between short-term and medium-term outcomes, and ultimately, final impacts. Underlying the project logic (if not always explicitly noted) is the overarching aim of designing an investment that ultimately supports poverty reduction through economic growth – i.e., MCC’s mission.
This work on developing project logics is typically completed through a collaborative project team effort, often involving key stakeholders (e.g., implementers, donors, target population) and producing a simplified summary diagram that aims to highlight how the project’s implementation will lead to the intended impacts. The project logic authors would also specify the assumptions and risks along each causal chain, specifically where outcomes depend on a change in behavior, co-funding, PIR, or buy-in from certain groups. Identifying these early on can mitigate risks by incorporating activities that ensure these requirements are met, or if not, support learning about them by clearly specifying testable assumptions about necessary complementary factors.
Overall, the exercise to jointly produce a project logic helps to ensure strong project management, the design of appropriate monitoring strategies that keep the project on track to meeting the objectives, allow for adjusting the design as needed during implementation, and provides information for accountability and learning. At MCC, monitoring and evaluation (M&E) are required for all projects, and the economists work closely with M&E leads to develop CBA models that are aligned with, and do not contradict, the project logic. As noted earlier, the economists are responsible for the economic analysis (CBA) at various points within project design, as well as during and after implementation.
Figure 1 provides a high-level or general project logic for education and training projects. The intention here is to give a sense for the structure and key components of an education or training project logic. It is not intended as a template for a given project, as logics are unique for each intervention and, therefore, tailored to capture the detailed characteristics of that intervention. The first two steps in the project logic – specification of the problem and of the potential interventions to address it—have already been described in detail above in Sections II.A and II.B and comprise the first two columns of the figure.
Subsequent steps in the project logic include the expected outcomes, distinguishing between those that are anticipated to occur in the short, medium and longer term, with the latter often referred to as the eventual impacts or even the potential benefits. These components, highlighted in bold text in the figure, will be discussed in more detail in the next section, where we link outcomes in the logic with quantified and monetized benefit streams for the CBA.
As the figure indicates, there may be interdependencies between the outcomes of a project. In particular, higher quality of schooling as a result of an intervention is also expected to induce greater enrollment and attainment. This of course just mirrors the linkages of the key problems noted earlier, with low education quality having the potential to lead to low quantity (enrollment or attainment).
Finally, the bottom of the figure (below the dotted line, boxes colored in light grey) depicts education system focused interventions (introduction of management information systems, international student assessments, improvements in planning, etc.). These interventions are listed as addressing the problem of overall weaknesses in the education system, though it may be more appropriate to view this as a root cause to one or more of the other core problems listed in the first column and described in Table 1, Section II.A. The aim of these interventions is to create a more efficient education system, which could occur through cost savings that would make possible an increase in overall investments in needed education interventions, in using data to support a better allocation of limited resources where greater outcomes can be realized, etc. In many cases, the improvements would result in the provision of more education inputs like those listed under the column of example interventions (inputs/outputs).
Figure 1: High-Level/General Project Logic for an Education or Training Program
III. ECONOMIC ANALYSIS OF EDUCATION AND TRAINING INTERVENTIONS
Cost benefit analysis (CBA), sometimes referred to as benefit-cost analysis, is a type of economic analysis that is conducted to determine if an investment is socially efficient, and with the purpose of being used to inform decision making. The overarching methodology is straightforward, but notable challenges exist in the exercise of estimating costs and benefits, and bringing all the necessary elements together to determine the resulting key metrics of interest. Before getting into the details of CBA for education and training interventions, this section’s introduction serves to summarize foundational aspects of CBA to support its application to the sector.
There are several types of analysis that aim to consider the use of scarce resources, and the opportunity cost of investing in a particular intervention. A brief description follows to frame the role of CBA and its linkages to related types of analysis, but the comparative strengths and weaknesses of each type is not elaborated here. To start, an initial step in project design is often to complete a cost-feasibility analysis to compare an intervention’s initial cost estimates against the available budget envelope (typically a range). This will support a project team in focusing their intervention design efforts – i.e., develop a project logic and potentially conduct a CBA– on the set of feasible alternatives. The cost analysis carried out as part of this exercise can also inform initial versions of a project’s CBA and cost-effectiveness analysis (CEA). As noted earlier, CEA is a related type of analysis that estimates the cost of obtaining an educational result that is quantified (e.g., learning, or graduation rates), but not monetized.
Both CBA and CEA are used ex ante to inform project design and decision making, but can also be done during implementation, and after project completion. These two types of analysis rely on relevant literature to determine an appropriate ex ante effectiveness estimate for an intervention and both could update the ex post value based on results from a specific intervention’s evaluation. However, the CBA would not stop there as its overall goal is to determine whether the project was a socially efficient use of funding. Therefore, the metrics reported from these analyses provide distinct information that can be brought together to tell a more complete story about the expected results of an investment. At MCC, the focus has been on cost-feasibility, CBA, and independent project evaluations, but not on CEA. While these and other types of analysis can work together, it is important to understand and capture the strengths, weaknesses, and limitations of each when using them to inform decision making.
What do we mean by the phrase ‘socially efficient’? Simply, that the costs of a given intervention are justified by the benefits generated. The question then arises, benefits for whom? As discussed earlier in the paper, understanding the population of interest is an important first step in appropriately framing the perspective of the analysis. This effort determines who has ‘standing’ within the CBA model, and this should be explicitly stated and justified. As CBA seeks to support decisions that maximize overall social welfare, the default in CBA is to consider society overall as the perspective of interest. For MCC’s analysis, society overall refers to the population of the county; benefits outside of the country, if there are any, are normally not included. Therefore, the metrics from CBA represent an analysis that has considered the full social costs and social benefits of an intervention to the country. Even when this is the case, it is useful to examine costs and benefits at the level of private individuals – both project participants and the general public, government, and then aggregated for the society overall. Furthermore, this work helps to inform the selection of groups for which to examine their costs and benefits separately to better understand the proposed project’s distributional impacts—e.g., the impacts by household income, gender, regions, etc.
When conducting a CBA, the analyst should take an organized approach to ensure all key steps are completed. This SCDP highlights 5 somewhat simplified steps to guide the discussion that follows.
- Develop a program logic that outlines the problem an intervention aims to address, and the relationship between the intervention’s intended inputs, outputs, and outcomes to achieve the stated objective of the investment. This was the focus of the previous section of the paper.
- Develop an economic logic to underpin the CBA model, ensuring this reflects the program’s logic. In simple terms, the economic logic is a summary of the CBA model’s framework. An economic logic uses a well-defined counterfactual to capture the program logic’s outputs and outcomes as the CBA model’s intended benefit streams, with note on how these change across time – considering when benefits may begin to fade out, length that they will persist, etc. It aims to summarize the key decisions within the CBA model’s framework by acknowledging who has standing, the counterfactual, time horizon, discount rate, and notable assumptions and risks that could influence the project in meeting its objective and that cause uncertainty in the CBA results. While costs are not depicted in the program logic, the economic logic would typically outline the key cost categories required to obtain the intended benefit streams. Therefore, while the program logic and economic logic should not contradict one another, they serve different roles in the project design process. At MCC, CBA should be conducted at the lowest possible level of disaggregation, in accordance with the program logic, and when it is feasible and cost effective (for MCC), considering when project components may make sense to group together based on their level of complementarity or joint necessity – another aspect of the economic logic to highlight.
- Identify, quantify, and monetize all project-related social benefits, adjusting to reflect estimates as present values. In these steps, the analysis moves beyond the outcomes listed in the program logic to define them as potential benefit streams, determining how to measure and monetize (if possible) them for inclusion in the CBA model.
- Identify, quantify, and monetize all project-related social costs, adjusting to reflect estimates as present values. All costs required to obtain the intended benefit streams would be included in the CBA model, regardless of who provides the funding, and following the same expected time horizon laid out in the economic logic.
- Calculate metrics that summarize the results of the CBA, conduct sensitivity analysis to test the robustness of the results, and carry out distributional analysis to inform decision making. The key statistic summarizing the benefits relative to costs of an intervention over the project lifetime is the economic rate of return (ERR), use of which enables comparison of different projects as well as with the ‘hurdle rate’ or MCC’s required 10% ERR threshold. Sensitivity analysis captures the robustness of this result to uncertainty about key parameter assumptions, while distributional analysis moves beyond the economy-wide or aggregate result to assess how benefits are distributed across different groups in the population (capturing among other outcomes the potential for poverty reduction impacts).
With these general steps to conducting a CBA in mind, the paper shifts to focus on the application of the methodology for education and training projects, with a particular emphasis on the MCC experience. To simplify the remainder of the paper, the focus is on three main interventions that have comprised the bulk of MCC funding in education and training, the first two are specific to general education and the last to TVET, which encompasses a wide range of investments: (1) School Infrastructure; (2) Teacher Training and (3) TVET. While this appears to omit many other types of interventions mentioned in the previous section, there are broader insights provided through the three examples. And, even with this narrowed focus, it will not be possible to go into significant detail on all aspects of CBA covered in this section. The aim will be to incorporate additional intervention examples as well as references to specific CBA models that are forthcoming on MCC’s external website in a future version of the guidance – as outlined in the final component of this section.
Section III.A provides a brief description of key elements within the economic logic and therefore the typical framework of CBA models in education and training. Section III.B focusses on the work of identifying, quantifying and monetizing benefit streams. The following Section III.C outlines the main cost considerations to ensure that inclusion of all social costs required to achieve the previously outlined benefits. Section III.D brings together the work of the previous sub-sections (i.e., framework, benefits, and costs) to determine the CBA results, with consideration of sensitivity analysis that can help to inform the CBA metrics reported, and distributional analysis that can highlight results by important project beneficiary groups. Finally, Section III.E concludes by summarizing the thematic areas that MCC can expand upon in future versions of the SCDP in education and training.
A. Economic Logic and CBA Framework of Education and Training Projects
Building off the program logic, the analyst is responsible for taking an economic lens to determine how the paths to reaching intended outcomes will be represented as costs and benefit streams within the CBA model. The overarching work to develop the CBA framework is the focus of this subsection, which is organized around defining and discussing the important considerations for 3 key elements: counterfactual, time horizon, and discount rate. This supports framing the later discussions on benefits and costs (Section III. A and B, respectively), which are also part of the economic logic, and bringing all these components together to report on the CBA results.
Counterfactual
A critical step to CBA is to define an appropriate and justifiable counterfactual: what would have happened in the absence of such an investment. Both project costs and benefits are measuring against the counterfactual, which is not static – that is, it requires consideration of changes across time. Often, the counterfactual may be thought of as comparing the MCC-supported investment scenario to a “business as usual” scenario, where the items or inputs provided under the project would not otherwise be provided. Defining the counterfactual requires the analyst to have a strong understanding of the context for the country, sector, general trends, market or government failures, existing or planned efforts to resolve the problem at hand, etc. Helping with this is the fact that much is learned during the CA and RCA phases of compact development, as well as through partnerships with our country counterparts and key stakeholders. In fact, MCC’s model facilitates collaboration across team members with technical expertise in certain sectors, countries, and methodological areas to ensure that the analyst is making informed decisions throughout the CBA process.
For projects in education and training, useful information could include that on demographics, years of schooling completed, annual rates of schooling (i.e., enrollment, promotion, repetition, and dropout), literacy rates, results of national, regional, or international exams, labor force statistics, and spending on education. As programs are defined more narrowly, the data obtained will also need to become more specific to the intervention. The analyst would begin by examining indicators at a national level and then, where possible or applicable, disaggregate by relevant groups such as geographic region (e.g., state/county, urban/rural), household income (e.g., quartiles or quintiles), male/female, age groups, ethnicities, native language, etc. or even by different levels of education. The time component is also important to consider. With respect to establishing trends, the analyst seeks to obtain the most recent data available, as well as any historical data, and hold discussions with informed stakeholders to understand future investment planning that could impact indicators of interest. It should be clear, but worth saying, that the data quality should also be assessed as the analyst defines the counterfactual.
Table 3 below outlines potential types of data that can be used to inform the counterfactual for education and training interventions. In most cases, this is also helpful for defining the with-project scenario, as it supports reasonable expectations of benefits based on the current situation and trends. This Table intends to provide a sense for how data could be used but does not capture the complexity of defining a specific intervention’s counterfactual. As noted above, the counterfactual is the basis for measuring both the benefits and costs, so further considerations for defining an appropriate counterfactual will be discussed in the next two subsections (Section III.B and III.C), particularly through the three example interventions provided when discussing benefits.
Table 3: Data for Defining Counterfactuals for Education and Training Interventions
| Indicators | Description of Indicators | Potential Uses for Defining a Counterfactual |
|---|---|---|
| Demographic trends | Birth rates, fertility rates, life expectancy, mortality, migration patterns, population, etc. |
|
| Average school completion | Average years of schooling, or annual completion rates |
|
| School Progression: Enrollment, promotion, repetition, and dropout rates | Enrollment, promotion, repetition, graduation, and dropout rates provided on an annual basis reported by grades and education levels |
|
| Literacy rate | Percent of population aged 15 years and older who can read and write; Examining by age provides a sense for how obtaining literacy skills has changed across time. |
|
| Indicators of Schooling Quality | Results of standardized math and language exams (national, regional, or international assessments); Results of focused subject matter exams |
|
| Labor force Statistics and Wages/Salaries | Labor force participation, employment, and unemployment rates; Composition of workers across economic sectors or fields of studies; wages and salaries |
|
| Social, public and private, spending on education | Total education spending in terms of percent of GDP, percent of total government spending, and total values in real terms, allocation of funding within education – at point in time and across time |
|
The description above has given a sense for the complexity in defining and justifying a counterfactual, particularly given that we cannot see into the future and data is often limited in availability and quality. As such, it is important to document counterfactual-related decisions for transparency. This can also support future updates to the existing counterfactual or applying lessons learned from that experience to improve defining counterfactuals for similar projects in the future. The subsections that follow will incorporate counterfactual considerations when discussing the benefits and costs.
Time Horizon
Another key aspect of the CBA framework is to determine the appropriate time horizon. The default time horizon for MCC investments is 20 years, after the investment is completed and the output is ready for use. For example, the clock would begin after a school is built and teachers can begin instructing students there, and the school building would have a 20-year life expectancy, if properly maintained. Regardless of the time horizon chosen, the decision should be informed by expectations of the investment’s sustainability, considering the likelihood that it is properly maintained, the projected time from investment completion to when benefits begin to be realized, and how those benefits are anticipated to behave across time (e.g., Will they decrease after a certain period? Will they tend towards zero or what other steady state value?), incorporating any associated costs required to reach the included benefits.
MCC has generally adopted the default time horizon for education and training investments, but with one important caveat: CBA models include all student cohorts that would complete their education or training within those 20 years (e.g., during the expected life of a newly built school), but it also follows each of those cohorts for at least 20 years after their training is completed. Therefore, the total years included within an education and training CBA model will typically appear as a total of 40 years. This practice has been adopted to more accurately capture the long-term benefits associated with the provision of education and realized during an individual’s working lifetime. While theory and empirical evidence suggest that this concept is generally agreed upon, the length and strength of these benefits across time can vary significantly by intervention, so these assumptions should be adjusted based on available literature.
As mentioned above, the analyst has flexibility to adjust the time horizon based on the specific investment. There are two noteworthy considerations for education and training projects. First, on the topic just discussed, is the determination of how long one should track each cohort. An analyst could make a reasonable argument to extend the tracking of cohorts from 20 years, even up to the number of years until the student cohort reaches retirement age. This would capture the benefit streams for their entire working lifetime. Analysts can make this decision for the CBA model or incorporate this within their sensitivity analysis. However, following each cohort for 20 years has been selected for simplicity, remaining close to the default at MCC, and because benefits far into the future will be more heavily discounted and, therefore, less likely to impact the results of the CBA.
The second consideration for time horizon adjustments is for typically considered for infrastructure investments, but applies to other interventions as well, such as teacher training. When the likelihood of investment maintenance appears to be low, the analyst could adjust the CBA model in a few different ways (which are not mutually exclusive): (1) use a time horizon of less than 20 years, say 10 years as an example; (2) keep the time horizon at 20 years but incorporate a reduction in benefits for cohorts that enter past year 10 (as an example), due to the deterioration of the asset and therefore the resulting benefits; or (3) keep the time horizon at 20 years but use the other two scenarios in the sensitivity analysis to demonstrate the potential differences in efficiency under such circumstances. Consultations with MCC infrastructure experts and local building engineers can help determine the appropriate assumptions regarding likely maintenance behavior and asset life under different scenarios.
Discount Rate
Another decision closely related to time horizon is the selection of an appropriate discount rate. This rate, similar to a weight, is applied to all costs and benefits to put them into equal terms across time, allowing for aggregation to estimate the present value of net benefits. The discount rate, as the name implies, assigns a lower weight to costs and benefits occurring further in the future. There are two rationales for using a discount rate: (1) to reflect the opportunity cost of investment resources on a given project rather than investing them elsewhere (the cost of capital perspective) and (2) to reflect the time preference of individuals (or society in general) to consume today rather than wait to consume in the future, resulting in a reduction in the value of consumption the further it occurs in the future.
While the use of discounting is uncontested among cost-benefit analysis practitioners, there continues to be debate on the specific discount rate to apply when assessing a project’s efficiency. In practice, discount rates have ranged between the range of 0% – 12% are often cited as being appropriate for CBA of projects in general, with lower average rates of around 3% often used for social investments in education and health, particularly in the United States, and there has been a general trend, across countries, to revise rates downward to a range of 3% – 7%, on average or for use in sensitivity analysis. MCC’s focus on the ERR as the main CBA metric of interest to inform decision making avoids a heavy reliance on discount rate selection, as compared to those that rely on the NPV. At the same time, the current use of the 10% ERR as a threshold to inform decision making, implies the use of a constant discount rate across projects of all sizes, sectors, countries, etc. The agency has reviewed this aspect of the CBA framework against the latest research and evidence and continues to monitor the situation to inform their future discounting practices.
As discussed, discounting places a higher value on nearer term benefits, and is typically a constant ratio from year to year. The use of a higher rate, like the implicit 10% used at MCC, would (mathematically) require projects with significantly delayed (long-term) benefits to produce far greater total benefits than projects with more near-term benefits. If these results strongly influence initial decision making, then this could have important consequences for investments in education and training. Early childhood education, primary school, and perhaps even lower secondary school level education and training could have large long-term benefits, but these occur well after an initial investment so they will be heavily discounted. As an example, the intervention may occur when a child is 5 years old, while the bulk of the benefits may not be realized until 15 years later when they enter the labor market, and other benefits that happen earlier may be difficult to quantify or monetize. On the other hand, TVET investments may be focused on older youth and adults, with shorter education and training programs that can provide benefits more quickly, and therefore such investments may be perceived as more favorable, all else equal. Additionally, benefits that will occur sooner after an investment is completed are more likely to be realized than those that are expected far in the future, due to the potential for confounding or external factors to play a role in disturbing the anticipated benefit stream trajectory. Therefore, the risks to achieving benefits from the shorter-term investments may seem more enticing for those reasons as well. For such reasons, MCC has designed its investment criteria to include other important aspects that align with Agency priorities – e.g., gender and social inclusion, climate change, poverty reduction.
B. Benefits
This subsection aims to outline the work to identify, quantify and monetize potential benefit streams for inclusion within the CBA of an education and training project. The narrative highlights challenges associated with each step of the process and seeks to provide useful practices to support the development of a comprehensive, evidence-based CBA model. After introducing the key elements of the work to identify, quantify, and monetize benefit streams, three investment examples, drawing heavily from MCC experience, are provided to speak about these concepts and the counterfactual in more concrete terms. The illustrative interventions will include two for general education – school construction and teacher training, and TVET overall but with a focus on employer-driven programs.
Identify
The analyst should begin with an open mind, considering the world of potential benefit streams tied to the intervention’s design, the intended outcomes outlined in the program logic, and even those that may be unintended (externalities). This exercise can be facilitated by specifying the groups of individuals that could be directly or indirectly affected by the intervention, ensuring that their perspectives are considered. Insights can also be acquired thorough conversations with sector experts and a review of relevant literature to improve the analyst’s understanding of the core theoretical relationships along the logic chain and obtain empirical evidence on the results of similar projects. Some of this work may already have been carried out as part of the efforts to develop the project logic, but it is likely that full attention was not given to the economic factors needed to inform the CBA.
The high-level, general program logic introduced in Section II.C depicts the logic chain from problems to interventions to outcomes. Figure 2 pulls information from that image to focus on the three main medium-term outcomes that have typically become benefit streams in MCC CBA models, when they are expected as a result of the particular intervention. If these medium-term outcomes were realized, then one would expect the longer-term, labor-related outcomes to follow, defined as both improved employment – higher rates of employment and of the quality of the employment – and higher wages/salaries. Within the CBA model, all these outcomes are measured relative to a well-defined counterfactual. The three medium-term outcomes and the longer-term outcome of improved employment would need to be quantified before being monetized, so they will be covered in the quantify subsection that follows. As wages are already in monetary terms, this will be defined further in the monetization subsection, along with discussing how to monetize the potential benefits quantified within the subsection that precedes it.
Figure 2: Converting Project Logic Outcomes into Benefit Streams
As noted, a major benefit in both the theoretical and empirical literature is that additional years, better quality, and/or more relevant education and training would lead to higher earnings, on average. However, there are other positive impacts to education and training projects that should be considered, and if relevant and feasible, included in the CBA. In the early stages of the CBA process the list of potential benefits should remain broad regardless of whether they naturally lend themselves to being quantified and/or monetized. This ensures acknowledgement of a wide range of potential benefits that will be assessed in the following steps to determine whether evidence exists to justify their inclusion in the final CBA model. If not, then the best that an analyst can do is to explicitly state the expected benefits that could not be incorporated.
A few examples of non-earnings based potential benefit streams:
- An education or training intervention that targets at-risk youth could reduce their participation in crime, with associate reductions in social costs.
- School rehabilitation to replace woodstoves with central heating in schools located in colder climates could improve the air quality, resulting in respiratory health benefits for students and staff.
- Programs focused on increasing girls’ overall access to education could change their life trajectories, e.g., by delaying marriage and childbearing, and increasing their likelihood to enter the labor market. If these women become mothers, then they would be more educated and there could be intergenerational benefits associated with the improved nutrition and schooling outcomes for their children.
The analyst should review the range of potential benefits to ensure that their inclusion in the CBA model would not result in double counting or incorporate transfers. With regard to double counting, before adding all the benefits together, the analyst must make sure that all benefits are separate from one another, and any overlaps are identified and removed. Overlap can happen in practice when the analyst follows the project logic too closely, attempting to capture every element, so that an earlier outcome and later outcome may appear as separate steps along the logic chain, but for the purposes of the CBA, the later outcome would capture both elements. Including both outcomes in the CBA model as separate benefit streams would result in double counting, i.e., overstating the true benefits. Regarding transfers, these are items that would be a cost to one part of society and a benefit to another, such that the net impact is zero. A common example given is tax revenue, which is a cost to the individual but an equivalent benefit to the government. All transfers should be identified and not included in the CBA model, as either costs or benefits.
Quantify
Using the identified list of potential benefit streams (including externalities), the next step in the process is to determine which aspects can be measured in quantitative terms. In the face of limited time and resources, the analyst must strategize how to focus efforts to quantify and monetize the list of potential benefits. This can often involve prioritizing the largest anticipated benefit streams, and potentially those most valued by the decision makers. When making the decision of where to focus efforts, recall that for education and workforce development projects many important benefit streams are realized farther in the future and, despite discounting, these may remain the largest benefits to an intervention.
During this process there are likely to be three categories of potential benefits: (1) those that can be removed from further consideration relatively quickly, given that they may be intangible, difficult to quantify, or are associated with a well-known gap in the evidence literature; (2) those that could be measured, but the evidence seems sparse or of questionable quality; and (3) those that are typically measured and, as such, there is strong evidence to draw upon, even if there are uncertainties in how to apply it to the specific intervention’s context. For those that are eliminated from further consideration, the recommendation under double counting was to note these explicitly with supporting documentation used to inform decision making. This practice is particularly important for sectors or types of projects that theoretically have notable benefits (e.g., policy and institutional reforms), but for which the empirical evidence does not exist, or the details of the project are limited such that the analyst cannot produce a reliable estimate, or even a range that can be used for sensitivity analysis. Some potential benefits may be quantifiable but are removed later when it becomes apparent that they cannot be monetized, as will be discussed after the quantification of benefit streams.
The remainder of this subsection examines each one of the three medium-term benefits and the longer-term improved employment outcome separately – those introduced in the identify subsection. The description of each outcome begins with examples of interventions that are often expected to achieve that outcome, an explanation of how to measure the outcome, including potential data sources, and insights on how to use the available data within the CBA model. As noted throughout the paper, the analyst must use the project’s design and the counterfactual in the given context to further define and justify the parameter decisions for inclusion in the CBA model. The example interventions included after the monetization subsection will draw upon specific literature and provide insights on plausible parameter estimates to quantify the benefits.
Quantifying Quantity of Schooling Increases (Medium-term Outcome)
Change in the quantity of schooling is the first of the three medium-term outcomes to explore as a potential benefit stream. Gains in years of schooling or additional levels of schooling are most often associated with building new learning centers that are closer to students, often offering a new level of education or fields of training that were previously unavailable or the new learning center reduces the total costs incurred by individuals to reach similar, existing facilities. On the demand-side for education, interventions that provide scholarships, stipends, transportation, or even food rations for attending local schools could be sufficient to offset opportunity costs that families experience when their children attend school. These types of interventions, and perhaps information campaigns in communities to promote the value of education, could result in higher enrollment rates, lower dropout rates, and potentially higher completion or graduation rates.
When interventions, like those mentioned above, are targeted to impact certain groups such as low-income or rural areas, or to increase the participation of at-risk youth or women, there could be greater impacts in reducing poverty and/or inequality. This summary has not provided an exhaustive list of interventions that could result in additional years of schooling, but it gives a sense for when this benefit stream may be a reasonable expectation. The determination of whether the expected change is a year or years of additional schooling, or a higher completed school level, will depend on the design of the intervention. This is important to clarify as this would impact the quantification and monetization of this potential benefit stream, which will be discussed more below.
Regarding data on these outcomes, years of schooling can most easily be measured by examining school administrative data contained in a country’s educational management and information system (EMIS). national governments often support schools in collecting annual information on the number of students that enroll, dropout, repeat, complete, graduate, and are promoted. Together, this information can indicate students’ or cohorts’ status within the general education school system, and tracking across multiple years can inform trends in obtaining additional years and levels of education.
As many low-income countries do not have EMIS or their existing EMIS are limited by the availability or quality of the data to reflect the reality throughout the country, other sources of data may be required to obtain indicators related to years of schooling. Even when EMIS data exist, a country may not assign a unique identifier to students, meaning that they cannot be tracked through the system to learn about educational pathways, particularly when students move from one school to another. Due to these challenges, MCC has in some countries collected information related to quantity of education through independent project evaluations. This would be based on an evaluation design focused on learning about the outcomes specific to the target population, and the comparison group (if relevant). Depending on the strategy, this could involve data collection at multiple points in time to inform the related baseline, midline, and/or the endline study.
While EMIS would provide data on general education, other sources would be required to inform the CBA of interventions in preschool, TVET, or higher/tertiary education. For TVET, individual centers or national coordination across centers may operate to develop tracer systems that would similarly track students that enroll, dropout, and graduate from TVET programs. Typically, these systems also include data on trainees collected shortly after their training is completed to better understand their labor market outcomes – a key component that is missing from general education EMIS datasets but may be picked up in national household surveys, another data source to consider. There will be differences by country on the systems available to obtain comprehensive data on preschool and higher education, so the analyst may need to work with education providers to compile similar data.
As mentioned just above, insights on years of schooling can also be drawn from representative national surveys, typically labor force surveys or household surveys (e.g., using LSMS – living standards measurement study methodology). Their related data collection tools would ask all individuals within a household their age, whether they are currently enrolled in school, the year/level of study that is ongoing or the highest year/level that they have completed, and perhaps why they stopped their education. While many surveys ask respondents for both the years of schooling completed and the highest level, others record only the level. If only level data are available, this would limit the analysis that can be done for the CBA model. Additionally, while the administrative education data would provide indicators for a census of students in a given year and across years, a national survey would typically be conducted at most only every five years, limiting the ability to measure trends. The design and sample size may also not may not be such as to generate representative data in rural regions or for specific groups which may be the focus of the intervention.
With these data sources in mind, how is the information used for the CBA? Essentially, for the counterfactual, information on levels and trends in the outcomes from the sources just noted are used to estimate the trajectories of these outcomes under the without-project scenario. This data can be supplemented with discussions with local experts, who can give insights into existing and forward-looking plans for programming and funding, which will help sharpen the counterfactual estimate. For example, information about major current or planned initiatives of the government in education may imply that future levels or trends in say, secondary enrollment, will diverge from past patterns, so that the counterfactual needs to be adjusted. Examining EMIS and tracer system data during compact implementation and, more notably, after compact closure can support updates to the counterfactual and the expected results for the with and without project scenarios, particularly when a strong comparison group has been established through the evaluation strategy.
There remains, of course, the need to estimate ex ante the with-project parameter estimates for the CBA: how will levels or trends in the outcomes change with the project? For this, the analyst must turn to literature relevant to the specific intervention’s design and the country context. Below, the subsection ‘In Practice: School Construction’ provides a concrete example by outlining and pulling together the concepts of the counterfactual, identifying, quantifying, and monetizing benefit streams, where the main benefit stream is expected to occur through additional years of schooling.
Quantifying Quality of Schooling Improvements (Medium-Term Outcome)
The second potential benefit stream highlighted within the document occurs through an improvement in the quality of schooling that students receive, or put another way, a change such that students learn more, on average, in the same number of years or when completing the same level of education or training. These benefits are often anticipated to result directly from interventions that develop or update school or TVET program curricula. This may involve bringing them up to a certain standard associated with becoming accredited or meeting an international certification, particularly in the case of TVET or higher education. Curriculum improvement interventions are often coupled with efforts to provide new or improved instructional materials, school supplies, and/or technical equipment (e.g., computers, white boards), as well as training of educators and school administrators in how to teach the new/revised curriculum. While combining interventions in this way could lead to greater quality improvements or even be necessary for any impact, in other contexts it could be effective to provide standalone interventions.
It is also worth mentioning more indirect interventions that could improve the quality of education. Scholarships and stipends could mean that students do not have to work while in school, leading to more hours dedicated to their studies or less school absence, or the funding could provide opportunities to attend higher quality schools. There are also types of health and nutrition interventions at the household or community level to improve early childhood development, leading to improved student learning outcomes. Of particular relevance to education are projects that reduce the prevalence of stunting, which can result in lifetime benefits associated with the significant gains in an individual’s cognitive abilities. Finally, there are PIR interventions that can improve quality or learning in a variety of ways. These may include changes in incentives and human resource policies that induce more effort from teachers or allow them to work more effectively (including class size reduction); changes in the overall efficiency in the education system such that there is more funding available to invest in quality improvement interventions; or the building of EMIS or the implementation of national assessments to obtain more information for decision making about the allocation of education resources. As with the years of schooling description, this summary has not provided an exhaustive list of all possible projects to improve the quality of education and training but should give the reader a better sense for potential interventions that could lead to this medium-term outcome.
How are quality improvements in education measured? Quality is assumed to impact learning outcomes, and these are typically measured through the results of standardized tests or other means to assess students on changes in knowledge and its application, as it relates to the level of education and the specific school subject. Exam results are typically converted to a standardized score that can facilitate comparing scores across individuals and even time, as relevant, speaking in terms of standard deviation (SD) improvements. As with the quantity of schooling, current levels or trends in these measures can be used to generate the counterfactual for learning outcomes, and relevant impact evaluation literature, can inform expected outcomes under the with-project scenario. Improvements in learning outcomes can differ greatly by interventions and even the education level of the same type of intervention, so caution must be taken when using studies to inform CBA parameter estimates. Below, the subsection ‘In Practice: Teacher Training’ provides a concrete example by illustrating the concepts of the counterfactual, identifying, quantifying, and monetizing benefit streams, where the main benefit stream is expected to occur through improved learning outcomes, leading ultimately to improved labor market outcomes.
It can be helpful to provide an example project that includes two activities that work together to the same outcome, say quality as measured by standardized tests (in standard deviations, SD). The literature may suggest that the new textbooks/didactic learning materials/computer-assisted learning being planned for the project could reasonably be expected to produce an impact of 0.2 SD improvements in test scores. This would be an excellent outcome. Say the project also includes a teacher training activity, for which the literature on interventions similar to that being designed suggests that a well-implemented project could also result in a 0.2 SD improvement in test scores. The analyst may be inclined to add the two SDs together and say that the project overall could lead to a total 0.4 SD, but it is not that simple. Few studies have found such a large impact from any intervention or coupling of interventions, suggesting that there is an upper limit—or at least, diminishing marginal returns—to combined interventions. This suggests the need to be conservative when assigning benefits in such cases to ensure the expectations accord reasonably with prior experience. Sensitivity analysis should also be deployed to provide more clarity on how the net benefits and ERRs would change based on movement in this critical outcome variable.
Quantifying Labor Market Relevance (Medium-Term Outcome)
The third main benefit stream is an improvement in the labor market relevance of the education and skills obtained – that is, greater alignment between skills obtained and those demanded by employers. As discussed earlier, this is a focus of TVET programs and higher education but is also obviously a key objective of general education in broader terms. A range of education and training program components are designed to ensure the provision of labor market relevant knowledge and skills and in the case of TVET, generally involve links with employers or the labor market. MCC’s most recent TVET investments include employer participation in various ways such as developing curriculum, having professionals serve as trainers, and supporting the requirement for trainees to obtain direct workplace experience (e.g., via internships) as a complement to classroom instruction. Some centers have also sought to develop programs that will offer occupational certifications that can be recognized nationally, regionally, and even internationally. At the system level, countries may also engage in initiatives to develop or improve upon a national labor observatory or labor market information system to inform the education path choices of students, and the program offerings of education and training centers.
Measuring ‘relevance’ of an individual’s education or training is not as straightforward as measuring the other medium-term outcomes, ‘quantity’ or ‘quality’, for which we have indicators such as years or levels of schooling or scores on standardized tests. Instead, relevance has to be defined with respect to something—namely, the needs of the labor market. If, through the constraints analysis or employer interviews or surveys, skills needs can be clearly identified then one could say (with a bit of circularity) that graduates of a program that provides those specific skills have ‘relevant’ skills. The increase in the outcome ‘relevance’ would be measured by the number of individuals obtaining the skills (and perhaps, being certified in them using international standards) and obtaining employment in their fields of study, relative to the counterfactual. The latter measure of employment is, together with earnings, the long term outcome defined above, so essentially TVET CBA focuses directly on the long term benefits : Are graduates obtaining employment—especially in the targeted occupation or industry, if the training has such a focus—and are their earnings higher? This will be discussed directly below in quantifying employment outcomes, and then in monetizing wages, with more concrete programmatic examples included ‘In Practice: TVET’.
Quantifying Employment Outcomes (Longer-term Outcome)
The longer-term outcomes are those experienced in the labor market, including employment and earnings. Since ‘earnings’ is the key element of monetization, we leave that to the next section and focus here on employment – the final of the main components to quantify. Impacts on employment outcomes can occur through the three main medium-term outcomes but, as will be described below, literature and practice suggest a stronger connection with quantity increases in schooling and market relevance, than quality of schooling. Before diving into those details, it seems useful to define employment outcomes more completely and how to measure them, including data sources. To start, we review some basic concepts about employment categories that inform how the statistics may be used for the CBA. The three categories are described below to ensure an analyst understands how to appropriately estimate related parameters.
- Labor Force Participation/in the Labor force: All persons of working age who are either employed or unemployed. The labor force participation rate is estimated as the labor force divided by the total working age population.
- Employed: Individuals of working age who are involved in paid employment or self-employment within a specified reference period. The employment rate is the total of employed individuals divided by the number of those in the labor force.
- Unemployed: Individuals of working age who were not in paid employment or self-employment during the reference period, but are currently available to work and seeking work. The unemployment rate is calculated by dividing the total number of unemployed individuals by the number of those in the labor force.
With these high-level labor categories defined, the next step is to clarify what is specifically meant by ‘improved employment outcomes.’ Figure 2 states that beneficiaries experience improved employment outcomes, with a few notations in parentheses: higher labor force participation and employment rates, and more employment in formal sector or with benefits. The first two aspects reference the high-level labor categories introduced above. Higher rates of both labor force participation and employment would indicate that individuals are moving from a non-income earning category to an income-earning category. With respect to the counterfactual scenario, those that would have been outside of the labor force would enter, and those that would have been unemployed or have now entered the labor force would become employed. This would clearly not be a 100% shift, so the analyst would need to determine what percentage would be likely to shift from one group to another due to the intervention. As noted, this shift would have clear monetary gains to capture in the CBA model, as discussed in the subsection that follows.
The other aspects referenced within parentheses under improved employment outcomes in Figure 2 relate to what could be called ‘better quality employment’. There is a desire, particularly by education specialists, to move beyond simply indicating that individuals have entered the labor force or become employed, and further assess the quality of that employment. In many circumstances, the wages/salary an individual receives would reflect the higher quality of the employment. However, as stated earlier in this section, it is important to identify other potential aspects of the benefit stream that may not be captured in the simple estimations typically used. These better-quality aspects may range from more subjective attributes that are difficult to quantify to those that can be both quantified and monetized.
In developing countries, the distinction between formal and informal employment is often used as a rough proxy for considering the quality of employment. Individuals with formal employment would often be provided a contract or position that translates to greater employment stability – assurances against unlawfully being fired, predictable hours, more likely to provide a living wage, and the potential to provide benefits such as paid vacation and sick leave, medical insurance, etc. and potentially more opportunities to learn new skills and progress in a career, as employers seek to invest in their people. While some jobs are inherently riskier than others, the better-quality aspect would translate into being compensated for the tasks, considering the risks involved, and being provided appropriate training and equipment to reduce the risk and severity of negative impacts (e.g., injuries) on employees. There may also be aspects related to psychologically safe working conditions, such as protections against harassment or discrimination. While not all these benefits are guaranteed to those with formal employment, they would be more expected than for those with informal employment. This example of formal employment has sought to provide a sense for the realm of possible benefits related to better quality employment that could be further considered.
With the employment outcomes more clearly defined, we return our attention to the three medium-term outcomes described above —education quantity, quality, and relevance. This subsection proceeds by outlining when an intervention resulting in a certain medium-term outcome is expected to lead to the longer-term employment outcomes, and then shifts to considerations on how to measure to this outcome for the counterfactual and with-project scenarios.
For education quantity, additional years or levels of schooling could lead to an increase in rates of labor force participation and employment but would depend on the country context. While the relationship between years of schooling and earnings is quite clear from the literature, the relationship with employment outcomes is more nuanced. Generally speaking, highly-educated individuals in developing countries often report greater unemployment levels than those with lower education. This is thought to be due primarily to them having greater freedom in choosing whether to participate in the labor force, and then whether to accept a job. That is, greater levels of education tend to be associated with higher socioeconomic status; meaning that highly-educated people are more likely to be able to afford not working than people with lower socioeconomic status – i.e., enjoy a higher reservation wage. The level of education will therefore be important, but also by type of education. General education interventions do not typically include a focus on employment outcomes, so the intention to reach this objective would either need to part of the program logic or would occur only through a transition to a grade/level of education that is associated with higher employment outcomes, on average. In this situation there is no indication that there would be increases in ‘better quality employment’ to consider. TVET is clearly a different case, which will be summarized after discussing the three medium-term outcomes.
The second medium-term outcome is improvements in education and training quality. As discussed below within ‘In Practice: Teacher Training’, the best empirical evidence available demonstrates a clear relationship between education quality, as measured for general education by SD improvements, and increased wages or lifetime earnings (Chetty et al. 2014a and 2014b), but it does not specify a linkage with employment outcomes. While a positive effect on employment outcomes such as obtaining a formal sector job are quite plausible, the lack of direct evidence to support this linkage means that it is often omitted in CBA for these types of investments. Instead, it is more practical to draw on the available evidence to generate expected changes in earnings from quality improvements, which would implicitly incorporate any intermediate steps through employment outcomes.
The final medium-term outcome is increased relevance of the education or training. This is a particular objective of MCC’s demand-driven TVET programs, and here interventions are often directly geared toward specific employment outcomes, namely, obtaining work in a given field or occupation. Additionally, shifts may be expected in graduates obtaining better quality employment as well, but this would depend on the focus of the training centers. For example, if the center aims to support a sector that has high levels of informality, and there is extensive outreach to recruit individuals currently working in informal employment with the aim to train them for formal employment, then additional benefit streams should be considered. This practice has not been fully adopted by MCC but is being explored. TVET will be discussed more fully in the subsection ‘In Practice: TVET’ below.
The next step in the process is to determine the data available and how to measure employment outcomes, both in the with and without project scenarios. For general education interventions that aim to increase the quantity of schooling, national household or labor force survey data provide the most common way of generating expected employment status estimates. A national survey would typically have a module dedicated to understanding each household member’s (typically above the age of 14 or 16) role in the labor market, and this can be linked with the education and demographic or gender data to inform the counterfactual and obtain a sense for expected intervention outcomes for those with different characteristics. For example, if the project is expected to increase years of schooling on average by 3 years, or to improve secondary graduation rates by some percentage, the impacts on labor force participation or employment of a certain type may be obtained from econometric estimates of the determinants of these outcomes, typically using a probit model. The estimates from the model can be used to generate predicted probabilities of the outcome for individuals with the lower and higher levels of schooling or with and without a secondary diploma, controlling for other characteristics. For this analysis, one would need a recent survey and one that has the requisite education categories. While years of schooling are usually recorded sometimes only the highest level attained is recorded; or the surveys will have years of schooling, but not highest level competed.
It should also be stressed that these impacts will be both country and gender specific. For example, women typically have lower rates of labor force participation and employment in most low-income countries, which may mean that there are substantial impacts on labor force activity over their lifetime. However, country context is important here. While analysis going back decades (Psacharopoulos and Tzannatos 1989; Klasen 2020) shows a generally positive association of female education and labor force participation, there are many exceptions (see Psacharopoulos and Tzannatos 1989 and Heath and Jayachandran 2018). In countries where there are cultural, discriminatory or even legal barriers to women working, investments that raise female educational attainment may not result in higher rates of labor force participation of women.
As discussed, employment outcomes include both changes broad work status (participating, employed, unemployed) and more nuanced measures of the quality of employment. The quantification of better-quality employment would require additional work, and in some cases these benefits may be minimal if the bulk of the benefits are already captured by wage increases. Household or labor force surveys may include information on job benefits such as health insurance and leave, which in principle can be monetized when the information is available. However, information on factors such as job security, work environment, and prospects for advancement are not typically collected in household surveys (some labor force surveys may do better in this regard), making it difficult to quantify and monetize these more subjective dimensions of employment quality, and therefore complicate efforts to incorporate into CBA. Given these challenges, CBA practice at MCC and elsewhere generally does not consider these kind of job attributes as part of benefits, instead focusing on expected earnings modified by employment probabilities, as described below. But this is clearly not the whole picture, and an important area for future work is to develop approaches to gather information on and monetize a broader range of job outcomes. MCC is actively exploring and trying different techniques to consider inclusion of such benefits in some of our CBA models.
Lastly, for TVET investments, ascertaining the expected change in employment outcomes can be more challenging, primarily because existing survey data may not contain information or adequate sample size on training experience. Or (very likely) what is contained in the data does not correspond to the specific type of training program or level being considered – for either the counterfactual or with-project scenario. Another potential data source is TVET tracer studies. Such studies focus on tracking graduates of specific training centers. A key parameter of interest is referred to as the insertion rate, meaning insertion into the labor market. This can be thought of as a short-term employment rate, as it considers all graduates (akin to the working age population), whether they enter the labor market, and then whether they obtain employment. Data is typically collected within the first 12-18 months after program completion and may include several data collection efforts with a frequency of every 3-6 months. Eventually, the CBA model would shift from talking about insertion rates to employment rates, but evidence remains limited to inform assumptions about how long the employment-related benefits of the training would continue, i.e., whether it truly changes their lifetime earning trajectory. These tracer surveys may also ask questions to understand if graduates obtain better quality employment, whether they are working in the field of their studies, and their wages. The extent of the survey will determine the ease in capturing the expected total benefits to TVET graduates. Information for existing centers, and their past performance, could be useful to serve as the counterfactual, while evidence on intervention outcomes could potentially come from other country experiences. More discussion of the TVET case is provided below under ‘In Practice: TVET.
Monetize:
With potential benefit streams quantified, the next step in the process is to put them into income terms – i.e., to monetize them. As shown in the project logic of the previous section, an increase in wages is seen as a principal long-term outcome resulting from the three highlighted medium-term outcomes: additional years of schooling, improved quality in the education received, and greater relevance of skills obtained. The overall benefits for earnings and income will also work through changes in the other longer-term outcome, employment: most simply, a higher probability of gaining employment means higher expected overall income. This subsection focuses on methods for estimating wage benefits, which are clearly already monetized, and then briefly notes approaches to estimate shadow prices for other potential benefit streams that have been quantified but do not have a clear market value that can be easily assigned to the outcome of interest.
As described in the introduction of this document, an overall aim of education investments is to increase future earnings. One of the most robust associations in research on economic development is that between earnings and level of education, bearing out the prediction from human capital theory that more educated people are, on average, more productive and therefore have higher incomes. Across countries research has shown a global average of private (individual) returns to an additional year of schooling to be about 9 percent – a result that has remained stable across decades (Psacharopoulos and Patrinos, 2018). This is motiving, but clearly remains at a higher-level than what can be used for estimating the benefits of a particular education or training intervention.
For the purposes of the CBA, the analyst must develop a lifetime earnings profile for intervention participants in the with and without project scenarios. This typically starts with estimating the effect of changes in schooling on annual (or some other unit) earnings using nationally representative household or labor force survey. With such data in hand, the most widely used analysis is to employ the Mincer earnings function to estimate the returns to a particular year or level of schooling.
The most simple model takes the following form:
ln (yi)=rSi+β2 Xi+β3 Xi2+μ1,i, where ln(y) is the is the natural log of monthly income from labor earnings (e.g., salary and wages), S is a variable for years of schooling, r is the returns to schooling, X is potential work experience, equal to Age – Education – 5, and μ1 is the error term. When graphed with age on the x-axis and income on the y-axis, this depicts an upward-sloping and concave curve to represent the age-earning profile. This trend makes sense as there is more potential to increase earnings in youth, but the rate of growth in earnings would slow down over time, as one gets older.
This human capital model is simplified, but a useful starting point. Annex III provides a table that includes several Mincer regression specifications that the analyst can explore to inform the decision on the specific parameters to estimate for a CBA model. This analysis should be done alongside examining the labor market outcomes (discussed just above) and examining wage increases found in similar interventions that have conducted rigorous evaluations. Together, these sources of information can help determine the base case value for the main CBA model, and then the ranges or alternative with project scenarios to consider.
The estimates from Mincer earnings functions will provide a snapshot of the situation at a given time, but the CBA requires the development of an individual’s lifetime earning profile. For this, it could be helpful to simply look at the distribution of wages and employment probabilities by age groups and levels of education to better understand how these may shift across time, and see the current experience of recent graduates with limited experience – the most relevant group for a general education intervention and many TVET interventions. If possible, examining various data points across time could indicate how a certain ‘cohort’ in society is tracking across time. However, in many developing countries the education and career pathways expected now are not the same that were expected, or even possible, say 20 years ago, due to improvements in infrastructure, presence or resolution of conflict, improvements in economic inclusion for women or minorities, and other factors. These possibilities should be considered when developing both the counterfactual and with-the-intervention estimates. One exercise used in Georgia II was to estimate the average increases in real wages across time using national data available. These estimates were used to project future expected wage increases that were incorporated into the with and without project scenarios of the CBA model.
These recommendations have been general to education, but there are notable challenges for estimating wage benefits from TVET. One is sample size. Household and labor force surveys in many developing countries have few observations for those that have completed any level of TVET. This is likely due to the general lack of existing programs and/or their popularity for those that meet the education prerequisites. This is further complicated by the fact that there is great variation in TVET levels of certification, making it necessary to disaggregate further. For example, for the purposes of estimating earnings impacts, it would not be appropriate to combine individuals who completed post-lower secondary TVET training and those who have attained the highest level of technical training and are qualified to serve as a senior technician or engineer. The sample sizes are typically too small when examining by diploma level obtained to even use for informing sensitivity analysis. This is true for both earnings and employment data. In addition, the quality of training obtained by individuals in the survey may be below that expected for the proposed TVET program, which may be designed precisely to remedy shortcomings in existing programs. For these reasons, as elaborated below in the section on TVET, the analyst usually must rely on other data sources, such as tracer studies, rigorous evaluations, or focused employer surveys.
With the basics of estimating wages discussed, we return to the three medium-term outcomes to determine what this may mean for a particular type of intervention. This work is most straightforward for interventions directed at the first medium-term outcome, an increase in the years or level of schooling. Imagine that a given intervention would build new schools to facilitate transition from lower secondary school (middle school) to upper secondary school (high school) in an area with currently limited access to the higher level of schooling, and provide complementary activities to support that transition. For the percentage of students that would be expected to continue onto and complete upper secondary schooling (this would need to be determined above in the quantifying process), the lifetime incremental gains in earnings from the expected increase in educational attainment could be derived in straightforward fashion using Mincer regressions and the approach just described.
The second medium-term outcome is an increase in student learning from an intervention that would improve the quality of schooling for a given year. The literature remains much more limited on the relationship between learning (measured by test scores) and wages. This may be in part because improvements in test scores are also associated with additional years of schooling, so it may be difficult to disentangle the two. Regardless, several researchers have attempted to estimate the returns in the labor market to these learning improvements. Earlier MCC CBA models based the value of this parameter on the findings from the Hanushek (2010) paper. This is still considered one of the strongest papers on the topic, but two new companion papers by Chetty et al. (2014a and 2014b) also employ rigorous methodologies and micro data to estimate the relationship of earnings to test performance. Bacher-Hicks (2014) supports the strength of the methodology used in both Chetty’s papers and the Hanushek (2010) paper, which is further supported by Murnane (2000), Mulligan (1999) and Lazear (2003). Overall, these 6 papers find results that around a 10% – 15% increase in annual income for a 1 SD improvement in learning. The latter would represent a significant increase in test scores but help to standardize the findings across papers in this literature. Based on these findings, CBA models at MCC typically use a parameter estimate of a 12% or 13% increase in annual income for a 1 SD improvement in learning.
The third medium-term outcome is greater alignment between skills obtained and those demanded by employers. While the overall approach is the same as in the preceding cases, the analysis has additional complexities because of the variation in types of TVET and the need to consider a range of employment outcomes. Therefore, we leave that discussion to the more detailed section on TVET CBA below.
In Practice: General Education – School Construction
This section provides a hypothetical lower secondary school construction project to illustrate the concepts introduced earlier in this document. The example begins by summarizing the program’s logic, the core problems the project aims to address, and then identifies the main benefit stream, relevant counterfactuals, and considerations for quantifying and monetizing the benefit. This example is intended to provide overall insights to the approach rather than specific details that would be provided in the documentation of a complete CBA model.
The CBA model is based on the program logic that constructing new lower secondary schools will increase access to education in two peri-urban and rural regions of the country, leading to improved transition rates from primary to lower secondary school, and ultimately more years of schooling, which will result in higher lifetime incomes for the students in these schools who complete additional years of schooling.
In these two regions, the enrollment and graduation rates for lower secondary school are among the lowest in the country, particularly for girls. In the root cause analysis, a main reason for this was found to be a low density of lower secondary schools, meaning that many school-aged students in the target regions cannot feasibly travel to the schools to attend. However, questions also remain about the household demand for education in the target areas, so a complementary intervention on community development is envisioned to enhance understanding of the wide array of potential barriers to attending school and to determine how to reduce those barriers through targeted activities, including social and behavior change initiatives.
The main benefit stream is expected to be the increase in (incremental) lifetime earnings for students who obtain additional years of schooling. These additional years of schooling would result in more learning, but as the schools are anticipated to be of equivalent quality to existing schools, in all aspects, the gains are expected from years of schooling alone; that is, there is no anticipated change in education quality or learning for a given grade attainment. Therefore, the key drivers for the CBA metrics are the average annual earnings of graduates of lower secondary schools, and this estimate relative to the annual earnings of primary school graduates. The community engagement component is intended to support reaching these benefits, as are national level policy and institutional reforms to develop a national scheme for school operations and maintenance (to increase the lifetime of the infrastructure investment), and a strategy to recruit qualified teachers to serve in these more rural areas.
In constructing the counterfactual (i.e., without project scenario) the analyst must account for two main possibilities. The first is that without a lower secondary school nearby, the students would have finished their schooling with completion of primary school, which is particularly likely for girls. A second possibility is that some students would have transitioned to a distant lower secondary school but incurred additional costs (time, fees, transport costs, risk of safety, etc.) to attend this school, and would have potentially dropped out of school earlier due to these costs. For those who fall into the second category, there may be no difference in lifetime earnings (as would be the case for the first counterfactual possibility), but a small benefit stream should be included to capture these cost savings, relative to the counterfactual. For the CBA it will be necessary to define an informed assumption about the relative shares of primary graduates in the targeted region who terminate after primary and who continue on to lower secondary. This could be based on information from recent EMIS or household survey data, or discussions with the communities to record current and recent schooling behavior in the relevant age groups.
Turning to the with-project outcomes, as noted, the main benefit stream identified is additional years of schooling. The next step is to quantify the potential increase. Project feasibility studies and initial design work was conducted to determine the best location for the lower secondary schools, based on the location of existing primary schools (to determine which schools would feed into the new lower secondary schools), and lower secondary schools (ensuring that the new schools were not too close), and census data that indicated the density of houses and estimated the potential flow of school age children. These efforts were further supported by initial conversations with local governments and communities to ensure there would be buy-in on the new locations, and those that did not get them understood why they did not. Together these efforts support developing a stronger project to reaching the intended objectives and can inform the potential estimates to use in the CBA model.
The analyst would hope to draw upon the literature to determine reasonable parameter estimates for schooling impacts; however, rigorous evidence on the impacts of building lower secondary schools, particularly in developing countries appears to be quite sparse. Additionally, although schools had been recently built in other parts of the country, including more rural areas, there had been no monitoring or evaluation data to consider for the CBA. The next best option would be to examine the literature on similar impacts at the primary school level in developing countries. However, only three rigorous studies were found, and they seem to suggest that the two levels are quite distinct, particularly as their country contexts also appear very different from that of the country where this intervention will take place. Therefore, caution is needed in using these findings to inform the CBA estimates. In this situation, the quantifying would prove more difficult than the monetizing.
Based on the three studies, and what is known about the country context where the project would occur, the greatest influence on additional years of schooling would be expected to occur from increases in enrollment in the first year of lower secondary. This enrollment rate would be determined by considering the project’s potential impact on primary school completion, as that would be a requisite for advancing to this level of schooling, and transition rates from primary school to lower secondary school. This information would inform the expected flow of students, outside of the first few years when older students may transfer to this closer school or reenroll after previously dropping out. If data were available, perhaps through an EMIS or household survey, on primary school completion, lower secondary enrollment and transition rates between the two for other areas where schools are equally accessible to the population, then these could be a useful benchmark.
However, transitioning to and enrolling in lower secondary alone would not result in attaining additional years of schooling. Students who enroll would need to remain in school for at least a year, and hopefully go on to complete the lower secondary level (typically 3-4 years). To quantify this parameter with and without the project, the best indicators would be estimates for dropout and completion rates by grades and the lower secondary school level. The current trends or estimates for the targeted regions should be used to inform the counterfactual, with perhaps national averages or those from areas more advanced, in terms of the infrastructure accessibility provided, used to inform the with-project estimates. EMIS data would be best, but household data may provide sufficient information to support estimating these parameters. If there was sufficient evidence to suggest that the community engagement component could effectively address barriers to staying in school, then perhaps this could result in incremental improvements beyond those currently experienced elsewhere, with similar infrastructure and physical access to lower secondary schools as what would be expected from the project. If justified, then this could be factored into calculating a slightly higher parameter estimate for additional years of schooling.
Once the analyst has taken all these aspects into consideration to determine the various schooling-related rates, these would be applied to the number of students in each cohort to convert them into the number of potential beneficiaries. As shown, this requires using probabilities for the two potential counterfactual scenarios, and then forming a narrative on expected school pathways with the average rates of enrollment, transitions, dropouts and completions. The assumptions on schooling after lower secondary would follow national averages or of similar communities, as this intervention makes no attempt to support transition to upper secondary, TVET programs, or beyond. More school-aged students would have that potential to advance their schooling, given they had finished lower secondary school, and that would be reflected, but not that they are more likely than any other graduate of lower secondary school in the country to transition to upper secondary school. Similarly, the project does not have an aim to directly impact labor market outcomes related to participation, employment or unemployment. As with further education, these would be expected to be similar to the current trends of those that graduate lower secondary school, or wherever their education path ends.
Finally, these ultimate increases in schooling would be monetized using a household or labor force survey dataset as described in the previous section – determining the wage differential between students that would leave with say primary schooling and lower secondary schooling, adjusting based on the expectation for the project determined in the quantification exercise. The average gains expected for each student would be aggregated for a given cohort in the first year after they graduate (adjust for where learning paths may end), and tracked for 20 years, as described in the time horizon discussion above. Assuming the national scheme for operations and maintenance is successful, the investment would be expected to last 20 years, and therefore include 20 cohorts. Each year of benefits is put into present value terms so that these can be combined across cohorts and across time to calculate the key CBA metrics. As one can see, this becomes complicated rather quickly, so MCC has developed ‘cascade’ spreadsheets to help facilitate analysts in producing these calculations – i.e., not starting from scratch and adjusting on the margins to reflect the project-specific situation.
In Practice: General Education – Teacher Training
The second intervention example for general education focuses on a hypothetical, nationwide in-service training program for secondary school teachers and directors. Following a similar structure to the previous example, we begin by summarizing the core problems the project aims to address, the program logic, and then identify the main benefit stream, relevant counterfactuals, and considerations for quantifying and monetizing the benefit. This example is intended to provide overall insights to the approach rather than specific details that would be provided in the documentation of a complete CBA model.
As referenced earlier in the paper, developing countries have experienced significant increases in access to education over the past five decades, leading to significant improvements in the quantity of schooling attained. However, education quality and learning has lagged and is often poor. This is the case for the country where the example intervention takes place. Detailed problem analysis determined that secondary school teachers lacked the necessary pedagogical skills and subject-matter knowledge to support students in achieving grade-level learning targets, as seen by scores on standardized national and international exams – particularly for math and science.
The main component of the intervention was to provide over 100 hours of face-to-face training to existing secondary school teachers, providing separate modules to cover pedagogy and subject matter content. The training was complemented by creating networks of teachers, based on location, to support discussions of the training materials, with continued exchanges and collaboration planned for after the training concluded. At the same time, the intervention supported the national government to establish a continuous professional development scheme for secondary school teachers that provided salary increases and other incentives to improve teacher quality, and even a financial bonus to encourage voluntary retirement for those close to retirement age and uninterested in continuing their professional development. Lastly, a separate training curriculum and discussion group components were designed for school directors to ensure that teachers had strong educational leadership to further support teacher development and students in achieving improved learning outcomes.
The CBA model is based on a program logic in which increased knowledge of subject matter and pedagogy among secondary school teachers leads to improved student learning results. The mechanism through which this happens is that the trained teachers learn new information, retain it, apply it in the classroom, and this improves the experience and learning for their students. The national level intervention schemes and improved school management further supports achieving these results.
The main benefit stream from this teacher training investment occurs through increased student learning outcomes. The intervention does not directly aim to increase the years of schooling obtained. There is a possibility that students that enjoy school more and perform better so are more interested in continuing their education—or that parents recognize the benefits and decide to keep their children in school longer–but this potential benefit is not considered in the CBA model. If future literature demonstrates this secondary impact, then this should be examined more closely to consider its potential inclusion. In some circumstances, it could make sense to incorporate a reduction in repetition rates as an additional benefit stream, as improvements in learning can be expected to increase students’ chances of passing each grade. However, in the country of the intervention, the repetition rates are already quite low for all levels of education, so this benefit stream was not considered further for inclusion. Pulling these aspects together, the counterfactual is simply that students would have remained in the schooling system and been taught by lower quality teachers, resulting in less learning, and therefore lower scores on standardized exams.
To quantify this benefit stream, it is helpful to start by examining the literature. In 2019, MCC carried out a more extensive literature review on the topics related to this hypothetical intervention. The summary starts by outlining 12 key factors describing the context and potential components of teacher training programs that should be examined to determine the effectiveness of proposed training programs. The literature reviewed was examined by these factors and compared against the design plans for a similar program, considering the relevance of the county context in comparison to where other evaluated programs were implemented. There are potentially other factors of external validity to consider but overall, this is quite comprehensive in its examination and uses for this purpose. These key factors are highlighted in Table 4, each informed by the literature.
The focus of the intervention is on secondary school students – middle and high school. However, the bulk of the literature, such as the 226 studies review by Evans and Popova (2015), evaluate training of primary school level teachers. This could be indicative, but evidence suggests that it becomes more difficult to impact learning outcomes the later the grade of education (see Factor 1), so caution would need to be taken in directly applying these estimates to the level of our hypothetical intervention. Additionally, this intervention is nationwide, whereas most impact evaluations have been focused on a smaller group of teachers or targeted to specific regions of a country (see Factor 6 below). For example, the average number of teachers trained in the evaluations reviewed by Popova et al. (2016) was 609. The general limits to the literature on secondary education teacher training is summarized in a 2017 Mathematica review.
Based on the review of the literature using the 12 outlined factors, the hypothetical intervention is expected to improve test scores, relative to the counterfactual, by 0.20 SD. This is therefore the value of the parameter used in the CBA model. This is within the range of the Evans (2015) paper, which cites 0.12 – 0.25 based on their work. The sensitivity analysis may decide to use a range of 0.17 – 0.23 for this parameter, with close to a normal distribution, or draw upon the range just cited from the 2015 systematic review. This would be based on the specific context and should be justified by the analyst.
The last step in the process is to monetize this benefit stream. Improved learning outcomes are expected to result in increased future earnings. Based on an MCC 2019 updated literature review (described above in the subsection Monetize), the parameter point estimate is assumed to be a 13% increase in annual income for a 1 SD improvement in learning. Pulling together the quantification and the monetization results, the average predicted impact for a given student would be 0.2 SD x 13% = 2.6% increase in annual salary. This salary increase would be on top of the predicted annual salary for a graduate of X grade/level, based on standard Mincer estimates from household survey data. This would then be multiplied by the number of students within a given cohort that go on to obtain employment – i.e., factoring in the labor market outcomes discussed above, participation, employment, and unemployment.
| # | Factor | Description |
|---|---|---|
| 1 | Schooling Level | It may not be reasonable to expect similar learning results as those found in primary school teacher interventions, but the relationship between level and learning is unclear and may be highly reliant on the quality of the program design to meet the needs to of the students. |
| 2 | Region of the World | The country or regional context should be considered when reviewing the literature, as historical, language, religious, rural/urban, etc. features may warrant further attention. Some of this would come through in other key factors listed but should be weighed when choosing a parameter estimate/range. |
| 3 | Starting point of student learning |
|
| 4 | Starting point of teacher’s training and education |
|
| 5 | Training Exposure Period | This is defined as the number of hours that teachers are trained and within what period of time. This would include all components of the training program, although it is understood that not all methods should likely be treated equally. Assuming that the quality of teacher training is equal across X programs, then the assumed relationship here is that with more hours of training the teachers would learn more, be more capable of applying their learning in the classroom, and this would lead to greater impacts on education outcomes for their students. However, at some point more hours would not be better. The literature does not appear to provide clear guidance on the number of hours, and this would likely be related to factor #4 just described. |
| 6 | Training’s Reach: Number of Teachers Trained | There are two key aspects worth mentioning: first, the success of implementation and second, that widespread training could support desired behavior changes. Programs that train fewer teachers are likely easier to implement, but overall success of implementation would influence the results of all types of programs. With respect to results, it may also be difficult to compare those that are nationwide with a smaller focus. When all peers and supervisors are trained there is more likely to be shifts towards desired behavior changes. These would likely stem from having training-related topics discussed more frequently, more pressure from social norms and expectations, support from others to implement new methods, etc. |
| 7 | Training’s Reach: Only Teachers or inclusion of School Directors/Principals? | Related to factor 6, it would seem reasonable to assume that if teachers and their supervisors are trained then there is likely to be greater adoption of the approaches and material introduced. The inclusion on School Directors or Principals within the training scheme, in some capacity, would seem to be an advantage of a program – particularly if they are not currently being trained. |
| 8 | Training content: mix of pedagogical and subject matter training | The mix of training content should be driven by the problem that the training is trying to address within the specific population. However, within the developing country context, the level of education and training among teachers typically necessitates that both pedagogical and subject matter training be provided (see factor 4 described above), particularly for secondary school teachers. Pedagogical training is clearly key, but if the teacher does not know the material that they are expected to teach then that training alone may be of low value. Likewise, having a teacher with a master’s degree in X subject does not necessitate that they will be a good teacher, and in that case pedagogical training would be a priority. |
| 9 | Training method (e.g., in-class, in-the-classroom, study groups, teacher networks, etc.) | The preference seems to be a mix of methods, rather than a focus on one particular method. Given different learning styles and the need to quickly apply the learning within the classroom, mixed methods are more likely to ensure learning and adoption of new practices. However, that specific mix does not seem to be known within the literature. There is likely a larger set of literature on the training of trainers that would support this – although not explored here as it is a bit outside of the scope of this literature review. |
| 10 | Teacher Incentives | There are various types of incentives, such as linking a teacher’s career opportunities (improved status, promotion, or salary), that have been used in teacher training programs to encourage attendance, completion and/or implementation of the training materials within their classrooms. In principle this makes sense, but the incentives need to be properly aligned to get the desired results. There are some specific studies that have found an impact from aligned incentives in education interventions but overall, the literature is limited. |
| 11 | Time for benefits to kick in | The literature does not suggest any specific lag between when training is delivered and when student achievement is improved, but there is some suggestive evidence based on the timing of endline evaluations that this lag is about one year. |
| 12 | Sustainability of impacts on student learning | For the purposes of CBA, it is important to understand how long the benefits of a given teacher training program are expected to impact student learning. If looking at the general literature of professional training, then it stands to reason that continuous follow-up, opportunities to learn, practice and improve upon newly learned skills should be built into a teacher’s job and performance expectations. There does not appear to be any longitudinal studies on teacher training programs to inform how these potential impacts may diminish across time, particularly if continuous training is not provided. |
In Practice: TVET
Section II.B noted that there are three main categories of TVET: (1) pre-employment, usually occupation or industry-specific, training, (2) training-related active labor market programs, typically for the unemployed, and (3) on-the-job or continuous training for those already working. The evolution of MCC’s TVET investments was also briefly discussed, noting in particular a shift to more employer or demand-driven programs that should more closely meet the needs of the labor market. Thus, the focus has been more on addressing needs for specific (often fairly advanced) skills to enhance growth than on programming directly targeting the poor or addressing high unemployment among low-income youth. Reflecting this, MCC programs tend to be more oriented to the first and third types of TVET, rather than Type 2 TVET that typically provides skills to disadvantaged groups. For example, the TVET centers funded under the Georgia II Compact are generally oriented to individuals with some workforce experience who are currently employed. Participants are 25 years old on average and have about 6 years of work experience. Some are upgrading their skills within an occupation, while others are switching fields. Levels of schooling tends to be high, e.g., completed secondary. Despite this change in focus, the second TVET category does still feature as part of some recent programs, for example, the Morocco II Compact’s Results‐based Financing (RBF) for Inclusive Employment sub-Activity, which targets women, at-risk urban and peri-urban youth that are unemployed or not in the labor force. Therefore, approaches to CBA must be flexible enough to be able to deal with the range of TVET types.
Annex IV provides detail on the year of compact signing and program type for MCC’s TVET investments and shows, among other things, the preponderance of on-the-job experience components in recent TVET investments. Program components are described in more detail in Annex V. It is obvious that the range of potential program components is wide, encompassing not just the ‘usual’ education components such as construction, equipment, curriculum development, training of instructors and overall sector management, but also development of occupational qualification standards and certifications, mechanisms for employer or labor market feedback, and job placement assistance.
The wide range of program components—on top of the differences in TVET types—makes CBA of TVET challenging, since the analysis in principle should account for the impacts of each significant part of the program on the outcomes of interest, which as discussed, generally consist of improved employment outcomes and increased earnings for beneficiaries. Further, the existing evaluation literature is limited in terms of how much it can help to generate parameter estimates for these impacts. For one thing, recent rigorous impact evaluations have mostly focused on Type 2 TVET, or active labor market program activities providing basic skills for specific disadvantaged populations (usually low income or unemployed youth). These evaluations are likely to have limited relevance to the industry-centered TVET featured in newer MCC compacts. Other research, including evaluations surveyed by Tripney et al. (2013), does capture a broader range of TVET types, but most of these studies are not as reliable methodologically. Second, unlike for general education, there is little systematic evidence that disentangles the effects of specific components of TVET interventions such as infrastructure improvement or educator training. One can draw on evidence on these impacts from general education studies, but it may be difficult to generate predictions of impacts for TVET in which one can be reasonably confident. At the very least, this calls for extensive sensitivity analysis of the hypothesized parameters.
Benefit streams for TVET investments
Following most CBA of training interventions, MCC practice has focused on benefits accruing to the trainees themselves, via the two pathways of improvements in employment outcomes and increases in earnings conditional on employment. These pathways are elaborated on below. First, however, it should be noted that this standard approach omits other potential benefits of training, namely those to firms. Employers of course benefit from better trained, more productive employees. However, even if workers are paid a wage equal to their marginal product, which cannot necessarily be assumed in many developing country contexts, firms may enjoy an increase in their consumer surplus (as demanders of labor) which translates into higher profits. In some cases, firms’ need for (and their benefit from) specific skills is acute. For example, if critical equipment lay unused because no one can operate it or repair it, then there could be a loss of profit occurring in the counterfactual scenario. With more access to trained graduates, there would be less and shorter gaps in stalled production, resulting in higher profits. Employers could also benefit from lower turnover and savings on recruitment costs, particularly if in the absence of domestic trained staff, they normally must recruit some skilled workers from abroad. Hence it is important to keep in mind the potential limitations of the current CBA model, particularly for demand-driven TVET.
One reason these firm benefits are typically omitted in CBA of training interventions is that it is usually harder to measure them from survey data that are typically available. In contrast, as discussed, there are usually readily accessible data sources that can be used to predict employment and wage outcomes for trainees, namely household or labor force surveys or direct interviews of employers. That said, the incorporation of employer benefits into TVET CBA remains a topic of investigation and future versions of this document will provide guidance on this topic.
Below we discuss approaches to generate expected changes to employment and earnings outcomes for participants in TVET programs, followed by a discussion of data sources for these approaches.
What is the counterfactual?
The analyst first needs to carefully specify the counterfactual, i.e., the without-program scenario, in order to estimate program impacts on employment and earnings. For TVET interventions, this can be quite complicated. Consider for example, the range of possible training and employment outcomes that might occur in the absence of an intervention to provide training for a specific field or industry. The possibilities under the counterfactual are that the individual would:
- Attend TVET in the same field as the proposed program but at the current, lower quality. The quality improvement could result from supplementing classroom instruction with work experience, updating the curriculum, or other changes.
- Attend TVET in the same field but achieve a lower-level certificate or length of training. This is especially relevant if there is an upgrade to TVET in an industry that involves additional or more in-depth training, e.g., qualifying trainees as senior technicians rather than technicians.
- Attend TVET in a different field (at the same or different level as the proposed program).
- Not attend TVET at all, stopping their schooling at the previous level, which could be lower secondary, upper secondary, or university.
For employment outcomes in the without project scenario, the individual might:
- Find employment in the same field.
- Find employment in a different field.
- Not enter the labor market or not find employment (is unemployed).
These possibilities are particularly relevant to pre-employment (Type 1) TVET programs or to those that, like recent programs in Morocco, Georgia, and Côte d’Ivoire, combine elements of pre-employment and continuous training TVET. Clearly, the possible without-project training and employment outcomes together lead to a large number of potential counterfactual possibilities. Each implies a different earnings profile, hence a different net impact of the project, since the TVET project’s outcomes are compared to what would have happened if there was no project. The challenge is finding reliable information on what earnings would be for individuals in each of these counterfactual categories. And of course, one also must come up with assumptions about how the participants would have been distributed across each of those categories (e.g., the share of participants who would or would not have attended TVET in the field, who would have gotten a job in the field or some other type of work, etc.).
It should be noted that the counterfactual is generally simpler for training programs providing basic skills like literacy, IT, or soft skills for unemployed individuals—TVET interventions of the second type. Here the analyst generally would only consider the expected increment to earnings of these basic skills in the labor market, and the probability of any employment, or possibly, formal or ‘high quality’ employment, in the with and without project scenarios. The counterfactual would also be relatively straightforward for standard interventions of the third type of TVET, on-the-job or ‘in-service’ (or continuous) training. Here it may be reasonable to assume that the individual, who is already in a particular industry or occupation, would remain so in the absence of the training, but at their current skill level and earnings (or, rather, earnings trajectory over time).
Returning to the thornier example laid out above, obviously simplifications must be made. Current work for the MCC Côte d’Ivoire Compact’s TVET Activity suggests an approach. The counterfactual cases above were simplified by assuming that all students would otherwise have been enrolled in an existing TVET program, reducing the counterfactual for training to two scenarios: (1) enrollment in an existing TVET center that does not provide the specific training at same level as the intervention, so they would have finished their schooling at a level below (e.g., as a technician rather than a senior technician), mostly likely in the same field or a similar field; or (2) enrollment in an existing TVET center that provides this training at the same level but of a lower quality than the intervention will provide. Note that these scenarios are similar to the basic distinction discussed for general education, with the first case implying an additional level of training obtained under the program, and the second case indicating a quality improvement, i.e., learning more within the same years/level. Both counterfactual scenarios would be expected to result in lower rates of employment, particularly in their field of study, and lower wages, relative to participating in the intervention.
With just these two counterfactual possibilities defined, the required information (or assumptions) reduces to: earnings trajectories for those who have completed TVET in the same or similar fields but at a lower level and for those who have completed TVET in the same field, at the same level but at the existing, lower quality; the likelihood of being in each of these two without-project scenario groups; and the probability of each of these groups gaining employment. This establishes the counterfactual. For the with-project scenario, one would need to estimate the earnings and employment probabilities for those participating in the new TVET intervention or equivalently, the expected incremental outcomes under the new TVET intervention.
Generating estimates of expected benefits
Earnings
For earnings benefits, there are several potential sources of information. The choice will depend on data availability (and the possibility of new data collection) as well as the nature of the TVET intervention being considered. Ideally the data or estimates will be disaggregated by gender; certainly, this should be done for any new data collection.
Household or labor force surveys: When there is a recent survey containing information on employment and earnings, this can be used to generated expected earnings benefits. Several approaches are possible, depending on the nature of the TVET. Perhaps the most obvious—but potentially problematic—approach is to obtain an estimate of the impact of having vocational or technical training on earnings from a Mincer regression (household or labor force surveys will usually have information on training degrees as well as general education). As discussed earlier, one problem with this method is that sample sizes in most surveys may yield too few observations with TVET—or the relevant TVET—to permit reliable estimation of earnings impacts. Further, the intervention may be designed to provide new forms of TVET not captured in existing surveys, or to improve the quality or relevance of existing training, for example by increasing its linkages to employers via on-the-job experience. Either situation would mean that estimates of returns based on earlier or current forms of TVET will be more reflective of the counterfactual and likely too low, all things equal, so would at best be considered a lower bound estimate for the with-project scenario.
An alternative, especially for industry or occupation specific training, is to estimate how the training will change the occupational categorization or level of the trainees. For example, for Georgia II, completing the TVET program was expected to be equivalent to changing one’s job grade at entry from “plant and machine operators and assemblers with an elementary vocational school education” to “plant and machine operators and assemblers with higher education”. An estimate of the difference in pay between these grades was taken from the household survey. Similarly, for new TVET centers under the Morocco II compact, graduates are expected to become qualified for more advanced occupation levels within the industry. This approach can be informed by discussion with TVET administrators and employers. Note that it requires that the survey has detailed occupational classifications (though direct information from employers on pay by level, discussed below, can substitute for surveys).
In some cases, it may be reasonable to employ earnings regressions by assuming that the training provides an increase in skills comparable to increasing years of schooling or school level by some amount—for which the implications for earnings are more easily obtained with survey data. For the Morocco I Compact, for literacy training for individuals involved in handicrafts, fishing, and arboriculture, earnings returns were assumed equal to the impact of basic education as estimated in Mincer regressions. For a residential vocational training for artisan in the same compact, the earning effects was assumed equivalent to the 2nd yr. of secondary, again derived from Mincer results. A similar approach was taken for the El Salvador I Formal Technical Education subactivity. This approach is probably best suited for TVET involving training in basic skills (including literacy) as these are most comparable to what would be obtain in general schooling. Of course, to use the Mincer regression results it is still necessary to ascertain which level (or gain in) of general schooling would be equivalent to what the training will provide. While this is bound to be somewhat arbitrary, discussions with employers could help move toward a plausible assumption.
Interviews/surveys of employers: Particularly for industry-focused TVET programs, interviews with or surveys of potential employers in targeted industries may provide more accurate estimates of likely wage gains from training interventions. Typically, the employers would be asked what they currently pay new hires with standard skills (which could be entrants with the current TVET, or none at all, depending on the context) and what they would pay new hires with the higher skills to be provided by the new program. Clearly, the former would aim to reflect the counterfactual and the latter the with-project scenario. Getting the latter case right is crucial, but also much harder. In the case of MCC’s Mongolia TVET investment, it was assumed that the skills of graduates were equivalent to those of new international hires. In a variant of this method, for Georgia II a sample of employers was presented with two hypothetical candidate CVs, one for a graduate of the best current Georgian engineering program and the other a graduate of a good US engineering program, and asked the salary offer that would be made to each. Note that the assumption that the new TVET center or program would be of the same quality as a good international program may be a strong one.
External evidence: The analyst can also explore whether evidence from other countries can be brought to bear. This may be in the form of impact evaluation findings from a (hopefully) similar TVET program in a comparable country context, or from a meta-analysis or literature surveys drawing from multiple such studies. Ideally these studies would be randomized controlled evaluations or evaluations using good quasi-experimental techniques. Examples of such meta-analyses or literature reviews are Tripney (2013) and McKenzie (2017). The difficulty here is finding a study or group of studies for which the characteristics of the intervention are similar to those for the program under consideration, and similarly for the country context. As already noted, there appear to be relatively few prior rigorous evaluations of the kinds of industry-focused, pre-employment TVET programs MCC is currently focusing on.
The preceding discussion on information on earnings benefits has been focused on the field of study, or the occupation or industry on which the TVET is focused (e.g., interviews with employers in the industry). But some training participants may not be able to secure a job, and some may find employment but not in the field of study. Estimating employment probabilities is discussed in the next subsection, but it is useful to note briefly how this relates to earnings. For those who end up employed in other fields, an assumption needs to be made about whether the skills obtained in training have value (and raise earnings) in those fields. If there are no such impacts, those who do not enter the field get no benefit from the training, though of course they enter on the cost side. However, this is likely too strong an assumption. Skills obtained from the training may often be transferrable to other fields or attaining a certificate/diploma could signal to employers an individual’s general skills or willingness to learn, leading to higher employment probabilities and earnings offers overall. The Georgia II TVET evaluation results indicate earning benefits for such individuals. The expectation of gains in earnings, even if employed in other fields, is built into the later Côte d’Ivoire and Morocco CBAs.
Employment probabilities
As noted, the earnings or productivity benefits to TVET will be conditional on getting a job–and one for which the skills obtained are relevant. A well-designed TVET program should improve the chances of employment, particularly in the relevant field. In theory this should be strongly helped by the inclusion of workplace experience in the TVET program, as this provides participants with relevant work experience as well as direct contact with a potential employer. The analysis requires determining appropriate employment probabilities, otherwise known as insertion rates, under both the without and with project scenarios. For industry or occupation-specific TVET, estimates of employment probabilities in the targeted field or occupation and elsewhere are needed for a complete accounting of potential benefits, since as noted above, impacts on earnings will likely differ for the two cases.
There are several possible approaches for coming up with an appropriate estimate for (changes in) employment probabilities, mostly using the same sources as for earnings above. As with earnings, efforts should be made to develop estimates separately by sex. One source would be household or labor force surveys, which could provide econometric estimates of probabilities of employment in a given sector or occupation as a function of TVET training, controlling for other factors. However, this approach has (by now familiar) limitations related to small sample sizes and that any regression estimates capturing the impacts of existing (or past) TVET programs would likely be more reflective of a counterfactual, as the intervention is expected to improve TVET relative to the counterfactuals defined above. These estimates could inform with-project estimates when TVET is being expanded without significant changes in content or quality.
For most cases, a better source of information would be impact evaluation findings from a (ideally) similar TVET program in a comparable country context, or a more comprehensive meta-analysis or literature surveys drawing from multiple such studies. For example, for the CBA of Morocco II’s industry-specific TVET, MCC economists used Tripney et al. (2013) meta‐analysis estimates of the improvement in employment probabilities for TVET programs with on‐the‐job training relative to TVET with theoretical training only. Additionally for Morocco, another donor had recently created a TVET center under the same demand-driven principals as the MCC investment being designed, and the initial insertion rate data of graduates was used to inform potential insertion rates that the MCC-supported centers could achieve relative to that for the existing centers that were less demand driven.
An additional source of information on employment rates is tracer studies, which are essentially follow-up data collection efforts with trainees at various points after graduation to assess their success in finding employment. If such a study has been done for the country for earlier TVET graduates, as was the case for the Georgia II TVET analysis, this may provide more precise information about employment outcomes for a specific type of TVET than standard household surveys. They do normally suffer from similar disadvantages, however, including only capturing impacts of existing, hence potentially different or lower quality, programs than envisaged under the intervention. Tracer studies could inform the counterfactual, but they only follow program graduates, not others.
CBA for TVET Grant Facilities
As noted earlier, reflecting the effort to orient TVET more strongly to private sector and industry needs, recent MCC compacts—including in Côte d’Ivoire, Georgia II, and Morocco II, and somewhat earlier, Namibia— have employed a competitive grants program to allocate funds to assist with existing and/or new TVET programs. Organizations (e.g., industry associations) applying for the grants must also supply resources of their own, i.e., have a real stake, further ensuring the tie to the needs of private sector employers. The characteristics of the grant mechanisms vary, with implications for the CBA:
Georgia II put limits on where the investments would be focused–the TVET programs were supposed to be in STEM related fields and at higher education levels (levels IV and V). In practice, this restriction was not always strictly adhered to, but it did allow some focus for the initial economic analysis.
Morocco II in contrast did not have a particular focus but grant amounts differed depending on whether the application was for (1) an existing center converting to the new model of private public partnership with some rehabilitation or extension; (2) a completely new center constructed with the new private sector integration.
Côte d’Ivoire was limited to 3-4 centers, with one center in the public works sector classified as pre-qualified for support before compact signing. This decision was based on their response to a call for ideas and the connection with the other MCC compact investment in the Abidjan Transport Project. The request for concept notes was not limited to economic sectors or levels of training. At the time of the preparation of this paper selection is still in progress.
The main challenge for the economic analysis is that at the time of the original—and most decisional—CBA (that is, when the Investment Memo for the compact is submitted for approval to MCC’s Investment Management Committee), the TVET centers or programs will not yet have been selected. Instead, the analyst must assess the entire grant facility based on limited information. This results in a provisional economic analysis based on expectations about the type of centers, i.e., in what fields and at what levels, that will be funded. Where the number and type of eligible TVET centers is relatively circumscribed (for example, in a handful of industries) the task will be easier. Still there inevitably will be more uncertainty about potential employment and earnings benefits, as well as other factors such as cohort size, than in a more standard (single and known) TVET project. This uncertainty needs to be emphasized in the documentation of the CBA. A further difference from a single-sector or pre-selected TVET center would be that the grant facility mechanism includes additional costs to administer and manage the grants, which must be added to TVET project costs. In practice, these costs are not added to each individual center, but instead considered as a project-level cost to include when the individual CBAs are aggregated to estimate the project-level CBA. For the original CBA, individual centers are held to a higher threshold ERR to consider these additional grants management costs, typically 12% instead of the normal 10% – this is used for grants overall at MCC and not just for TVET.
Once the grant facility mechanism is set up, a first step would be a call for ideas or concept notes that would serve as the first round of competitive selection of potential grantees. Those selected would be asked to prepare a detailed proposal that would be used for final selection. The MCC and MCA county teams would work together to assess these proposals, a process that involves conducting a CBA on each of the proposed centers. Initially, the Georgia II compact call for concept notes provided applicants with a simplified CBA template and requested applicants to submit their own CBA with their proposals. This was not ideal as applicants lacked the necessary skills to do the analysis, and it was relatively easy to manipulate the expected benefits or costs to ensure an ERR above the threshold. Based on that lesson learned, MCC decided to conduct the analysis either on their own, with support from a consultant or with the MCA to calculate an ERR for each proposal (a non-trivial task) for Côte d’Ivoire, Morocco, and later steps in the process for Georgia. Once grantee selection is complete, the individual center results are used to provide a more accurate, updated ERR for the overall grant project.
C. Costs
The essential principal behind the costs side of CBA is that costs of all inputs required to produce the project benefits must be included, in monetized form. This includes costs borne by MCC as well as contributions of other actors, including other donors, the government, and private firms and households. The contributions can take the form of cash, in kind contributions, time costs, and use of or depletion of any assets used to deliver the benefits. Administration costs associated with compact implementation are also accounted for, including those incurred by the Millennium Challenge Account (MCAs). It is irrelevant from a CBA perspective if the burden is borne, or the funds come from, outside the compact country: all are included.
Examples from education of non-cash contributions might include: the time of community members volunteering in the construction or maintenance of a new school; inputs such as building materials donated by local residents or businesses; public land provided by the partner country government to build education and training centers; equipment or machinery donated by businesses to a TVET center; and instruction time volunteered by professionals in an industry to such a center. In each case the project is not purchasing the inputs, but they would be included in costs and valued at market rates (which for donated time would be the appropriate market wage of the individuals involved). Also potentially significant are the time costs of students while they attend school, i.e., their opportunity cost. This will be especially relevant for higher levels of schooling and TVET, since students here are older hence more likely to have the possibility of working (TVET considerations are discussed below). However, as noted in the general education examples above, those under the legal working age may be employed within or outside of their household, so that should be examined and considered in the CBA model, as relevant.
Project cost accounting distinguishes between initial or fixed costs, including capital costs, and recurrent or ongoing costs of administration and operation, including among other items the salaries of teachers, management, and support staff and maintenance of buildings and equipment. Note that fixed costs, in addition to school or training center construction, includes a range of other up-front expenditures, e.g., development of new curricula and training of educators. Operating costs in contrast are incurred over the life of the project (20 years or longer, to align with how benefits are specified), and could be higher in the with-project scenario if, for example, an infrastructure improvement investment now results in incremental utilities costs like heating, cooling, and electricity. For both types of costs, as indicated, it is necessary to account for (and monetize) any in-kind or donated inputs. Finally, any MCC-funded project would include additional, indirect costs related to the overall compact, namely MCA administrative costs (e.g., staff salaries, building operations) and M&E-related costs including data collection and analysis. These costs are typically prorated to the project at hand based on its overall share of the compact’s total project expenditures. These MCC-specific costs are only incurred during the 5 years of the compact’s implementation.
A more fine-grained, activity-based categorization for use in costing education and training projects are the ‘standard reporting categories’ used by USAID and described in a recent guide. These categories are listed below, and align with the various elements of interventions mentioned previously in this paper:
- General operations, management, and reporting
- Assessments and evaluations
- Higher education/Pre-service teacher-training
- In-service teacher training
- Teaching and learning materials
- System strengthening
- Private sector engagement
- Parents/community engagement
- Safe schools and infrastructure
- Grants, scholarships, and cash transfers to individuals/families
- Grants to organizations
- Other
It should be noted that the infrastructure category would include potential resettlement costs and environmental and social impact costs. MCC has specialists in environmental and social performance on each country team lead related work to ensure that practice conforms to International Finance Corporation (IFC) performance standards. ‘System strengthening’ would cover a range of measures including PIR in the sector. There is no need to follow these categorizations (or their labels) exactly, though they form a useful starting point for thinking about the costs of education and training projects. Each of these categories would potentially consist both of fixed and recurrent costs and would include a standard list of inputs such as labor, materials, rent, utilities such as electricity, and other expenditures.
As with benefits, costs over the life of the project are to be represented in real terms using an estimated rate of inflation. CBA is conducted in real terms—that is, future costs and benefits are all expressed in current units of the currency (usually the compact country currency, in some cases USD, using the same base year for both). Both the IMF and the World Bank provide forecasts for inflation of most countries And as with benefits, discounting is used to obtain the present value for aggregating costs across time.
Dealing with uncertainty
As with project benefits, costs cannot be known with certainty (though it is probably easier to predict and measure costs than benefits), and this uncertainty is greater the farther into the future one goes. A fair amount of research has investigated this issue for general construction costs that can be applied to the education and training sector. As noted in the recent MCC guidance document, “Vertical Structures Development and Implementation Guidelines’ (July 2021), research shows that that project cost estimates for buildings and facilities systematically underestimate actual costs. In the past, underestimating construction costs has led to MCC project ERRs being significantly revised downward from their original decisional estimates and required MCC to modify project designs. Therefore, current MCC practice for construction projects is to build in a generous 30% contingency to these costs, at least when the actual design details are mostly unknown. As the project design progresses to become more defined and as construction approaches completion, a better picture of the actual costs emerges, and this contingency is gradually reduced to zero as a percent of construction costs. This practice will be applied to cost estimates of construction components of education and training projects hence will be reflected in the CBA model and ERR calculations, adjusting over time for the reduction in cost uncertainties. For the various other inputs into education and training projects, there is less firm guidance, but at the very least, the analyst should conduct sensitivity analysis for major cost components of the project.
Maintenance
For any project involving infrastructure, assumptions about maintenance are crucial. Maintenance is needed to retain, to the extent possible, the efficiency of productive assets. While the importance of maintenance is perhaps most obvious for infrastructure such as roads and power facilities, it is also essential for schools and training centers. Unless adequate maintenance expenditures are made, the life of the asset (e.g., a school building) will be shortened and/or will experience declining efficiency in terms of producing the intended outcome such as learning or grade attainment.
The issue is not resolved by simply building in the costs of some appropriate level of maintenance expenditures over the 20-year (or longer) period. It may be completely unrealistic to expect that the government will make these expenditures, if traditionally it has failed to do so on existing school facilities. A more appropriate approach may be to assume a future pattern of “business as usual” with regard to maintenance on a new facility, so that the with- and without- project scenario with regard to maintenance would be the same. Where business as usual implies inadequate levels of maintenance, there are two options for the CBA, as noted earlier. One is to assume a shorter useful life of the asset, say 10 rather than 20 years. The other is to keep the time horizon at 20 (or higher) years but incorporate a reduction in benefits for later cohorts to account for the reduced productivity of the asset.
In some cases, the project design will take the problem of poor maintenance head on. In Morocco II Compact’s Secondary Education Activity, MCC has incorporated a complementary investment to improve the operations and maintenance (O&M) process and system for similar outputs of the education intervention. The country’s performance in implementing the new O&M scheme will inform the assumptions regarding maintenance and hence future benefit streams and ERRs as the project proceeds. . Ultimately, the approach used should be based on a realistic appraisal of the past performance of the relevant government authority and the likely commitment to future maintenance needs under the project.
Additional considerations for TVET
While the above discussion is fully general and applies to TVET programs as well as general education, we should note a few areas where emphasis may differ substantially between the two. First, TVET that is strongly demand or industry driven, which is now a typical feature of the programs that MCC supports, tends to have significant involvement of the private sector. For projects funded through a grant facility, applicant organizations (which will usually be private industry groups or community organizations or consortiums of relevant actors) are expected to come up with a share of the funding. In addition, these organizations or firms may donate equipment, space for centers, individuals’ time in teaching or curriculum development, and supervision of students during on site experience. As emphasized above, all these private costs need to be accounted for in the CBA.
Second, the opportunity cost of time of TVET participants is usually more of a factor than for general education students, as the former are older hence more likely to have income generating work as an alternative use of their time. First, however, the analyst should assess whether trainees in fact are unable to work while in the TVET program. Some continuous learning TVET programs may accommodate working students by scheduling the training in the evenings, and some employers may continue to pay worker’s salaries while they train, especially if the training is very industry specific. For TVET of the first type, however, in which the individual is effectively extending their schooling instead of entering the labor market, opportunity costs are more clearly relevant, but will still depend on the counterfactual assumptions. Earnings predictions from Mincer regressions for the relevant level of education and experience (potentially zero experience in this example) can be used to assign the appropriate opportunity cost. Finally, for TVET of the second type described above, that is directed at individuals who are unemployed or not in the labor force, the standard MCC practice is to assume zero opportunity cost of time.
Third, equipment costs usually loom large for TVET in ways that they do not for general education. To take one vivid example, pilot training without expensive flight simulators—and/or real planes—would hardly be effective. This is not a qualitative difference from general education cost analysis, but the analyst should make sure that all necessary equipment costs for a particular type of training are budgeted, including any donated equipment.
D. PUTTING THE CBA PIECES TOGETHER TO INFORM DECISION MAKING
This section outlines the process to estimate CBA metrics and to carry out sensitivity analysis to test the robustness of the results, and better facilitate the use of this analysis for decision making. The discussion is just intended to highlight the main elements of this step, which are common to CBA in all sectors and are covered in detail in the MCC general CBA guidance.
Metrics for decision-making
The purpose of economic evaluation at MCC is to support the agency to make better decisions and design projects that efficiently use US taxpayer funding, and to further consider equity and tradeoffs in making such investments. As discussed earlier, MCC uses other investment criteria alongside the CBA, which is warranted particularly when some benefits cannot be easily measured or are difficult to monetize. Distributional factors may also influence project assessment.
MCC uses the economic rate of return (ERR) as an investment criterion because it facilitates comparison across projects as well as relative to the threshold of 10%, as formalized in 2015 in the MCC Investment Criteria. As indicated earlier, the use of this threshold value is the subject of continued discussion at MCC. The analyst also calculates an additional statistic, the Net Present Value (NPV). While the ERR represents the discount rate when the NPV equals zero, and is therefore a percent, the NPV is a monetary measure (i.e., in USD) of the discounted net benefits. As such, the NPV can be greatly influenced by the scope or scale of the project, making it difficult to compare projects of different sizes. The ERR is not affected by scale and thus helpful in comparisons, and a factor in why this has become the primary metric at MCC for deciding among investments or whether an investment is economically and socially worthwhile. ERRs are calculated for each project, at the lowest level of aggregation possible, with the original ERR reported in MCC’s investment decision document, the Investment Memo. As described in MCC’s CBA Guidance (page 18-19), the agency officially reports final ERRs at potentially five key points in time, and the ERR at each of these points is given a specific title and definition. The first four are normally completed by MCC Economists and considered ex ante CBA models (Original, EIF, Revised and Closeout), while the fifth is completed by a separate entity contracted by MCC and considered an ex post model.
Supporting documents reporting a final ERR (e.g., CBA models published on MCC’s website, M&E Plan) should include at least the following standardized information to comply with MCC’s full disclosure requirements and be aligned with the practices noted in our CBA guidance: the ERR (mean), the probability that the ERR is above MCC’s threshold, the NPV (mean), the present value (PV) of all benefits, an outline of critical parameters (i.e., those that are most influential to the ERR estimate, including the sources and ranges for these parameters), and an indication of when project components with separate logics do not have separate ERRs including a note on why and if an ERR is expected in the future. This additional information aims to provide insights into the uncertainty and distribution of summary statistics to support greater understanding of an investment’s potential risks and benefits. Additional specifics for each supporting document are included within the templates developed and approved by management of the economic analysis division.
Sensitivity analysis—Assessing Uncertainty
Risks and potentially inaccurate assumptions may lead to an inability of the project to attain the benefit streams as captured in the CBA model. The economic logic described above can be used to assess where in the logical chain from investment to benefits the risks are most significant. Many of these risks come down to expectations about behavioral or social change. In addition, results may be affected by broader external factors well outside of the control of the project – such as unexpectedly high inflation, currency risk, climate change and climate events, COVID-19, and changes to policy and institutional reforms, and any other complementary investments that are expected but may not materialize or not materialize on time. The economic analysis can help project design by highlighting where mitigating actions can be developed to reduce potential negative impacts from external factors, or perhaps incorporate additional components to an intervention that would support behavior change in participants to increase uptake for a given intervention.
MCC’s Cost Benefit Analysis Guidelines distinguish the following types of uncertainty:
- Parametric uncertainty, i.e., uncertainty regarding the value of numerical inputs that represent the relationships embodied in the logic and CBA models. Examples would include parametric assumptions about how learning changes from an increased in teacher training, and the relationship of earnings and employment probabilities to changes in school attainment or training.
- Structural uncertainty, a more fundamental form of uncertainty, concerning whether the model captures the key relationships (including benefit streams) and their interactions.
- Scenario uncertainty, referring to changes in broader factors outside the project but that affect benefits or costs. These include the macroeconomic environment, climate factors, broad policy changes, as noted above.
Several approaches are available to deal with these sources of uncertainty. Scenario analysis is particularly useful for understanding the implications of the third type of uncertainty above, for example, changes in inflation affecting costs, or in other policies that affect benefits. Here the analysis would show how vulnerable the ERRs are to changes in assumptions regarding these external factors. Scenario analysis would also be the means of assessing the impacts of changing basic aspects of the model, such as the time horizon or discount rate.
For parametric uncertainty, there are several possible approaches. If a statistical distribution can be reasonably assumed for the parameter, Monte Carlo methods can be used to translate the parameter uncertainty (across multiple parameters at once) into outcome or ERR uncertainty, thereby by producing a probability distribution for the ERRs. This in turn allows the analyst to establish the probability that the ERR is above the hurdle rate. Use of Monte Carlo analysis is now the default approach for conducting and reporting sensitivity analysis in the Investment Memo. However, these results rely on the validity of the distributional assumptions for the parameters in question. Where there is little or no prior evidence on the appropriate distribution for a parameter, it is better to use simpler approaches such as simple sensitivity analysis (varying the value of the parameter), bounding values, or “break-even” approaches in which the parameter value required to cause the project to fall below the hurdle rate is identified and then assessed for plausibility. More detailed discussion of these issues can be found in the MCC CBA guideline document.
Beneficiary Analysis
Research has shown that economic growth in a country tends to reduce poverty, but there are significant questions about the potential pathways through which this can occur, and which policy actions or interventions could expediate poverty reduction. As MCC’s mission is to reduce poverty through economic growth in our partner countries, this topic is closely considered during project design. At this more micro level, the assumption is that not all investments are guaranteed to reduce poverty, and there could be notable differences in the poverty reduction potential of intervention alternatives that aim to address the same core problem. The CBA, as presented above, aligns with a focus on overall economic growth, not distribution: final outcomes are typically measured as increases in income and it is the aggregate gain in incomes that is used to calculate the ERR. A complementary, distributional analysis could outline who incurs the costs and obtains the benefits of a given investment, and the distribution of those costs and benefits across certain groups. Together, these analyses can highlight tradeoffs between the potential for economic growth and poverty reduction among a set of potential interventions, as well as how a given intervention could be adjusted, or a complementary investment introduced, to better meet the aim to reduce poverty or increase benefits for one or another marginalized groups.
Thus far, the CBA has implicitly assumed that society equally values benefits to all potential beneficiary groups and furthermore that all beneficiaries value a dollar’s worth of benefit equally. However, on the first assumption, project funders like MCC often have an explicit preference for investing in projects that will provide more benefits to the poor or other specified groups. Equal valuation of costs and benefits may not produce results that reflect this motivation. On the second assumption, it is known that groups differ in how they would value the benefits and costs associated with a given project. For example, $500 in annual benefits would mean less to a wealthy family than a lower income family, for which this may represent a significant percentage increase in annual household income (in economic terminology, the marginal value of income or consumption is assumed higher for the poor). On the cost side, students who obtain additional years of schooling experience an opportunity cost to attend, which typically impinges more on those from poorer households, which may lack the resources to do without the foregone income.
As discussed in the Sections II. A and B on problems and interventions, these opportunity costs could be so high that individuals decide not to participate in an education and workforce development program and therefore miss out on the potential to become beneficiaries. In fact, even after school, those from non-poor families tend to have higher reservation wages as they can remain unemployed until they receive an enticing job offer. Individuals in lower income households do not have this luxury, so they may accept less than ideal employment, even if it is unrelated to the training that they just completed. These aspects of inclusion should be considered, and distributional analysis conducted earlier in project design could help to highlight who is intended to benefit and why, and identify who may be excluded. This can further inform the detailed design of an intervention to obtain a better distribution of benefits overall – supporting donor or government investments that not only achieve social efficiency (favorable results from a CBA) but could have a greater poverty reduction impact.
For the purposes of the CBA itself, there are two methods that economists typically use to assess the distributional impacts of projects. The first method would adjust the CBA model itself by applying distributional weights to account for variations in the valuing of costs and benefits of different groups. While this sounds like a logical approach, in practice it is challenging to determine a justifiable, non-arbitrary set of weights to use in the analysis. What should be assumed about the social valuation of a dollar given to a low-income household relative to a middle-income household? For this and other reasons, economists tend to avoid applying distributional weights in CBA.
The other method for distributional analysis is the one that has been adopted by MCC, and is referred to by MCC as Beneficiary Analysis (BA). The BA attempts is a separate exercise that attempts to estimate the number of beneficiaries, and measure how the benefits forecast in the CBA model accrue to different groups. More specifically, this analysis aims to estimate the flow of benefits to various income categories (especially the ‘poor’), as well as to assess the potential impact on populations of particular interest, such as women, ethnic or religious minorities, those within a particular region, the aged, or children.
Together, the results of the CBA and BA will inform decision making on both the intervention’s potential for economic growth and for poverty reduction, as well as to provide insights on its ability to respond to an MCC agency priority of gender and social inclusion. As MCC has three priorities (gender and social inclusion; climate change; and blended finance) and several other outlined investment criteria (see footnote 15), tradeoffs often arise when making investment decisions. The CBA and BA results help project teams assess whether an investment could be adjusted to better allocate cost and benefits to support poverty reduction, or possibly, could be coupled with other investments that are more clearly pro-poor or targeted to groups that may be historically more marginalized (e.g., women, certain ethnic groups).
Defining Beneficiaries
According to the MCC Guidelines for Economic and Beneficiary Analysis, beneficiaries of projects are considered individuals that are expected to experience better standards of living due to program activities. These better standards of living can be materialized as financial gains or improvements in other social outcomes, but ultimately the CBA measures these in monetary terms, as an increase in their real incomes. A CBA model provides details on benefit streams through which beneficiaries should experience increased income or enhanced wellbeing through improved outcomes (for instance, the value of longer, more productive lives, the value of home production, such as childcare and domestic services, and changes in future welfare associated with a country’s natural assets) as a result of the intervention.
Reporting total beneficiaries is an MCC statutory requirement. For a given intervention these estimates would include all members of any household that has at least one individual directly benefitting from that intervention, aggregating across 20 years (or the alternative time horizon established in the CBA model). All household members are included because research indicates that when one member experiences an increase in income, it generally benefits the entire household, even if the intrahousehold distribution of these benefits is not equal. As the household is typically defined by when the beneficiaries begin to accrue benefits, for most education interventions this would not be when the student is a child, but when they have made their own household.
In addition to the total beneficiaries, there are two other groups that are often defined, particularly for education and training projects: participants and direct beneficiaries. In some cases, participants are distinct from direct beneficiaries, e.g., teachers may be trained as part of an intervention so are participants but are not considered beneficiaries, which are the students. In other cases, participants are also likely to become direct beneficiaries, e.g., students wo attend a newly constructed school. Table 4 that follows provides an overall definition for each of the three groups – participants, direct beneficiaries, and total beneficiaries, as well as examples for how these could be defined for the example interventions that are used in the previous sections: general education, school construction; general education, teacher training; and demand-driven TVET. This is followed by a real-life example for an education intervention in an MCC country.
| Participants | Direct Beneficiaries | Total Beneficiaries | |
|---|---|---|---|
| Overall Definition | Individuals engaged in an MCC intervention. Some participants will become beneficiaries, but not all. | Individuals who realize improved standards of living due to their participation in an MCC-funded intervention | All members of a household with at least one direct beneficiary. |
| General Education, School Construction | Individuals from non-urban regions, with a greater likelihood to come from low-income households, who are enrolled in newly constructed, MCC-funded schools | Participant students who complete additional years of education due to the newly constructed, MCC-funded schools and enter the labor market. | All members of a household with at least one direct beneficiary |
| General Education, Teacher Training |
|
Participant students who complete their years of education with the newly trained teachers and enter the labor market. | All members of a household with at least one direct beneficiary |
| TVET, new demand-driven centers with new training programs | Individuals enrolled in new demand-driven TVET centers. | Participant students who complete training provided by new demand-driven TVET centers and enter the labor market. | All members of a household with at least one direct beneficiary |
The Morocco II Compact can serve as a useful example. The Education and Training for Employability Project includes a Secondary Education Activity that is estimated to have the participants, direct beneficiaries and total beneficiaries across 20 years as described in Table 5. Starting with participants, two main groups are highlighted: trained educators and secondary school students. As noted above, the teachers remain only as participants, while students have the potential to become direct beneficiaries. In this particular example there are far fewer students that are expected to be categorized as both participants and direct beneficiaries. This occurs for a few reasons that are helpful to note.
The first reason is that the CBA model is built on the assumption that students can only become direct beneficiaries if they graduate from the with-project school (lower or upper secondary), rather than simply finishing an additional year of schooling. The second reason is that not all students who graduate will enter the labor market and then obtain employment. This is particularly important for estimating the number of female direct beneficiaries, as this subset of the population has low labor force participation rates. As the program was not designed to improve labor force participation rates, unemployment rates, or address any labor regulations that could be impacting these employment outcomes, the status quo was assumed in both the with and without project scenarios. Therefore, boys are more likely to shift from participant to direct beneficiary status.
This example helps to highlight that designing interventions that improve student learning is a necessary but not a sufficient condition for increasing individual incomes. If the labor market cannot absorb these graduates or other factors are inhibiting them from entering the labor market, then the expected benefits and number of beneficiaries would be reduced. This can be restated as a lesson learned: the importance of labor market conditions, as well as behavioral aspects of labor market decisions, should be considered in project design to determine whether complementary investments are needed, given the country context, to increase the potential for obtaining benefits, particularly for groups of interest, such as women and girls.
| Group | Estimate | Definition |
|---|---|---|
| Participants | 550,000 |
|
| Direct beneficiaries | 180,000 | Students in with-Project schools who graduate and find employment |
| Total beneficiaries | 830,000 | Direct beneficiaries and their family members, assuming 4.6 people per household |
The final step in this process is to estimate the total beneficiaries for (in this example) the Secondary Education Activity by multiplying the direct beneficiaries (180,000) by the national average household size (4.6 persons). Since total beneficiaries include all household members, the aggregate division of benefits by sex is likely to appear as closer to 50-50, reflecting the average, relatively equal gender composition of households in the community. Therefore, while women/girls may be underrepresented as participants or direct beneficiaries, the total beneficiaries could mask these important findings. This demonstrates the need for examining each the three categories outlined above to produce a more complete narrative about which groups are benefiting and why certain groups may not realize the anticipated economic benefits.
A compact will have an expected number of beneficiaries over 20 years (or the alternative time horizon established in the CBA model), which is an aggregation across all projects, with adjustments made for any potential double counting. Addressing the potential for double counting beneficiaries is important and would be determined on a case-by-case basis. Considering the intervention examples provided earlier, if there was a nationwide teacher training activity and a second activity for building schools within several rural regions, then this could result in a group of individuals who enroll in an MCC-supported, newly constructed school that also has a newly trained teacher. The analyst would estimate the participants, direct beneficiaries, and total beneficiaries separately for each activity, but could not simply add the two activity-level estimates together to report the project’s total number of beneficiaries of each type. The expected overlap would need to be accounted for, ensuring that households that would benefit from both activities were only included in the total project beneficiary estimate one time to avoid overstating the number of individuals impacted by the two activities.
We illustrate this with the Morocco II example discussed above. The Secondary Education Activity included many components, but they were implemented as a package so there was no need to adjust for double counting, at least from this perspective. Some households may combine, as students marry and decide to start their own families, but this additional step was not incorporated within this version of the CBA model and the BA completed. However, double-counting can still occur if a family had more than one lower- or upper-secondary-aged student in a with-project school, as this also has implications for calculating the total beneficiary numbers. While available student data are unlikely to be able to be used to match siblings in with-project schools, adjusting beneficiary counts could be facilitated through other data collection efforts, including in the context of focus group discussions with parents.
The definitions and example above focus on summarizing the process for reporting the total number of beneficiaries, aggregated across all groups. For the distributional analysis of a given intervention, the next steps require defining the groups of interest, calculating the number of beneficiaries for each group, and then describing and calculating the benefits accrued by each group. Specific methods for estimation will depend on data availability and activity-specific details, but MCC’s Beneficiary Analysis aims at least to describe and calculate benefits by income categories and sex (women/men, or girl/boy).
With respect to sex, there may be an expectation that girls will benefit more than boys from various education and training interventions, particularly those that have this aim specified in the program logic. As discussed earlier in the estimation of wages to monetize the benefit streams, given that the education, career and employment paths differ between men and women in most contexts, it should be standard practice to estimate employment outcomes and wages separately within the CBA model. The same could be true for examining other groups that may be historically marginalized. This distinction in the CBA model would facilitate the work of estimating benefits by groups in the distributional analysis.
This subsection has provided a summary of the importance of distributional analysis, its linkages with CBA, and introduces the reader to MCC’s Beneficiary Analysis more generally and then highlights current practices for the education and workforce development sector. MCC has been doing BA for more than a decade, and continues to examine the specifics of the analysis to improve the methodologies employed, the outputs produced, and the efficiency in complete the BA in order to support its use for decision making. As our BA guidance and practices evolve based on continued discussions on MCC lessons learned and advances in this field, future versions of the SCDP will aim to indicate what this means for the education and workforce development sector.
E. Areas for Future Work
Throughout the paper we have pointed to areas where MCC’s CBA of education and workforce development programs could potentially be expanded or elaborated further in a future version of this guidance. The list below is not comprehensive, but highlights a few areas of ongoing work:
- Accounting for externalities in education investments: While standard CBA of education investments typically focus narrowly on returns to the individual in the labor market, research has shown significant benefits beyond the labor market and the direct recipients of the investments, in particular, benefits for child nutrition and schooling, especially from educating girls. Other benefits may include greater social cohesion and reduced crime.
- Accounting for job quality: CBA practice typically considers improvements to employment and wages. While to a significant extent, higher wages capture higher job ‘quality’, jobs to which a better education or training provides access often have a number of non-wage attributes such as access to health care and vacation benefits and a safer or more pleasant work environment. Many of these attributes can be monetized in principle and included in the CBA, if the necessary data are available.
- Accounting for benefit to firms: As noted, TVET interventions (especially) often developed expressly to address skills needs of firms, whereas CBA of TVET programs is typically limited to measuring benefits to trainees. Further work is needed to ascertain when it is important to consider firms’ benefits in the CBA, and the appropriate data and methods for doing so.
ANNEXES
Annex I: MCC’s Historical Findings: Education as a Binding Constraint
The table below lists all countries where a Constraints Analysis was conducted, and education was found to be one of the binding constraints to economic growth. The last two columns indicate when a binding constraint was also found in the country related to labor market regulations or health, which are both tightly linked to the development and use of human capital. Below the table is a list of all the countries where education was not found to be a binding constraint, as well as countries where labor market regulations or health were found as a binding constraint.
| Country | Education Constraint | Labor Market | Health |
|---|---|---|---|
| Belize (2022) | Low quality of education has led to a shortage of trained professionals in all industries | ||
| Benin II (2012) | Failure of coordination in skills training. There is an inadequacy between the needs of the market and the skills available to the market by the Beninese education system. | ||
| Côte d’Ivoire (2016) | Low level of basic and technical and vocational skills as evidenced by very low literacy and firms circumventing the problem by offering training. | ||
| Georgia II (2011) | Insufficient qualifications and skills of workforce that do not meet the demand of the labor market, resulting in high unemployment. | ||
| Guatemala (2013) | Low quantity and quality of education services and unequal distribution of these services, particularly to indigenous citizens and those in rural areas as evidenced by high returns to education, businesses trying to circumvent the restriction by providing their own training, sectors that are not human capital intensive have been the largest export sectors for decades. | ||
| Indonesia (2010) | Despite good progress in primary school enrollment rates, inequality in access to secondary and vocational education remains high in Indonesia. High returns to education and the gap between wages for workers with primary education and those with university education has been widening. | ||
| Malawi (2012) | Returns to tertiary education are high and consequently Malawians make great efforts to study in overseas universities. Primary school completion rates are low and tertiary enrollment is very low. | ||
| Morocco II (2015) | Unequal access to education and poor quality of education and training system as evidenced by low enrollment rates and poor results of Moroccan students in international tests. | ||
| Mozambique (2007) | Overall, there is low educational attainment and half of the Mozambique population has no education or only basic literacy. Unequal access is also an issue. [1 of 6] | ||
| Timor L’este (2017) | Lack of tertiary education has resulted in 30-40% of professional jobs being filled by foreign nationals. | ||
| Zambia (2011) | Low quality of human capital as well as limited access to secondary and tertiary education for the poor, resulting in low levels of employability of the Zambian population. |
Notes:
- Education was NOT found as a constraint in the following countries (in alphabetical order): Benin, Cabo Verde II, El Salvador II, Ethiopia, Gambia, Ghana II, Honduras, Indonesia II, Jordan, Kenya, Kiribati, Kosovo, Lesotho II, Liberia, Malawi II, Moldova, Mongolia II, Mozambique II, Nepal, Niger, Philippines, Philippines II, Senegal, Senegal II, Sierra Leone THP, Sierra Leone, Solomon Islands, Sri Lanka, Tanzania II, Togo, and Tunisia.
- Labor market regulations or employment related aspects were found as constraints in the following countries: Kosovo (near binding, female participation rate), Morocco II, Tunisia (high fiscal and regulatory cost of employing workers)
- Health, a component of human capital, was found as a constraint in the following countries: Guatemala, Lesotho II, Mongolia II, Mozambique, Sierra Leone, Timor L’Este, Zambia
Annex II: MCC’s History of Education Investments: Linkage with Problems
The table below outlines MCC’s education and workforce development investments, and which were found to have education as a constraint, root cause, or neither, noting which were designed before these analyses were an MCC requirement.
| Country | Intervention Type | MCC Funding | Education Constraint | Education Root Cause | No CA; Not constraint or root cause |
|---|---|---|---|---|---|
| Burkina Faso | BRIGHT | $28.8M | No CA | ||
| Cote d’Ivoire | Secondary Education | $111.3M | Yes | Yes | |
| TVET | $35M | Yes | |||
| El Salvador | Human Development Project, Education and Training Activity (non-Formal Skills) | $5M | No CA | ||
| Human Development Project, Education and Training Activity (Formal Skills) | $16M | ||||
| El Salvador II | Education Quality (gen ed) | $69.5M | No | ||
| Education Quality (TVET) | $15.5M | ||||
| Georgia II | General Education | $73M | Yes | ||
| TVET | $15.7M | ||||
| Tertiary | $36.1M | ||||
| Ghana I | Education | $9.4M | No CA | ||
| Guatemala (THP) | Education Project: Secondary General Education | $14.6M | Yes | ||
| Education Project: Improving TVET in Upper Secondary | $4.7M | ||||
| Mongolia | Vocational Education Project | $49M | No CA | ||
| Morocco | Artisan and Fez Medina Project: Functional Literacy and Vocational Training | $32.8M | No CA | ||
| Morocco II | Secondary Education | $114.3M | Yes | ||
| Education and Training for Employability Project: Workforce Development Activity | $107.4M | ||||
| Namibia | Quality of Education (general ed) | $145M | No CA | ||
| Education Project: Vocational and Skills Training Activity (NTF/VTGF) | $28.4M | ||||
| Education Project: Vocational and Skills Training Activity (COSDECs) | $16.8m | ||||
| Niger (THP) | IMAGINE | $16.9M | No CA | ||
| Timor-Leste | Education | $40.2M | Yes |
Annex III: Mincer Regressions
Mincer specification
The table below provides several Mincer regression specifications that can be used to inform the analysts work and decision on the specific parameters to use within a CBA model. It is paramount that the analyst assess the results across these findings to determine the base case value for the main CBA model, and then the ranges or alternative scenarios to consider in the sensitivity analysis.
The first column provides the regression specification, and the second column indicates the strengths, weaknesses and limitations of the results from that specification. The general definitions of the variables included in the table are as follows:
- ln (y) is the natural log of monthly income from labor earnings (e.g., salary and wages).
- S is a variable for years of schooling
- r is the returns to schooling
- X is potential work experience, equal to Age – Education – 5;
- μ1 is the error term
| Specification | Description of Use (strengths, weaknesses, limitations, etc.) |
|---|---|
|
Simple Regression:  |
Here, ‘r’ is interpreted as the average returns to an additional year of schooling. The results do not separate returns by the different school levels. This is the most commonly reported finding. This option may be less ideal for the CBA because we know that there are often great differences by education level, which would not be captured here and highly relevant for understanding potential results of a given intervention. |
|
Level of School Completion (3 variables):  |
This specification includes three dummy variables for completion of primary, secondary, and tertiary, with less than primary school as the omitted category. The results provide returns to the highest level of education obtained, where each individual would only have one of the dummies equal to ‘1’ or all equal to ‘0’ if they did not complete primary. As opposed to the ‘simple regression’, the coefficients (r1, r2, and r3) require explanation on how to interpret. This distinction by level would be helpful for understanding potential intervention impacts against a counterfactual, especially if data is available across time. One weakness is that this specification does not allow the analyst to tease out the “sheep skin” effects (obtaining an additional year of schooling vs obtaining a diploma for completion), as is done in the following specification. |
|
Level of School Completion (5 variables):  |
This specification includes five dummy variables for primary completed, incomplete secondary, complete secondary, incomplete higher, complete higher, with less than primary school as the omitted category. The construction of the variables and the interpretations of the results are similar to that described above (see footnote 11). The benefit of this specification is that the analyst can observe the “sheep skin” effects of receiving a diploma and completing some portion of a given level of education. On the other hand, few countries may report results in this way, so it may complicate country comparisons. |
| Heckman Selection Model: See Heckman’s paper for details on specification | This specification uses a two-stage estimator to utilize a simple regression method while addressing a selection bias that may occur by using a nonrandomly selected sample, as there are clear differences in behavioral functions between those that make decisions around schooling and work. The use of this model was discussed at length and was ultimately discarded as a viable option due to the inability to reliably estimate a selection model using an instrument that affects probability of employment but does not affect earnings. In the absence of a valid instrument, the Heckman model does not appear to perform better than the general Mincer, and could possibly perform worse, due to collinearity issues between the Heckman corrector and the second-stage regressors. For the purposes of the Constraints Analysis, the results would not be comparable to other countries, as this approach is not commonly used, and furthermore as noted above, the comparison across countries helps to eliminate this issue because the same bias occurs in all country estimates. |
| Specifications for Deeper Examination | |
|
Identify Gender Disparities:  Women:  Men:  |
There are two main ways in which the gender differential can be estimated. Both are represented for the simple regression but can easily be used for the other two specifications as well. First, there is the option to include a gender dummy (F) this means it is equal to one for female and zero for males. The coefficient is likely to be negative indicating a lower income earned by women.
The preferred method is to run two separate regressions for women and men, as this elucidates differences in returns to schooling (r), rather than simply demonstrating what we already know – women make less than men on average. This appears to be particularly important in the developing country context, where significant differences in education and labor pathways. |
| Identify other potential differences: urban/rural, indigenous/non-indigenous, job type or employer (public vs. private), types of education (TVET, university, public/private, etc.) | It is not recommended that these terms be included in the above specifications as they are plausibly endogenous and collinear with (or causally affected by) education, and so may bias the education coefficient. However, the inclusion of these potentially endogenous factors as dummies in one of the above specifications or running separate regressions (preferred) can illuminate country-specific characteristics of employment and education returns. As with gender, the difference in income between groups is known, but an understanding of their returns is seen more clearly when separate regressions are run. |
| Probability of Employment | Regressions on the probability of employment can provide a more holistic view of the labor market at a specific point in time and across time (if data is available). For example, in some countries we have seen that higher levels of education for women may not significantly increase wages, but drastically increase the probability of employment. Variables such as urban/rural, gender, social class, level of education, socioeconomic status, type of degree, etc. can be very useful in explaining gaps in employment status. |
| Cohort and Across Time | Running separate regressions by specific cohorts can assess changes in returns to schooling over time. The general Mincer will reveal average returns to schooling. However, it may be that younger workers received different schooling than older workers due to investments or changes in public schooling programs, historical events, etc. Separating observations into cohorts and comparing cohorts can reveal differences in returns to schooling over time. |
Sample and Variable Considerations and Adjustments for Mincers:
- Include adults of working age and run regressions only for employed workers. Common practice is to run Mincer regressions for the population of adults aged 16 to 65 years of age that receive a non-zero income (i.e., working). Although the appropriate age range may differ slightly across countries, a common range provides ease in comparing across countries and to regional and world averages. One clear weakness is that this would not include women who provide non-remunerated domestic work.
- Include all adults of working age. To capture in-country differences and insights into the structure of the labor market, the analyst can also add one-unit of income to all zero-income observations (as natural log of zero does not exist and will drop these observations from the regression sample) and this will provide returns for all adults of working age, whether or not they are currently working.
- Standard adjustments to years of schooling as needed. Regarding the experience terms, using a measure of potential rather than actual experience is common practice given the absence of more complete data on work history. In the event that data is not available for actual years of schooling, it is recommended that for individuals whose highest level of education is “some” level of schooling – but not a completed level – that the years of education be imputed by taking the average number of years between the lower and higher levels. For example, if primary school is five years and secondary school is an additional five years, a person who reports having “some secondary school” as their highest level of education should be coded as having 7.5 years of education. Results would need to be interpreted with caution.
- Adjust income to reflect real labor earnings. The purpose is to discover the returns to labor income, which is assumed to be closely related to individual productivity and returns to the employment of labor. Therefore, the analyst should separate labor and non-labor income (e.g., transfers, remittances, interest or other capital gains, or rent, etc.), if possible. Some surveys measure consumption instead of income. This is a second best alternative that should be fine for in-country analysis but would not allow for meaningful comparisons to other countries since income is the variable of choice. Additionally, it is preferred to measure and test earnings over smaller periods of time (i.e., hours, days, weeks) than over longer periods of time (i.e., months or years). This will help to reduce recall biases, and wages per hour are a better measure of productivity than annual earnings if there is significant variation in employment over a year. However, understanding in-year variations may be important for the country context, taking note of survey timing with relation to agricultural seasons and other potentially meaningful variations.
Annex IV: TVET program type and skills focus
| Program type and skills focus | El Salvador I | Mongolia | Morocco I | Namibia | Côte d’Ivoire | El Salvador II | Georgia II | Guatemala | Morocco II |
|---|---|---|---|---|---|---|---|---|---|
| Year compact signed | 2006 | 2007 | 2008 | ||||||
| TVET type: | |||||||||
| Pre-employment training1 | X4 | X | X7 | X | X | X | X | X | |
| Training for unemployed/ inactive youth or adults2 | X4 | X6 | X7 | X | |||||
| In service training3 | X6 | X8 | X | X | X | ||||
| Program skills focus: | |||||||||
| Industry-focused skills training | X | X | X6 | X | X | X | X (it appears) | X | |
| General skills/literacy/ remedial/soft skills training | X6 | X2 | X |
- For individuals who are still in the formal schooling system, i.e., who have not entered the labor market, or who are currently working but in a different field, so the program provides training prior to entering the targeted field.
- Skills training (basic remedial or industry focused) for those who are no longer in formal schooling, and who are currently unemployed or economically inactive, i.e., out of the labor force.
- In service training for those who are employed in the field, typically provided by private employers alone or in partnership with the public sector
- The El Salvador I Human Development Project had two distinct programs: a formal activity directed at improving an existing TVET center (part of the formal education sector directed at students still in school) and an informal training activity of short skills course for informal occupations directed at vulnerable, out of school populations (rural poor, women, youth)
- For the formal activity, there was some internships provided to students
- Morocco I provided both basic literacy training and technical training to traditional artisans in several fields and to individuals seeking to enter these occupations.
- The Namibia Community Skills Development Center Sub-Activity rehabilitated or constructed and equipped community-based vocational institutes to serve primarily disadvantaged out of school youth, while the Vocational and Skills Training Activity focused on in school youth as part of the formal education system.
Annex V: Program Components of MCC TVET Projects
| Program component | El Salvador I | Mongolia | Morocco I | Namibia | Côte d’Ivoire | El Salvador II | Georgia II | Guatemala | Morocco II |
|---|---|---|---|---|---|---|---|---|---|
| Infrastructure and equipment (for new or renovated centers) | X (formal sub-activity) | x | x | x | x | x | x | ||
| O & M | x | ||||||||
| Curriculum/degree program development1 | X (formal sub-activity) | x | x | X | x | X | x | X | |
| Develop accreditation/ occupational training standards | x | x | X | x | |||||
| Assistance for teacher training or certification | X (formal sub-activity) | x | x | X (for community based centers sub act) | x | x | |||
| TVET sector management, governance reform/ funding, accountability mechanisms | x | X (for Vocational and Skills Training sub act) | X | x | X | x | x | ||
| Internships/apprenticeships to supplement classroom learning | X5 | X | X | X (implicit) | X (expected as part of grant apps)) | X | X | ||
| Mechanisms for continuous feedback from labor market to TVET | X | x | X | x | X | ||||
| Tracer system to track graduates | |||||||||
| Labor observatory/labor market information system3 | x | x | |||||||
| Job placement, employment services | x | x | x2 | ||||||
| GSI/Inclusion (outreach, quotas, scholarships, special programs, etc.) | X | X4 | x | x | x | x | x | ||
| Scholarships | X (formal sub-activity) | x |
- Refers to direct support of this activity. May also be supported indirectly through grants to private TVET centers
- For unemployed youth and economically inactive women, Morocco II provides job placement services to be funded through results-based financing. The remainder of the activities concern pre-labor market entry programming.
- Labor observatory/labor market information system differ from ‘mechanisms for continuous feedback from labor market to TVET’ in that they are broader, providing information to the government, public and various stakeholders whereas the latter are specific components of the TVET sector.
- A number of the artisan training centers receiving grants focus on women artisans.
REFERENCES
Almeida, Rita; Behrman, Jere; Robalino, David. 2012. The Right Skills for the Job? Rethinking Training Policies for Workers. Human Development Perspectives. Washington, DC: World Bank. http://hdl.handle.net/10986/13075 License: CC BY 3.0 IGO.”
Asian Development Bank. Guidelines for the economic analysis of projects, Appendix 18. 2017. https://www.adb.org/documents/guidelines-economic-analysis-projects
Bacher-Hicks, Andrew, Thomas J. Kane, and Douglas O. Staiger. 2014. “Validating Teacher Effect Estimates Using Changes in Teacher Assignments in Los Angeles.” NBER Working Paper No. 20657.
Barro, Robert J. (1991). “Economic Growth in a Cross-Section of Countries.” Quarterly Journal of Economics, 106(2), pp. 407-443.
Benhabib, Jess and Spiegel, Mark M. “Human Capital and Technology Diffusion.”December 2002. Forthcoming in Philippe Aghion and Steven Durlauf (eds.) The Handbook of Economic Growth, Amsterdam, The Netherlands: North-Holland, 2005
Boardman, A.E., Greenberg, D.H., Vining, A.R., & Weimer, D.L. (2011). Cost-Benefit Analysis: Concepts and Practice (5th ed.). Upper Saddle River, NJ: Pearson Education.
Borkum, E., Nichols-Barrer, I., Cheban, I., and Blair, R. Evaluation of the Georgia II Industry-Led Skills and Workforce Development Project. Mathematica. January 17, 2023. link
Burde, D. and Linden, L.L. “Bringing Education to Afghan Girls: A Randomized Controlled Trial of Village-Based Schools.” American Economic Journal: Applied Economics 5(3): 27–40. 2013. Link
Caselli, Francesco and Coleman, John Wilbur II. “The World Technology Frontier.” June 2005, American Economic Review.
Ciccone, Antonio y Elias Papaioannou (May 2006). Human Capital, The Structure of Production, And Growth. European Central Bank. Working Paper Series No. 623
Chetty, Raj, John N. Friedman, and Jonah E. Rockoff. 2014a. “Measuring the Impacts of Teachers I: Evaluating Bias in Teacher Value-Added Estimates.” American Economic Review 104(9): 2593–2632.
Chetty, Raj, John N. Friedman, and Jonah E. Rockoff. 2014b. “Measuring the Impacts of Teachers II: Teacher Value-Added and Student Outcomes in Adulthood.” American Economic Review 104(9): 2633-2679.
Davidson, E., de Santos, A., Lee, Y., Martinez, N., Smith, C., & Tassew, T. (2014). Bangladesh Inclusive Growth Diagnostic. USAID and DFID.
Davis, M., Ingwersen, N., Kazianga, H., Linden, L., Mamum, A., Protik, A., and Sloan, M. Ten-Year Impacts of Burkina Faso’s BRIGHT Program. Mathematica. August 29, 2016. Link.
Dee, Thomas S., “Are There Civic Returns to Education?” Journal of Public Economics 88 (2004), 1697–1720.
Duckworth, A.L., & Yeager, S. (2015). “Measurement matters: Assessing qualities other than cognitive ability for educational purposes”. Educational Researcher, 44(4), 237-251.
Dumitrescu, A., Levy, D., Orfield, C. and Sloan, M. Impact Evaluation of Niger’s IMAGINE Program: Final Report. Mathematica. September 13, 2011. link
Evans, David K., and Anna Popova. “What Really Works to Improve Learning in Developing Countries? An Analysis of Divergent Findings in Systematic Reviews.” World Bank Policy Research Paper 7203. Washington, DC: World Bank Group, February 2015.
Hanushek, Eric A., Guido Schwerdt, Simon Wiederhold, and Ludger Woessmann. 2015. “Returns to Skills Around the World: Evidence from PIAAC.” European Economic Review 73: 103-130.
Hanushek, Eric A., and Ludger Woessmann. 2012a. “Do Better Schools Lead to More Growth? Cognitive Skills, Economic Outcomes, and Causation.” Journal of Economic Growth 17:267-321.
Hanushek, Eric and Ludger Woessmann 2012b. Schooling, educational achievement, and the Latin American growth puzzle. Journal of Development Economics, 99 pp.497-512.
Hanushek, Eric A. 2010. “The Economic Value of Higher Teacher Quality.” NBER Working Paper No. 16606.
Hanushek, Eric A., and Lei Zhang. 2009. “Quality-Consistent Estimates of International Schooling and Skill Gradients.” Journal of Human Capital 3(2): 107-143.
Hanushek, Eric A., and Ludger Woessmann. 2008. “The Role of Cognitive Skills in Economic Development.” Journal of Economic Literature, 46 (3): 607-68.
Hausmann, R., Rodrik, D., and Velasco, A. ‘Growth Diagnostics’ 2005. John F. Kennedy School of Government, Harvard University. https://growthlab.cid.harvard.edu/files/growthlab/files/growth-diagnostics.pdf
Heckman, J., Moon, S.H., Pinto, R., Savelyez, P.A., and Yavitz, A. The Rate of Return to the HighScope Perry Preschool Program. Journal of Public Economics 94 p.114 – 118. 2010. https://heckmanequation.org/wp-content/uploads/2017/01/HeckmanMoonPintoSavelyevYavitz_RateofReturnPerryPreschool_2010.pdf
Heath, Rachel and Seema Jayachandran. 2018. “The Causes and Consequences of Increased Female Education and Labor Force Participation in Developing Countries” in the Oxford Handbook on the Economics of Women, ed. Susan L. Averett, Laura M. Argys and Saul D. Hoffman. New York: Oxford University Press.
Ianchovichina, E. and S. Lundstrom. 2009. “Inclusive Growth Analytics: Framework and Application.” Policy Research Working Paper No. 4851. Washington DC: World Bank.
Karoly, Lynn A. The Economic Returns to Early Childhood Education. (2016) https://files.eric.ed.gov/fulltext/EJ1118537.pdf
Klasen, S., T. Le Thi Ngoc, J. Pieters, and M. Santos Silva (2020). ‘What Drives Female Labour Force Participation? Comparable Micro-Level Evidence from Eight Developing and Emerging Economies’. Journal of Development Studies, 57: 417–42.
Larreguy HA and Marshall J (2017) The Effect of Education on Civic and Political Engagement in Nonconsolidated Democracies: Evidence from Nigeria. Review of Economics and Statistics 99 (3): 387–401 Levin, H.M., McEwan, P.J, Belfield, C., Bowden, A.B., & Shand, R. (2018). Economic Evaluation in Education: Cost-Effectiveness and Benefit-Cost Analysis (3rd ed.). Thousand Oaks, California: SAGE Publications, Inc.
Lazear, Edward P. 2003. “Teacher Incentives.” Swedish Economic Policy Review 10: 179-214.
Mankiw, N. Gregory; Romer, David, and Weil, David N (Mayo 1992). “A Contribution to the Empirics of Economic Growth.” Quarterly Journal of Economics, 107(2), pp. 407-437.
Milligan, Kevin, Enrico Moretti, and Philip Oreopoulos, “Does Education Improve Citizenship? Evidence from the United States and the United Kingdom,” Journal of Public Economics 88 (2004), 1667–1695.
Mincer, Jacob. Schooling, Experience, and Earnings. New York: Columbia University Press. 1974.
Mulligan, Casey B. 1999. “Galton Versus the Human Capital Approach to Inheritance.” Journal of Political Economy 107(6): S184-S224.
Murnane, Richard J., John B. Willett, Yves Duhaldeborde, and John H. Tyler. 2000. “How important are the cognitive skills of teenagers in predicting subsequent earnings?” Journal of Policy Analysis and Management 19(4): 547-568.
Nelson, Richard R. and Phelps Edmund S. “Investment in Humans, Technical Diffusion, and Economic Growth.” American Economic Review, March 1966, 56(1/2), pp. 69-75.
Null, Clair, Clemencia Cosentino, Swetha Sridharan, and Laura Meyer. “Policies and Programs to Improve Secondary Education in Developing Countries: A Review of the Evidence.” White Paper. Princeton NJ: Mathematica Policy Research. August 1, 2017. https://www.mathematica-mpr.com/our-publications-and-findings/publications/policies-and-programs-to-improve-secondary-education.
Popova, Anna, David K. Evans, and Violeta Arancibia. Training Teachers on the Job: What Works and How to Measure It. World Bank Policy Research Working Paper 7834. 2016
Pritchett, Lant. 2001. Where Has All the Education Gone?. World Bank Economic Review. © Washington, DC: World Bank.
Psacharopoulos, George; Patrinos, Harry Anthony. 2018. “Returns to Investment in Education: A Decennial Review of the Global Literature”. Policy Research Working Paper; No. 8402. World Bank, Washington, DC. World Bank. https://openknowledge.worldbank.org/handle/10986/29672 License: CC BY 3.0 IGO.”
Psacharopoulos, George, and Zafiris Tzannatos. “Female labor force participation: An international perspective.” The World Bank Research Observer 4, no. 2 (1989): 187-201.
Revenga, Ana and Dooley, Meagan. (2020) The constraints that bind (or don’t): Integrating gender into economic constraints analyses. Working Paper #137 Global Economy and Development at the Brookings Institution https://www.brookings.edu/wp-content/uploads/2020/04/Constraints-that-Bind.pdf
USAID. Education Activities Office of Education, Bureau for Economic Growth, Education, and Environment (E3). 2018. “Cost Reporting Guidance for USAID-Funded Education Activities”. https://pdf.usaid.gov/pdf_docs/PA00X69X.pdf
World Bank, Independent Evaluation Group. Later Impacts of Early Childhood Interventions: A Systematic Review. Working Paper. 2015. https://ieg.worldbankgroup.org/sites/default/files/Data/Evaluation/files/ImpactsofInterventions_EarlyChildhoodonLaterOutcomes.pdf
MCC DOCS
Bowen, Derick H., and Guyslain K. Ngeleza. Land Sector Cost-Benefit Analysis Guidance. June 1, 2019. https://www.mcc.gov/resources/doc/land-sector-cost-benefit-guidance
Carter, Andrew. Transportation Sector Cost Benefit Analysis Guidance. September 2020. [not yet published externally: internal link.]
Department of Compact Operations (DCO), Infrastructure, Environment and Private Sector (IEPS). “Vertical Structures Development and Implementation Guidelines’. July 2021.
Epley, Brian, Francis Mulangu, and Derick Bowen. Power Sector Cost-Benefit Analysis Design Principles. July 2021. https://www.mcc.gov/resources/doc/guidance-power-sector-cost-benefit-analysis
Heintz, Jenny. Insights from General Education Evaluations, September 2022. Millennium Challenge Corporation.
Myers, Jenn, Stefan Osborne, and Onay Payaam. Health Sector Cost-Benefit Analysis Guidance. Forthcoming.
Millennium Challenge Corporation. Economic Analysis Division. Cost Benefit Analysis Guidelines. June 24, 2021. https://www.mcc.gov/resources/doc/cost-benefit-analysis-guidelines
Millennium Challenge Corporation. Policy for Monitoring and Evaluation. March 15, 2017. https://www.mcc.gov/resources/doc/policy-for-monitoring-and-evaluation
Osborne, Stefan. Water Supply and Sanitation Sector Cost-Benefit Analysis Guidance. May 1, 2019. https://www.mcc.gov/resources/doc/water-sector-cost-benefit-guidance
Patel, Shreena and Sobieski, Cindy. “Independent Evaluation at MCC: An Evolving Practice 18 Year On.” January 2023. https://www.mcc.gov/resources/doc/paper-independent-evaluation-at-mcc
Ricou, Marcel & Moore, Ryan. “Training Service Delivery for Jobs & Productivity: MCC’s Lessons Learned in Technical and Vocational Education and Training.” 2021. https://assets.mcc.gov/content/uploads/paper-2020001233801-p-into-p-tvet.pdf
Szott, Aaron and Mesbah Motamed, Agriculture Sector Cost-Benefit Analysis Guidance. March 31, 2023. https://www.mcc.gov/resources/doc/agriculture-sector-cost-benefit-analysis-guidance
Tracy, Brandon. Guidance for MCC Economic Analysis of the Power Sector. 2013. MCC.
Credit:Source link
