November 16, 2026 · Youth Development · 4 min read
Written by MATSH Research & Editorial Team · Last updated

Evaluation design is about choosing evidence strong enough for the decision being made. A programme manager deciding whether to improve case management may need different evidence from a donor deciding whether to scale a costly intervention nationally. Strong evaluation starts with the decision, the causal question and the realistic comparison available.
The 2026 ILO–World Bank review of active labour-market programmes for youth synthesises impact evaluations across many programme types and illustrates why design, targeting and context matter when interpreting employment effects. OECD evaluation guidance likewise stresses using evaluation criteria thoughtfully and matching methods to purpose.
Source: ILO–World Bank — Impact of Active Labour Market Programmes on Youth · Source: OECD — Applying Evaluation Criteria Thoughtfully
| Question | Evidence need |
|---|---|
| Was the programme delivered well? | Implementation/process evidence |
| Did participants improve? | Outcome monitoring / before-after evidence |
| Did the programme cause the improvement? | Credible counterfactual / impact evaluation |
| Why did it work for some groups? | Mixed-method mechanism analysis |
| Should it scale? | Impact + cost + implementation + context evidence |
Monitoring tells you what happened to participants. Impact evaluation estimates what would have happened without the programme. Both are useful, but they answer different questions.
What blocks employment or earnings?
Which mechanism addresses it?
What should change before employment?
Placement, retention, earnings, quality or progression.
If a programme is piloting a new costly model or competing with other interventions for funding, causal evidence may be important. For routine service improvement, contribution evidence may be sufficient.
The counterfactual is the outcome participants would have experienced without the programme. Because that cannot be observed directly for the same person at the same time, impact evaluations use comparison strategies.
| Approach | Use / caution |
|---|---|
| Randomised assignment | Strong causal design when ethical and operationally feasible |
| Regression discontinuity | Useful where eligibility follows a clear threshold |
| Difference-in-differences | Requires credible comparison trends |
| Matching / weighting | Controls observed differences, not unobserved ones |
| Before-after only | Shows change, not attribution |
Immediate placement may not show sustained value. Plan follow-up for retention, earnings, job quality, progression and enterprise survival where relevant.
If impact is weak, you need to know whether the programme model failed or whether implementation failed. Measure service dose, quality, partner performance, employer engagement and participant take-up.
Outcomes may differ by gender, age, baseline disadvantage, location or programme pathway. Pre-specify important subgroup questions where possible and avoid over-interpreting small samples.
Wage subsidies, placements or entrepreneurship programmes can affect non-participants or shift opportunities. Programme impact should be interpreted at the right level.
| Cost question | Why it matters |
|---|---|
| Cost per participant | Operational efficiency |
| Cost per additional outcome | Compares programme effect with resource use |
| Scale cost | Tests whether unit economics change at larger size |
Interviews with participants, employers, counsellors and partners can explain why outcomes differ and whether the theory of change is plausible.
Economic cycles, hiring shocks and policy changes can affect outcomes. Record contextual changes and avoid interpreting every improvement or decline as programme performance.
Agree what the design can support: description, association, contribution or attribution. This reduces pressure to overstate findings later.
For a hypothetical youth employment programme, organise a follow-up matrix by participant cohort, baseline situation, intervention received, follow-up timing and employment outcome. Record job type, hours, stability and relevant job-quality information where collection is appropriate. Keep training attendance separate from employment outcomes and distinguish self-employment from wage employment.
Before calculating a placement or retention rate, define the denominator and document participants lost to follow-up. Check whether the easiest-to-contact graduates differ from those missing. A positive result among respondents cannot automatically be generalized to every enrollee, and a before-and-after comparison alone does not establish that the programme caused the change.
Use the matrix to identify decisions: which employer partnerships merit closer review, which support services need adjustment and what evidence is still missing. Where a credible comparison design is feasible and ethical, plan it before delivery. Report uncertainty rather than filling gaps with assumed outcomes.
MATSH’s Impact Measurement for Youth Employment Programmes Course helps teams design indicators, evaluation questions, comparison strategies, mixed methods and evidence-use systems without overstating causal claims.
Explore the courseWe run all our courses as private programmes for organisations across the GCC and Africa.
Request In-House →MATSH uses Google Analytics to improve courses. Optional analytics cookies stay off unless you allow them; limited cookieless measurement may occur. You can change your choice later.