Evidence for free school lunches: Are they worth investing in?

Value for Investment is a mixed-methods, participatory system that assesses how well resources are used and how much value is created.¹ We gathered inputs from community and government stakeholders to identify what value they expect the programme to produce. From this value proposition, we created 21 criteria in five economic domains to measure those values and validated them with stakeholders. Next, we collected Ka Ora, Ka Ako evaluations and monitoring data to create an evidence base for each criterion. Programme performance (excellent, good, adequate, poor) for each criterion was made by the research team but has yet to be validated by stakeholders. We present these preliminary assessments with more details available in the attached summary report and from the authors by request. 

Evaluation findings

The summary table of evaluation criteria and provisional assessments is shown. The effectiveness criteria (in red) are the primary outcomes, based on the original Cabinet papers for the programme. ^2,3

Preliminary assessments of Value for Investment for Ka Ora, Ka Ako

1. Effectiveness (Is it achieving the desired outcomes/values?)

Ka Ora, Ka Ako met or exceeded expectations for its primary outcomes. Students who previously had insufficient food, reported on average feeling 20% fuller after lunch than before the programme.⁴ Furthermore, lunches meet Ministry of Health nutritional guidelines and provide more than 33% of daily nutrient requirements for growth and cognitive development.⁵ Diet-related health outcomes include substantial mental health improvements (eg, 9-20% gains for children living in food-insecure homes).⁶ The annual savings for households ranges from $1,000 (1 child in primary school) to $5000 (3 children in secondary school) and qualitative evidence describes reduced whānau hardship, food insecurity and time burden.⁷ Local economies benefitted with about 2,500 new jobs created.⁸

Qualitative studies reported reduced barriers to education and most schools (46%-86%) reported that student attendance, engagement, behaviour, and achievement improved after the introduction of the programme.⁹ A newly-published report noted improved attendance for underserved children in some school terms amounting to about 3 days/year.¹⁰ Several New Zealand cross-sectional studies showed that food insecurity was associated with large impacts on test scores, equivalent to losing 2-4 years of schooling.^11,12

2. Economy (Is it buying inputs of appropriate quality at the right price?)

The evidence on quality measures for the management of resources (procurement, provision, distribution) shows good adherence with all relevant government and internal Ministry policies. A fair balance is struck between costs to government versus the quality of lunches, pay for staff, and profit for providers. Staff are paid a living wage¹³ and there is regular calibration of prices paid to suppliers¹⁴ and flexibility for small and remote suppliers.¹⁵ Complaints about food appeal and quality are rare (88 and 201 respectively in 2.5 years from approximately 1 million lunches/week).¹⁵ 

3. Efficiency (How well are inputs converted into outputs?)

The main barrier to improved efficiency is a lack of certainty about the programme’s continuation. Multiple procedures and policies are in place to ensure food safety, acceptability, age-appropriateness, and cultural appropriateness.¹⁶ The universality of lunch provision within schools contributes significantly to programme efficiency. Food surplus (untouched lunches) has improved and is now within public service standards (5-10%).¹⁷ Almost all surplus lunches go to students or families in need or are otherwise used. There is no reliable data on food waste (partially eaten lunches) or packaging waste, but this is now built into monitoring systems.  

4. Equity (How fairly are benefits distributed?)

The schools in the programme have students who are the 25% most disadvantaged, according to the Equity Index. This is fair given the total budget, but it means that the programme does not reach about 60% of students from food insecure households.¹⁸ The universality of lunch provision within a school is strongly supported by principals and whānau to minimise food poverty stigma.⁷ 

5. Cost-effectiveness (How much impact does the programme have relative to the inputs invested in it?)

A formal cost-effectiveness analysis has not been done but a Canadian study estimated that free school lunches return 2.5-7-fold on investment. The 20% increase in mental health scores among the students with food insecurity in Ka Ora, Ka Ako is a substantial intangible benefit. 

Conclusions

In evaluation work to date, Ka Ora, Ka Ako is currently performing very well against 21 stakeholder-determined criteria. The major improvements identified are getting secure, ongoing funding for the programme, identifying ways to expand the programme so that all children in Aotearoa New Zealand will benefit, undertaking a formal cost-effectiveness study, and building in more environmental sustainability.  

Authors details 

Prof Boyd Swinburn, Professor of Population Nutrition and Global Health, University of Auckland and Research Professor, EIT/Te Pukenga
Carolina Mejia Toro, Doctoral candidate, Population Nutrition and Global Health, University of Auckland

The authors are involved in the Value for Investment project funded by the National Science Challenge. Professor Boyd Swinburn is also co-chair of Health Coalition Aotearoa.

This is the third in a three-part series. Read Evidence for free school lunches: The impact of hunger and learning and Evidence for free school lunches: Bigger picture benefits.