The Heritage Lottery Fund (HLF) commissioned an independent review of the self-evaluation process and outcomes achieved for 78 completed Our Heritage projects completed between January 2015 and April 2016.

This review includes a comparative appraisal of the quality, scope and methodology of the self-evaluated reports against six criteria and the type, range and quality of activities and outcomes achieved by completed projects, including a mapping of the impact of their work onto HLF’s current framework of 14 outcomes for heritage, people and communities.

Key findings include:

Quality of the Self-evaluations

The quality of the submitted self-evaluation reports was assessed on a four-point scale (excellent, good, adequate or poor) using six criteria. These six criteria focused on the extent to which the evaluation:

1. Provided a logical framework
2. Included appropriate and methodological ways of providing robust evidence
3. Demonstrated that data was subject to robust analysis and provided evidence on outcomes
4. Was objective and free from bias
5. Presented the results clearly
6. Included sufficiently clear conclusions and recommendations to enable stakeholders to apply any lessons learned.

Overall just over a third, 37%, of reports were graded as good or excellent with just under two thirds, 63%, falling within the adequate or poor categories.

The aggregated findings in this report show that the quality criteria scores have significant dependence on several characteristics:

• Evaluation spend: Just under half of the projects asked for a specific budget to cover evaluation. Projects which had allocated a specific budget for evaluation were more likely to be graded as good or excellent and less likely to be graded as adequate or poor.
• Length of reports: The median length of reports found to be poor was just 4 sides.  The median length of reports found to be adequate was 8. The median lengths of the good and excellent reports were 18 and 35 sides respectively.  
• Use of HLF evaluation guidance: Reports that indicated they used HLF guidance were more likely to be excellent or good. Reports that did not indicate they used HLF guidance were more likely to be poor.  

Outcomes review

The Our Heritage guidance to applicants notes that the outcome valued most is that “people will have learnt about heritage” which is described as a weighted outcome.  92% of the reports indicated that this outcome had been achieved.  

The Our Heritage programme overall contributed most to the outcomes: volunteered time; learnt about heritage; better interpreted and explained; had an enjoyable experience; more people and a wider range of people will have engaged with heritage. Fewer Our Heritage projects were found to contribute to the following elements of the HLF outcomes framework: your local economy will be boosted; your organisation will be more resilient and your local area/community will be a better place to live, work or visit.