I'm commenting on the new evaluation guidance, specifically the recommended budget allocations. My experience is with LPS but I think my comments would be relevant to other grant streams too.
The previous guidance (2014) stated that HLF grant of evaluation was “limited to 1% of project costs for grants of £2m or more, and up to 3% for grants below £2m”.
The new guidance states on p19 that we should spend;
“£1m+ projects Spend on evaluation: 7-10% of total project costs and consider use of external supplier.
Use of baseline: Development of baselines & tracking against them
Acceptable Methodologies: Advanced qualitative and quantitative research
Outcomes: Focus on all levels of objectives”
(for £250k - £1m guidance is 5-7%)
NB the guidance is for a %age of total project costs, not total grant.
So the new guidance represents an increase in evaluation budgets by an order of magnitude. For a typical £2m+ LPS this would give an evaluation budget of ~ £200k or even more. I have no idea how one could possibly spend this much. The evaluation consultant we used on our last LPS, who has done a lot of work for HLF projects, has no idea either (though he’d be very happy to get the cheque!). I'm not even sure there is the consultant capacity to deliver a 10x increase in evaluation work.
Use of an external supplier is only to be “considered”… without using an external supplier you’d be hard pushed to spend even a fraction of £200k, and anyway for a £1m+ project I’m pretty sure HLF would require external scrutiny, they certainly ought to.
Thinking of the organisations and individuals on a typical Partnership Board, particularly community and Local Authority representatives, I think they’d be very unimpressed indeed at this level of expenditure on evaluation. They would question whether this was an appropriate use of public money and whether a project spending this much on evaluation was deserving of their support. I know this to be the case because I’ve had informal conversations with a few such people who were on the old TtT Board.
This change adds 10% to the costs of a project at a stroke, and the extra is all overheads. Evaluation is not something that a project partner is going to explicitly give partnership funding towards. So 10% of the budget going to evaluation will also mean a 10% reduction in the percentage of match funding and a 10% increase in the % of HLF grant. We know that HLF doesn’t often support projects asking for grant at the maximum possible %, so how will this step change in evaluation costs be considered?
The rest of the new guidance is all helpful, and I hope no-one needs convincing of the importance of evaluating projects (during as well as after) and sharing the results. The guidance does not, however, address a key issue for evaluation which is which is how to disseminate and share the results – even more important if you’re going to spend £0.25m on them.
Increasing the recommended evaluation budget by an order of magnitude without explanation of either the rationale for doing so, or an idea of how to spend so much money on this, is unhelpful and frankly a bit baffling.
HLF, please explain! Thank you.