More and more people are promoting data ‘standards’ for impact measurement and ‘social value’, to be able to describe programme interventions in terms of predictable social effects. Within Social Value and impact investing circles, there’s an ambition to get to a point where we can look at a raft of social interventions and estimate the likely social return on investment from given inputs. People hope that data standards will help structure options for decision-makers: ‘if we invest X, the return will be Y’.
If corporations and other social investors are going to base their decisions on data about inputs, outputs, and value, they need more and better data they can use. To change investor or corporate behaviour, we need to be able to factor in potential harms, to assess pricing and make decisions on this basis. Without standard metrics assigning value, how do you make them count? Further, there’s an incentive for standards that can improve the data supporting the birds eye view: reviewing the level of resource going into parts of the social sector, to make the case for a different magnitude of investment, for example.
So, as well as building knowledge, standards are hoped to give evaluation evidence more traction in the real world of business and government investment.
The flipside of this is the lack of realism that can be involved in these measures. In evaluation, we deal with the question of attribution, we acknowledge context, who an intervention is for, and how it is delivered, and these things temper our conclusions. They mean you can’t make abstract predictions about inputs to outcomes, and we should acknowledge models are often wrong and can even do harm. The evidence available is never enough. We can’t take judgement out of how we apply the evidence, nor lose accountability for that.
How can we standardise data in ways that are useful for a set of interventions and at the field level?
We are asking:
- Can we get greater clarity on the benefits and risks of standardisation in interventions?
- How can a purposeful, realistic agenda for standards be advanced?
- Can we find ways to better acknowledge the limitations of standards while embracing the opportunities?
- Can we agree what their best applications are, and where they should be avoided?
- Is the issue that we are using inaccurate means of coming towards usable standards, or that the wrong mix of people are involved?
Better collaboration between impact management, social value and evaluation fields may help us answer these questions. We’d love to know what you think in response to these questions, comment below with your thoughts, ideas and what else we should be asking…