The UN Secretary-General’s Independent Expert Advisory Group on a Data Revolution for Sustainable Development (IEAG) have completed their report. “A world that counts” is a cleverly crafted motivational manifest. But it is not a practical roadmap on how to apply a ‘data revolution’ to the sustainable development goals agenda. The report’s key weakness is that it conflates a lot of terms, and assumes automatic relationships between things like ‘counting’ and ‘knowing’. I write about it in more detail here. The mistakes made in the report are fundamental. Here I show some of the basic conflations in terms in four diagrams.
Not everything that counts can be counted
The report strongly suggests that everything that matters can be counted. We know that this is not true. If the guiding principle for the Sustainable Development Goals is to make decisions ‘as if’ everything can be counted, the end result will be very misleading.
Data is not the same as statistics
The data revolution hype is just one of many places where ‘statistics’ and ‘data’ are conflated. Data is not the same as numbers. Data literally mean ‘what is given’, so when we speak of data we are talking about observations – quantitative or qualitative, even figurative that can be used to get information.
To keep talking about data when we mean statistics may sound better, but it only contributes to confusion. The report suggests that the UN should develop a ‘global consensus on data’. What is that supposed to mean? That statement is meaningless if you exchange the word ‘data’ with ‘observations’, ‘knowledge’ or ‘evidence’. It can however make sense if you talk about ‘statistics’. International organizations do have a natural role when it comes to developing global standards for official statistics. Reaching a global consensus on how observations and evidence constitute knowledge is futile.
There are more methods to knowing than through counting
The report says (p. 2-3): “whole groups of people are not being counted and important aspects of people’s lives and environmental conditions are still not measured” and then that “Never again should it be possible to say “we didn’t know”. No one should be invisible. This is the world we want – a world that counts.”
I understand the enthusiasm, but I want to warn against hubris. This is certainly not the world I want. I think it should always be possible to say ‘we didn’t know’. Numbers, or the act of counting does not guarantee objectivity nor does it always make us wiser. It is a testament to the richness of life, and the poverty of numbers that all things cannot be counted.
More data does not mean better decisions
The report says that demanding more data will lead to better decisions. That is a statement of belief, not a a theory of of change. What is often thought of as ‘evidence-based policy’ turns out to be ‘policy-based-evidence’. In short, predominantly the causality runs from policy to statistics. And a lot of evidence does not inform policy at all – whereas a lot of decisions are made without or despite data. That begs the question - how much resources should one advocate that is spent on a global development monitoring framework?