Nowadays you can hardly throw a rock at the nonprofit sector without hitting the words "data driven," "outcomes based," "evidence based," or some other terminology referring to the desire to understand the efficacy of a given intervention. It makes perfect sense: given the roughly $30 billion / year that is donated to social service organizations, it's critical that we know if those funds are making a difference. However, in the rush to quantify impact, something extremely fundamental has been forgotten: it is shockingly expensive to collect, analyze and report this type of data, and the vast majority of nonprofits lack the funding to do so.
The simple fact of the matter is that when we talk about impact, we are usually referring to outputs: 'X' number of people receiving loans or 'Y' number of budgets built. We may go a step further and say that the average increase in FICO score of our clients is 75 points, which is true (and we are proud of that), but it's also misleading because, as far as we know, that's only true of the clients we are able to reach for follow up surveys. In other words, selection bias--the people most motivated to improve their credit are also the most likely to stay in touch long enough to do a survey--skews the numbers in our favor.
![]() |
| Photo Credit: refinerysource.com |
The simple fact of the matter is that when we talk about impact, we are usually referring to outputs: 'X' number of people receiving loans or 'Y' number of budgets built. We may go a step further and say that the average increase in FICO score of our clients is 75 points, which is true (and we are proud of that), but it's also misleading because, as far as we know, that's only true of the clients we are able to reach for follow up surveys. In other words, selection bias--the people most motivated to improve their credit are also the most likely to stay in touch long enough to do a survey--skews the numbers in our favor.



.jpg)

