One of the trends of the past few years, which is all over the non-profit sector, is the move towards evaluation and assessment of programs. Just a couple of weeks ago Hazon announced that they have just embarked on a comprehensive, first-time research study of their programs and the connections between food, outdoor education and Jewish identity development. A very exciting and large scale project. And overall, this focus on evaluation isn’t a bad thing at all. On the contrary, it is absolutely about time that organizations think one step ahead when they plan their programs, and make a real commitment to learning about the success and impact of those programs. It isn’t enough (any more) to have a good idea and raise the money to implement it. It isn’t even good enough to do a really good job of implementing. There is a really crucial need to ask questions about success, and figure out if the program is actually reaching its goals.
Last week I participated in a seminar run by the Mandel Leadership Institute Graduate Unit, all about evaluation and “Strategic Learning” in organizations. The seminar, led by Dr Kathryn Newcomer, the Director of the Trachtenberg School of Public Policy at George Washington University in DC, was a really interesting dive into some of these issues of evaluation, particularly in the sphere of non-profit organizations. The claim made by Dr Newcomer was that we need to distinguish between two approaches to evaluation. On one hand there is evaluation for accountability purposes. In this case, the donor or funder has a set of questions that are designed to provide some accountability for the money that is being used. Did the money get spent in the right way? Was the program run properly? Did the project meet its stated goals? All these kinds of accountability questions usually ask for a fair amount of basic data and can be very onerous on the grantee. This is unfortunate but probably unavoidable. Public agencies and donors have the right to ask about the use of their money. Yes, I know from my own experience that it is sometimes very difficult, at the best, and tiresome and intrusive, at the worst, to provide this kind of data, but it is really unavoidable, and quite honestly, if I were a funder, I would ask the same questions. But, that is only one kind of evaluation approach. The second is about evaluation that is really about strategic learning for the organization. It is focused internally, on providing answers to really important questions that the organization should be asking itself. Did our theory of change really prove itself? Were our assumptions about what we believe we are doing correct? What did we really do well, and not well? And what do we need to know in order to do better? We don’t always want to share the answers to these kinds of questions. In fact, in order to really probe them, we need to know that we will be protected from the negative data we might discover. But, if we really believe in organizational learning and on honest evaluation, this is what we must do.
So, for an organization interested in true strategic learning, you might need to start with evaluation for accountability but ask yourself whether you can evolve that data into an evaluation for the purposes of strategic learning. And then see what really emerges.