Special to the Philanthropy Journal

By Phil Buchanan

When it comes to assessing nonprofit performance, stereotypes and caricatures often get in the way of good practice. I can’t tell you how many times I’ve heard it said that nonprofits aren’t that interested in assessing their performance. This was definitely the prevailing view when I was a student at Harvard Business School two decades ago, and it remains so today – including, unfortunately, among some major donors and foundation staff. I hear donors talk about how nonprofit leaders don’t care about performance assessment – and need to be held to account by donors.

But this is wrong. Several years ago, my Center for Effective Philanthropy (CEP) colleagues and I surveyed foundation-supported nonprofits about their practices, and this is what we found:

• Almost all nonprofits we surveyed report collecting information to assess their performance; still, many nonprofit leaders want to collect additional—or better—data.
• The nonprofits surveyed are mainly using their performance information to improve their programs and services, inform their strategic direction, and communicate about their progress; to a lesser extent, they are using it to share what they’re learning with other organizations or to manage staff.
• A minority of nonprofits report receiving support from foundations for their performance assessment efforts.

“Critics sometimes describe nonprofits as uninterested in—and incapable of—using data to improve their work,” CEP’s Ellie Buteau and her colleagues wrote. “But our findings paint a picture of organizations trying to understand their performance and using the information they are collecting to make improvements.”

The simple reality is that performance assessment is more challenging for nonprofits than businesses. In business, performance is ultimately captured in financial statements. Not so for nonprofits. As management guru Peter Drucker – who, unlike many contemporary business gurus actually understood the nonprofit sector – put it well, “performance and results are far more important – and far more difficult to measure and control – in the nonprofit institution than in a business.” The fact is, they don’t generally teach the measurement techniques and approaches that are needed in the nonprofit sector in MBA programs.

Again, nonprofit leaders get this, and they care deeply about understanding their performance and impact. Most of the nonprofits we surveyed in our research do their level best to get the data they need, including collecting regular feedback from their intended beneficiaries. It’s not that nonprofits aren’t committed to assessing and improving performance. It’s that they need much more support and resources to do this work to the extent they seek to.

Although there are donors that do a great job supporting nonprofits in their work to assess performance, their numbers remain too few. Even those working at foundations in assessment roles agree. In a CEP survey of staff leading evaluation at larger foundations, 69 percent said that their foundations don’t do enough to improve grantee capacity for data collection or evaluation.

The place to start, as always, is by listening—seeking to understand what data the nonprofit believes it needs to learn and improve—because there is no one-size-fits-all set of measures. Let’s take two examples to illustrate the vastly different approaches to assessment that make sense in different contexts: a children’s museum and a youth development organization working with at-risk kids.

Different Strokes

The first organization, the children’s museum, has the more straightforward assessment challenge. Its mission is to “to bring to life the joy of discovery for children and their families through fun, creative, hands-on exploration of the world around them.” What data, then, would be important for the board and staff of the museum to gather to gauge performance? Obvious possibilities include number of visitors and surveys of parents and children—to see, for example, if they really are experiencing the museum as fun, creative, and hands-on. Ideally, the museum would collect both over time and com- pare them to other children’s museums in similarly sized cities.

But that alone isn’t enough. The staff and board would probably also care about the mix of visitors. Is it only affluent families who visit the museum, or is there socioeconomic diversity among patrons? This kind of focus on access might be in some tension with another important objective, which is that the museum be economically viable, perhaps generating enough revenue to cover costs, build an operating reserve, and allow for investment in new exhibits to ensure that the museum remains fresh and relevant.

So, presumably, the museum’s leaders would need to monitor the mix of earned and contributed revenue relative to some agreed-upon goals, and, inevitably, there would be fairly complicated choices to be made about pricing. They would need to discuss the use of the facility for revenue-generating special events as well as the degree to which they should offer discounts or scholarships to museum programs for those unable to pay. The museum would need to balance the economic realities with the desire to serve all children, and it would need fundraising to close the gap between earned revenue and costs.

The museum might also be wise to monitor a range of information about its employees, including qualifications, turnover, diversity, and employee engagement and satisfaction—the kind of measures any high-functioning organization, for-profit or nonprofit, would want to monitor.

But the museum likely wouldn’t seek to monitor any long-term changes in the life trajectory or educational outcomes of the children who visit it— even those who enroll in its summer day camps or other recurring programs. Given the relatively small role a museum would play in any person’s life, it wouldn’t make sense for the museum to hold itself accountable for that level of influence on a person, nor would it be practically viable to try to gather those data.

Measuring Impact on Lives

Assessment isn’t simple for the children’s museum, especially as it seeks to balance its need for financial sustainability with its desire to be open and accessible to all. But it’s more straightforward for the museum than it is for many nonprofits.

Take, for example, a youth development organization that aims to alter the life trajectories of those it serves. Roca, based in Boston, has a mission “to disrupt the cycle of incarceration and poverty by helping young people transform their lives.” That’s a big goal to hold itself accountable for, and Roca takes it seriously. Operating in twenty-one communities, the organization runs a “four-year intervention model . . . based upon the theory that when young people are reengaged through positive and intensive relationships they can gain competencies in life skills, education, and employment that keep them out of prison and move them toward living out of harm’s way and toward economic independence.”

An organization like Roca monitors a range of short- and longer-term indicators. In the short term, indicators include retention in its programs. But, unlike the children’s museum, an organization working with at-risk youth to change life outcomes must monitor those it no longer serves—to track employment and incarceration rates, for example. Given its mission, it only knows if it’s successful based on how its past clients’ lives turn out.

While investing energy and resources in that monitoring, the youth development organization must also pay attention to its day-to-day operations: ensuring financial stability, retaining and engaging its staff, and so on. On these more operational dimensions, the children’s museum and youth development organization may monitor some of the same metrics. But on the dimensions that are more closely connected to mission, the performance assessment approaches will be very different.

These are just two examples. Policy-oriented nonprofits face a different set of assessment challenges, as do multiservice human services organizations offering a wide range of programs serving a variety of populations. The point is that the right approach to nonprofit performance assessment varies widely.

Embracing Complexity

It’s time for nonprofits and donors to embrace the complexity of nonprofit performance assessment once and for all. This means turning our back on simplistic metrics designed to compare vastly different organizations working toward vastly different goals in vastly different contexts. I am talking about simplistic overhead ratios. I am talking about absurd and dumbed down measures that some “venture philanthropy” outfits have used, such as “lives touched.” I am talking about the seemingly never-ending quest for an analog to return on investment or profit, metrics by which you can compare performance of vastly different companies working in different industries.

It’s time to recognize that nonprofit performance assessment is a unique challenge requiring unique skills and approaches. And that they don’t teach those skills and approaches in business schools!

It’s time to understand that, rather than seeking to hold nonprofits’ feet to the fire, donors and foundations should ask how they can help nonprofits collect more and better data to improve their performance. The focus should be on support and collaboration. Givers should assume good faith and good intentions on the part of those nonprofits they support until there is reason to believe otherwise.

From this basis of trust, performance assessment can become an avenue for learning and improvement. The result? More impact on people, communities, and issues. And that, after all, is what it’s all about.

---

Phil Buchanan is president of the Center for Effective Philanthropy, a regular columnist for The Chronicle of Philanthropy, and author of Giving Done Right: Effective Philanthropy and Making Every Dollar Count, published this spring by PublicAffairs/Hachette. This article is excerpted and adapted from that book and reprinted with permission. Follow Phil on Twitter: @philxbuchanan