The giving season is currently in full swing as charities seek to maximize donations for the 2020 calendar year. However, individuals looking to give are confronted with a profuse landscape of nonprofits to choose from, and guidance on how specific charities measure on different criteria is often welcome. Charity Navigator, the long-trusted US-based nonprofit evaluator, released the beta version of its new Encompass Rating System this summer to much acclaim. This revamped rating system, inspired by the insights from our partnership in 2019, expands the number of nonprofits covered from 9,000 to over 160,000 — nearly a 1700% increase.
Benefits and Limitations of Charity Navigator’s Rating System
For years, donors have used Charity Navigator’s 4-star rating system to inform giving decisions to the organizations in their system, and charities have highlighted their ratings to draw in much-needed funding. But with over 1.6 million nonprofits registered in the US, Charity Navigator knew it could have a greater impact on the charitable sector if it could scale the system that the public knows and trusts. “Once the IRS Form 990 data became digitally available, we needed help seeing our own processes differently,” said Michael Thatcher, President and CEO, “Data Clinic’s engagement and analysis was instrumental for enabling the internal change management required so we could expand both the breadth and depth of our ratings.”
The 4-star rating system was built to help donors ensure that their donations would be used wisely and that they would ultimately be impactful. The system includes seven financial metrics based on the charity’s Form 990 tax returns to quantify how fiscally efficient the organization operates, and how much of donations go to actual programming. It also includes 17 accountability and transparency metrics, 12 based on the 990 tax returns and five from other information the organization might publish publicly, to provide donors clarity and insight into a complex world.
Charity Navigator carefully developed these metrics to take into account differences in operating structures and needs between sectors, organization size, organization maturity, and more. Based on the information needed, only charities with budgets over $1 million that have seven years of 990 history filed were rated (you can read about the full model and other requirements here). This criteria limited the pool of rated charities to large nonprofits with extensive public presence: for example, the American Red Cross, International Rescue Committee, World Wildlife Foundation, and many other household names.
While these organizations continue to play major roles in social impact, the nonprofit sector has evolved rapidly over time in response to the ever-changing challenges that society faces. More and more, smaller grassroot organizations are helping communities build tailored solutions with and for those in need. The number of charities with budgets under $1 million actually accounts for over two-thirds of all registered nonprofits in the US today.
Data Clinic’s Approach to Scaling the Rating System
Could Charity Navigator’s 4-star rating system scale to these organizations to become an even more valuable resource? In spring of 2019, a team of our Data Clinic volunteers set out to uncover its potential.
We rigorously analyzed previous research and built out a pipeline to pull in the raw 990 information from 170k+ charities that filed their returns online and run it through the Charity Navigator rating system. By focusing on the financial metrics, which could be fully automated from the tax returns, we found that simply relaxing some of the restrictions on size, age, and tax return history resulted in a statistically similar distribution of scores, both at the individual metric- and overall financial score-level. With these looser restrictions, Charity Navigator could increase financial ratings by over 630%.
We also built out the framework for a “health check,” which would allow users to compare ratings to a peer group of charities of similar size working toward related missions. While this could provide more nuance for the Charity Navigator audience, it would depend on an organization’s assigned “cause,” which currently relies on time-consuming manual assessment from Charity Navigator analysts.
The results were eye-opening for many on the Charity Navigator team, which had fine-tuned and constructed the current methodology based on a wealth of research and evidence over nearly two decades. “It was one thing for us to have an intuitive sense that scaling was possible,” said Stephen Rockwell, Chief Ratings Technology Officer, “but Data Clinic’s analysis imparted a sense of confidence to our team that we could expand elements of our Finance & Accountability methodology to a significantly broader group of nonprofit organizations.” For the first time, they saw these methods and principles successfully applied beyond their restricted cohort of approximately 9,000 nonprofits.
Change is not easy, especially for nonprofit organizations that function on limited resources. But Charity Navigator’s team came to the partnership with a dedicated learning and growth mindset. They recognized the power in the data-informed insights that came from a combination of our team’s analysis and Charity Navigator’s own guidance and foundational methodology.
Convinced that they had built a system that could ultimately evolve and benefit more nonprofits and donors, Charity Navigator set out to apply and adapt the concepts of relaxed prerequisites and alternative views of performance, resulting in its new (and ever-improving) Encompass Rating System. With 160,000 charities now rated (and counting!), we’re thrilled that this wealth of information could help empower many more donors seeking to strengthen the impact of their dollar.