by Rob Dugger, PhD
Managing Partner, Hanover Provident Capital, LLC
Co-Founder, ReadyNation
As the co-founder of ReadyNation and a stalwart supporter of quality early learning, I have great respect for the Pay for Success partnership that made Utah’s vision for quality preschool a reality. I also agree with Bill Crim’s assertions that the recent New York Times article questioning the recent assessment of the program’s impact in Salt Lake County was driven by several misleading comparisons of data.
Fortunately, I also see a silver lining in the debate about the program and its evaluation: while the educators, evaluators and funders work in partnership to improve outcomes for participating kids, they will continue to do so largely at the expense of the private sector. In fact, I highly doubt we would see this type of rigorous evaluation and examination of the results if this were simply a traditional state or federal program.
This is a great deal for the state of Utah and its taxpayers, and an even better benefit for the kids, the parents, the public and everyone who wants Utah’s kids to be ready-to-learn when they start school. One of the most sensible assets of the Pay for Success model is that it pays for what works. And if programs don’t work – meaning they don’t meet tangible, predetermined measures of success – they can be improved at the expense of the funders, or even discontinued before more taxpayer dollars are invested.
The good news is that the program in Utah is working – 99 percent of the participating children who were identified by a reliable assessment as at risk for needing special education in kindergarten did not. The even better news is that this reduction in special education – which also amounts to a reduction in costs borne by taxpayers – is but one benefit of quality preschool. Study after study shows links to longer term academic achievement, high school graduation, reduced involvement in the criminal justice system and more.
The Pay for Success model is not just about structuring effective public-private partnerships and giving more at-risk children needed services. It’s meant to be a game changer in how government conducts the business of remediating and preventing society’s problems and building a productive, healthy citizenry. This is exactly what the Utah Pay for Success model is doing. It has promoted early childhood education systems change in Utah. Programs are using data to guide decisions, improve practices and learn faster about what works best for children. It has also focused government on outcomes in a way that will better use taxpayer resources.
Debates about the methodology for determining success are nothing new. It’s common for cutting edge research results in any field of study to be diligently examined and commented on with respect to methods and the relationship to outcomes. I also want to note that the desired outcomes and outcome metrics for the success of this program were developed to reflect what the educators, program implementers and funders agreed was in the best interest of all stakeholders, particularly the children and their families.
It’s important to highlight a few technical points since the New York Times article discussed technical issues that researchers examined.
- Many special education and early childhood research experts support the methodology Salt Lake County used.
- An experimental design was not possible in the Salt Lake County Pay for Success program. The question is what do we do in Pay for Success when an experimental design isn’t possible? We need to remember why we are doing this: to expand access for low-income families to high quality preschool.
- Although not an experimental design, using the PPVT is a reasonable way of determining which students are at-risk for school failure and potential later special education placement. Research has shown that 3- and 4-year olds who score that low, without intervention, will score that low when entering kindergarten. That is, children who start behind tend to stay behind.
- Projected and actual special education avoidance rates in the range that Salt Lake County achieved are not unreasonable. A study of Pennsylvania’s Pre-K Counts program funded by the Heinz Endowments showed a comparable reduction of special education assignment of almost 90%.
- Typically, if Dual Language children are delayed in one language, they are delayed in both languages. And, architects of the program tell us that the pre-schoolers tested were not only Spanish-speaking but dual language learners.
- There is no perverse incentive in the Salt Lake County case for diverting children from special education if they would qualify based on assessment. That’s because special education assignment in kindergarten through grade 12 is blind to how a child was funded in preschool. Only the independent evaluator has the data on how a child scored on the PPVT at preschool entry.
And although I agree that the results reported to date are both accurate and indicative of the program’s success, I will be interested to see what others write about the results and if their challenges are sufficiently supported. This is the kind of healthy tension that will help to inform and strengthen the “Pay for Success” model.
In the meantime, Utahns should also note that although funders did receive a return based on the 109 children who did not need special education, that funding made quality preschool available for 595 low-income children who otherwise probably would not have been able to attend preschool. In fact, by the time this Pay for Success project has been completed, a total of 2,600 children will have the opportunity to experience quality early learning that otherwise would not have attended preschool. This is the kind of bottom line impact that should be celebrated as Utah and other states and communities consider Pay for Success strategies that foster social innovation and real life results for children, families and society as well.
