Along the way, we have picked up a few key pointers.  There is a difference between quantitative and qualitative research, with the former a stronger measure of effectiveness.  A medical model with control groups is the ideal, but is also impossible to achieve in the schools, as no student or no class wants to be the one that receives no instruction at all, limited to sit there like a bump on a log as every other child learns.  And methodology and documentation is king, particularly so others can scrutinize the process and replicate the research study to quiet the doubting Thomases.

When the National Reading Panel released its Teaching Children to Read report, it generated a firestorm of reaction and overreaction.  Many questioned the personalities involved.  Others scrutinized the methodology, and some the findings themselves.  So it fell to Rutgers University Professor Greg Camilli to replicate the research.  He applied a broader, more appeasing methodology, and found the same essential results as the NRP.  So the research debate ended (and the field focused on fights over personalities and implementation.)

When the Institute for Education Sciences released its Reading First Impact Study last year, it was seen as the final nail in the RF coffin.  Here was the gold standard in research models — IES — finding that there was no measurable impact of the high profile reading initiative.  But those who take a closer look at the research saw real problems in the methodology.  First, the study did not take the issue of “contamination” into question, all but saying that students in RF schools received different instruction, different textbooks, and differently trained and supported teachers than those in non-RF schools.  We know that is not the case.  But even more damning is that the Impact Study looked at a relatively small sample size to reach its conclusions, and did so in a way that the research can never be replicated.  There is no public record of what actual schools were studied.  And there is little hope that such information could ever be obtained through Freedom of Information Act requests.  So hundreds of millions of dollars in education policy has been decided by a study with questionable methodology that can never be validated and replicated by another researcher, as the field scrambles to try and validate (or invalidate) the findings.

Why all of this background?  To show that the quality of research and the transparency of the process is key.  We trust that the evidence will guide us to strong policy and funding decisions.  We look for the data, believing the numbers will serve as a compass pointing toward student achievement.  And despite Mark Twain’s warnings about statistics, we still hold them to be a primary driver in our decisionmaking.  Citing data and research studies is often the final piece to closing a deal in education improvement these days.

Which is why the most recent pronouncement coming out of Washington, DC Public Schools is all the more problematic.  By now, most know of the battle between DCPS Chancellor Michelle Rhee and the American Federation of Teachers over the issues of tenure and teacher pay.  For more than a year now, Rhee has been pushing the notion of dramatically increasing salaries for effective teachers (assuming the give up tenure) and has secured outside, private funding to accomplish the pay raises.  Critics have attacked her for many reasons, one of which is the sustainability of such pay raises.  What happens when the outside foundations or corporate sponsors move on to the next issue?  How will DCPS be able to sustain the new, higher pay structure?

On a radio program yesterday, Rhee stated she had a research report from an economic consultant showing that the plan can indeed be sustained.  But she won’t name the consultant.  And she won’t release the report.  She wants us simply to take her word that she can make up the supposed $100 million pledge to DCPS for pay raises if need be, while waiving an unnamed report from an unnamed researcher to prove her point.

The full story can be found in this morning’s Washington Post — www.washingtonpost.com/wp-dyn/content/article/2009/03/02/AR2009030202785.html?hpid=moreheadlines  

Let me say for the record that Eduflack believes that Rhee’s teacher pay structure can indeed be sustained.  Quite honestly, she has no choice.  If she signs a contact with the Washington Teachers Union outlining a pay structure that will offer $135,000 salaries to non-tenured teachers, she needs to honor the CBA, regardless of changes in her financial pipeline.  A deal is a deal.  (Though Eduflack seriously doubts that AFT will ever agree to the deal, at least as it has been presented to date.)  She’s resourceful and will find the funds from other donors, or from within the DCPS budget itself if necessary.

But waiving around an unnamed research study that supposedly proves your point, no questions asked, but refusing to provide details, identifiers, or even the study itself is just amateur grandstanding.  The “I have in my hand” approach asks us to trust Rhee when, quite frankly, she hasn’t earned the trust of those she is seeking to reach.

If Rhee wants to show her teacher pay plan is sustainable, she needs to release her research study know and get it into the hands of every member of the city council and every leader at WTU and AFT.  And she needs to get its toplines to every single teacher in DCPS.  She should be making the data case now.  If she has the research, post it online, distribute it at DCPS headquarters, heck, hand it out to everyone coming to visit the Lincoln Memorial.  Get the data out there, let it speak for itself, and let your opponents see the true strength of your argument.

Trying to sidestep a major question like sustainability with a “Trust me, I’m with the government” approach just doesn’t cut it in the new era of 21st century school improvement.  Our schools, educators, and students have been sold a lot of vapor in recent years.  Victory comes to those who can prove their point, and have the data to back it up.  Until Rhee releases this economic study on the sustainability of her pay proposal, she can’t win the day.