Last week, Eduflack opined on where all of the Reading First profiteers have gone. (http://blog.eduflack.com/2008/05/21/calling-all-rf-profiteers.aspx) As the program is under siege and the funding has dried up, those who personally profited the most are nowhere to be found. A word of thanks to the Core Knowledge blog for throwing some additional spotlight on the important issue.
Over the weekend, we received an interesting comment from Richard Allington, the former president of the International Reading Association. Sure, Allington has long been tagged as a RF opponent, but no one can question that he understands the concept of scientifically based reading research.
His posting no doubt got me thinking. But more importantly, it got Reid Lyon thinking. As a godfather of RF, Reid definitely knows what he is talking about, and the volume of his RF conversation has increased dramatically in recent weeks. And it is important that we listen.
So without further ado, Reid Lyon’s response to Allington’s thoughts on RF profiteers …
“I believe that these interchanges among individuals with different perspectives on Reading First are helpful, as improvements are impossible with productive debate. In my mind, the debates are more productive when sufficient details are presented to support a particular point of view. Riccards brings up the detail that publishers and vendors were selling to districts and schools before the Technical Assistance Centers were ever established. He is correct,. Many did not need a “list” to garner a substantial amount of reading First funding. Bob Sweet and I predicted that when the legislative language for Reading First was softened to its use of the “based on” criterion, that a feeding frenzy would ensue with everybody and their brother hawking a program based on SBRR.
Like Allington, we felt in drafting the initial language requiring program-specific language that publishers and vendors would be highly motivated to test their products. That still has not happened. I need more details on which programs were “banned.” I know that Chris Doherty was compelled by the law to not fund programs with no basis in SBRR and he followed that law. The Wright program was not funded because it was not comprehensive and did meet additional criteria in the law. The Wright program, to its credit, attended to the reviews of its product and made substantial changes so that it now meets all criteria.
Allington may be talking about Reading Recovery as a “banned” program but Reading Recovery was funded by some states using Reading First funds. The allegations made by Success for All are baseless as indicated by no findings by the OIG of that product being placed at a disadvantage in either its first major auditing report or its audit of New York State. There has been absolutely no evidence of any state or district being pressured by the Reading First office to either drop SFA or not implement SFA. In fact, emails between different state’s Reading First officials, SFA, and a Technical Assistance Center reveal substantial positive interactions in trying to ensure that SFA could participate fully in Reading First.
In addition, a review by Elbaum et al. (2000), it was found that gains for the poorest readers were often minimal, which Elbaum et al. suggested may be related to the need for more explicit instruction in decoding. A recent meta-analysis also found that RR was effective for many grade 1 students (D’Agostino & Murphy, 2004). This study disaggregated RR outcomes by whether the outcomes involved standardized achievement tests or the Observation Survey, which parallels the RR curriculum. It also separated results for students who successfully completed RR (i.e., met program criteria and were discontinued) versus those who were unsuccessful or left the program before receiving 20 lessons (i.e., were not discontinued) and according to the methodological rigor of the studies. When the comparison group was low-achieving students, average effect sizes on standardized achievement tests for all discontinued and not discontinued students were in the small range (.32), and higher for discontinued (.48) than not discontinued (-.34) students. This finding was consistent with Elbaum, Vaughn, Hughes, and Moody (2000), who reported that RR was less effective for students with more severe reading problems. D’Agostino and Murphy (2004) found that analyses based on just the more rigorous studies included in their meta-analysis in which evaluation groups were more comparable on pretests showed smaller, but significant effect sizes on standardized measures. Disaggregation according to whether the student was discontinued or not was not possible. Effect sizes were much larger for the Observation Survey measures, but these assessments are tailored to the curriculum and also have severely skewed distributions at the beginning and end of grade 1 that suggest the Observation Survey should not be analyzed as a continuous variable in program evaluation studies (Denton, Ciancio, & Fletcher, 2006).
By assessing in greater detail the degree to which well defined groups of students respond positively to well defined interventions, we increase the likelihood that particular programs will be implemented in a more thoughtful manner rather than as a magic bullet – and this is the case for all programs.
Allington also concluded that the IES Interim Report on the Reading First Impact Study should be the final word on the effectiveness of the program. Details are critical in drawing this conclusion and they are missing in both Allington’s statement and in the media coverage on the report. Two details are noteworthy – the sample is not representative of the universe of all Reading First schools nationally, and the ability to draw meaningful conclusions about the null results is very limited due to the contamination between Reading First and Non-Reading First schools with respect to shared professional development and common instructional programs.Allington has jumped to faulty conclusions in the past before. Recently he asked the field to read two invited papers in an issue of the Elementary School Journal that he edited that ostensibly overturned the results obtained by the Phonics Subgroup of the NRP. However, a formal replication of both these two studies published in a top ranked peer reviewed archival journal (Journal of Educational Psychology) did not support the conclusions of either paper regarding the impact of systematic phonics instruction on reading outcomes. This is science at its best when replication adjudicates claims arising from publication of data particularly when the process is characterized by mature scientific dialogue.
I predict that the jury is not yet out on the effectiveness of Reading First. Who knows, if the evaluation carried out By IES actually aligned with the evaluation required in the law, more detail would have helped us interpret the results with greater confidence. But I bet that even if these flawed comparisons showed Reading First Schools to be superior to non-Reading First schools, many would have argued that Reading had not been in place long enough to make these claims.”
The saga continues. Dr. Allington, I’ll offer you a chance to respond, if you are so inclined.