The Saga of RF Profiteers Continues

Last week, Eduflack opined on where all of the Reading First profiteers have gone.  (http://blog.eduflack.com/2008/05/21/calling-all-rf-profiteers.aspx)  As the program is under siege and the funding has dried up, those who personally profited the most are nowhere to be found.  A word of thanks to the Core Knowledge blog for throwing some additional spotlight on the important issue.

Over the weekend, we received an interesting comment from Richard Allington, the former president of the International Reading Association.  Sure, Allington has long been tagged as a RF opponent, but no one can question that he understands the concept of scientifically based reading research.

His posting no doubt got me thinking.  But more importantly, it got Reid Lyon thinking.  As a godfather of RF, Reid definitely knows what he is talking about, and the volume of his RF conversation has increased dramatically in recent weeks.  And it is important that we listen. 

So without further ado, Reid Lyon’s response to Allington’s thoughts on RF profiteers …

“I believe that these interchanges among individuals with different perspectives on Reading First are helpful, as improvements are impossible with productive debate.  In my mind, the debates are more productive when sufficient details are presented to support a particular point of view.  Riccards brings up the detail that publishers and vendors were selling to districts and schools before the Technical Assistance Centers were ever established. He is correct,.  Many did not need a “list” to garner a substantial amount of reading First funding.   Bob Sweet and I predicted that when the legislative language for Reading First was softened to its use of the “based on” criterion, that a feeding frenzy would ensue with everybody and their brother hawking a program based on SBRR. 

Like Allington, we felt in drafting the initial language requiring program-specific language that publishers and vendors would be highly motivated to test their products.  That still has not happened.  I need more details on which programs were “banned.”  I know that Chris Doherty was compelled by the law to not fund programs with no basis in SBRR and he followed that law.  The Wright program was not funded because it was not comprehensive and did meet additional criteria in the law.  The Wright program, to its credit, attended to the reviews of its product and made substantial changes so that it now meets all criteria.
 

Allington may be talking about Reading Recovery as a “banned” program but Reading Recovery was funded by some states using Reading First funds.  The allegations made by Success for All are baseless as indicated by no findings by the OIG of that product being placed at a disadvantage in either its first major auditing report  or its audit of New York State.   There has been absolutely no evidence of any state or district being pressured by the Reading First office to either drop SFA or not implement SFA.     In fact, emails between different state’s Reading First officials, SFA, and a Technical Assistance Center reveal substantial positive interactions in trying to ensure that SFA could participate fully in Reading First.

There are two points that Allington makes where more detail would be very helpful.  First, Allington makes the point the WWC found that Reading Recovery  (RR) has strong evidence that it improves general reading achievement.    This is a very general statement.  My colleagues and I have published a number of papers over the past several years addressing the effectiveness of Reading Recovery and in each review concluded it was effective – for some. Concerns about the efficacy of RR have been based, in part on whether the program is successful with the lowest performing students – students typically served in reading First programs.    Reading Recovery has typically targeted students who perform in the lowest 20% of their classes.  The actual performance level of participants varies from school to school.  Although the research from the developers of RR continues to indicate efficacy for about 70% of the students in the program ( a very strong degree of effectiveness) , its reported effects are much weaker when students who do not meet the program’s exit criteria are included in the analyses of outcomes (see Fletcher, Lyon, Fuchs, & Barnes, 2007 for review). 

In addition,  a review by Elbaum et al. (2000), it was found that gains for the poorest readers were often minimal, which Elbaum et al. suggested may be related to the need for more explicit instruction in decoding.  A recent meta-analysis also found that RR was effective for many grade 1 students (D’Agostino & Murphy, 2004).  This study disaggregated RR outcomes by whether the outcomes involved standardized achievement tests or the Observation Survey, which parallels the RR curriculum.  It also separated results for students who successfully completed RR (i.e., met program criteria and were discontinued) versus those who were unsuccessful or left the program before receiving 20 lessons (i.e., were not discontinued) and according to the methodological rigor of the studies. When the comparison group was low-achieving students, average effect sizes on standardized achievement tests for all discontinued and not discontinued students were in the small range (.32), and higher for discontinued (.48) than not discontinued (-.34) students. This finding was consistent with Elbaum, Vaughn, Hughes, and Moody (2000), who reported that RR was less effective for students with more severe reading problems. D’Agostino and Murphy (2004) found that analyses based on just the more rigorous studies included in their meta-analysis in which evaluation groups were more comparable on pretests showed smaller, but significant effect sizes on standardized measures. Disaggregation according to whether the student was discontinued or not was not possible. Effect sizes were much larger for the Observation Survey measures, but these assessments are tailored to the curriculum and also have severely skewed distributions at the beginning and end of grade 1 that suggest the Observation Survey should not be analyzed as a continuous variable in program evaluation studies (Denton, Ciancio, & Fletcher, 2006).   

By assessing in greater detail the degree to which well defined groups of students respond positively to well defined interventions, we increase the likelihood that particular programs will be implemented in a more thoughtful manner rather than as a magic bullet – and this is the case for all programs.

Allington also concluded  that the IES Interim Report on the Reading First Impact Study should be the final word on the effectiveness of the program.  Details are critical in drawing this conclusion and they are missing in both Allington’s statement and in the media coverage on the report.  Two details are noteworthy – the sample is not representative of the universe of all Reading First schools nationally, and the ability to draw meaningful conclusions about the null results is very limited due to the contamination between Reading First and Non-Reading First schools with respect to shared professional development and  common instructional programs.Allington has jumped to faulty conclusions in the past before.  Recently he asked the field to read two invited papers in an issue of the Elementary School Journal that he  edited that ostensibly overturned the results obtained by the Phonics Subgroup of the NRP.   However, a formal replication of both these two studies published in a top ranked peer reviewed archival journal (Journal of Educational Psychology) did not support the conclusions of either paper regarding the impact of systematic phonics instruction on reading outcomes.  This is science at its best when replication adjudicates claims arising from publication of data particularly when the process is characterized by mature scientific dialogue.

I predict that the jury is not yet out on the effectiveness of Reading First.  Who knows, if the evaluation carried out By IES actually aligned with the evaluation required in the law, more detail would have helped us interpret the results with greater confidence.  But I bet that even if these flawed comparisons showed Reading First Schools to be superior to non-Reading First schools, many would have argued that Reading had not been in place long enough to make these claims.”

The saga continues.  Dr. Allington, I’ll offer you a chance to respond, if you are so inclined.
 

2 thoughts on “The Saga of RF Profiteers Continues

  1. Lyon makes a number of reasonable points but I do not expect that waiting longer to observe RF effects is a strategy that deserves much consideration. If, after 5 years any program has no measurable effect on achievement, it seems unlikely that effects will suddenly appear in year 6 or 7 or…. As for 2006 articles in the Elementary School Journal vs. the 2008 articles in Journal of Educational Psychology, this is a case of trying to deduce how many angels can dance on the head of pin. The basic issue debated in these sets of articles is whether the small observed effects the NRP found for systematic phonics instruction, and they were small effects, should have motivated so much federal attention to phonemic awareness and phonics instruction. The ESJ authors say no, the JEP authors say yes. Readers will have to read both sets of articles to decide which side they agree with. I will simply argue that there were many other issues involved in reading instruction that have much larger observed effects and had RF policy attended to those issues/factors the evaluation study might have had a different outcome. We can continue to debate the RF design the IES study, or instead begin the work required to truly improve American reading instruction.

  2. What Mr. Lyon fails to point out and/or understand is that districts receiving RF funding are under such an oppressive microscope from their state DOEs and Technical Assistance Centers that most decision makers from those districts DARE NOT veer from the “recommended” lists that came out of Oregon State and FCRR; therefore, those lists became de facto “approved” lists. If Reading Recovery found its way into some districts under RF, they probably had to jump through unfair, unnecessary, and incredibly time-consuming hoops to do so. Kudos to the Reading Recovery folks who pushed through those partnerships under RF against all odds! The fantastic progress that so many of our children have made under Reading Recovery is indisputable. Only a company with vision and a mission would undertake the extra effort necessary to win a contract that was so unfairly stacked against them. Perhaps if Reading Recovery had moved their headquarters to Houston they could have seen a lot more RF funding coming their way. But alas, too late.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s