IES

“Read”ing All About It

Eduflack IES, PD, Reading First, SBRR

Today, the final shoe dropped on the Reading First era.  The Institute of Education Sciences released the final version of the Reading First Impact Study.  A surprise to no one, the final impact study came to the same conclusions as the interim study.  The summary of summaries, RF schools aren’t doing a better job of making student reading proficient, compared with non-RF schools.

The full story can be found here at Education Week — www.edweek.org/ew/articles/2008/11/18/14read.h28.html?tmp=1344181825  
When the interim study came out, many, including Eduflack, pointed out the vast flaws in the study’s methodology, chief among them being the issue of contamination, or the impact of RF programs and materials on non-RF schools.  Back in September, the Reading First Federal Advisory Committee issued its review of the interim study, calling for some wholesale changes before the final report was issued.  Unfortunately, little, if any, of the recommendations coming from the Advisory Committee were addressed in the final Study.
I’ve been mulling the issue all day.  As a surprise to many an Eduflack reader, I am not here to once again defend the goals of Reading First and point to the data that demonstrates that scientifically based reading is having an effect on schools, both those receiving RF funds and those that do not.  In the simplest of terms, been there, done that.  I’m a pragmatist.  I know that RF is dead.  It was dead the day the IG report came out almost two years ago, and the find shovels of dirt were thrown on the program with the release of this Impact Study.
And no, we are not here to eulogize RF, to discuss its merits, or to hash out why it failed to meet its promise or fulfill its mission.  Such tasks are best left to the think tanks and the academicians who can give a careful eye to how the research translated into practice, how effective that practice was, and how effective the measurement and feedback of the program was across its lifetime.
The question should not be what happened.  Instead, we must ask what comes next.  How do we move on from here?
The legacy of RF leaves us with three key buckets of policy we must consider — research to practice, a federal reading program, and IES.
At its heart, RF was a thorough attempt to move research into practice.  It was the development part of the R&D equation, an opportunity to take decades of research on literacy and reading acquisition skills and put it to use in the classroom.  How is the research applied to core materials, such as textbooks?  How is the research applied to teacher development, both pre-service and in-service?  How is the research embedded in instruction and in key interventions designed to get all kids reading?  And how does the federal government effectively do it all, guiding SEAs, impacting LEAs, and doing it all without endorsing specific commercial products or approaches?  
On some of these issues, RF provided a blueprint for success.  On others, it provided a clear portrait of federal failure.  Through it all, RF raised the profile of research in the instructional process, better equipped classroom practitioners to deal with education research, and increased the profile of data-based decisionmaking.  All of those are pluses for school improvement efforts moving forward.
Now onto stream two — a federal reading program.  For decades, the federal government has enhanced literacy instruction for K-12 students.  Before RF, we had the Reading Excellence Act. Before REA we had other federal programs.  That commitment is not going to disappear.  Long after RF is forgotten, there will still be dedicated federal investment in reading instruction. The question before us, now, is how do we do it.  How do we transform Early Reading First into a meaningful component of early childhood education efforts?  How do we enhance instruction for struggling readers, particularly in the early grades?  How do we promote literacy skills across the curriculum, using science and social studies in particular to boost reading skills for all?  What do we do for struggling readers in our high schools, those who have fallen through the cracks?  Now is the time to apply lessons learned and build a new federal reading program that delivers instruction to the kids who need it, that provides content-based PD to the teachers in need, and that boosts student achievement and closes the achievement gap for all students, from our urban centers to our rural schools.
And finally, IES.  The RF experiment has clearly demonstrated that IES is not functioning as it was intended.  Was IES tasked with determining the effectiveness of RF or the effectiveness of RF funding?  Has it providing findings that aid in the improvement of federal reading instruction?  is it serving the public good by providing clear research findings that are received, understood, and applied by practitioners in the field?  At the end of the day, IES needs to better serve the consumer — the schools, their teachers, and the students they serve.  It needs to  do a better job engaging the entire community, and not simply serve as a lifeline between educational researchers.  If anything, the RF experience has provided us a starting point for improving IES (and the What Works Clearinghouse) and transforming it into the R&D arm of the U.S. Department of Education, with the D being just as important as the R.
Will we take advantage of these lessons and build some real improvements?  That question will remain unanswered for some time now.  But now is the time we start talking about how we move forward and build on the RF experience.  A new program will rise from the RF ashes.  It falls to the program’s most ardent supporters and most critical adversaries to ensure that what comes remains solidly focused on a singular goal — empowering all kids with the reading skills they need to achieve and getting all kids reading at grade level as soon as possible (and maintaining it).    

Re-Prioritizing the U.S. Department of Education

Eduflack charter schools, early childhood education, ELL, High School, IES, Math, national standards, Obama, Online Education, STEM, Teacher Education

As President-Elect Obama and his Administration-in-waiting begin working through the transition, they have a terrific opportunity to shape the direction of future policy and future successes.  With each new administration, particularly with a change in party leadership, there is the opportunity to reorganize Cabinet departments, the chance to emphasize new priorities and to turn back the efforts of previous administrations.  While Stephen Hess of the Brookings Institution cautions against overhauls and reorganizations at the start of an Administration, now is definitely the time to look at a new organization for the U.S. Department of Education.

In the coming weeks, the Obama transition team will begin reading through the ED transition notebooks, interview staff (particularly the career staff), and quickly making staffing decisions, from EdSec down to a slew of congressionally-approved assistant secretaries.  This is a lot of work, and it will be happening simultaneously in all agencies.  But the amount of work should not keep us from thinking about education — and education improvement — a little differently.
For the most part, the Bush Administration took on the structure that Clinton EdSec Richard Riley left behind.  But if recent years and new thinking are any indication, an Obama Administration may need a very different framework to focus on the issues emphasized on the stump, in policy platforms, and by its strongest advocates.
So how do we do it?  Never shy about such things, Eduflack has a few ideas for the new Obama Administration:
The New Approaches
* Office of Early Childhood Education — Obama has really driven home the importance of early childhood education and its ability to prepare all students — particularly those from at risk families — for the instructional, social, and emotional challenges of elementary school.  The creation of this office systematizes that commitment.  And if you really want to be bold, move Head Start over from HHS and put it under ED, and this new office’s, purview.  While early childhood has long been the official territory of HHS, ED has always had a chip in the game, and Obama’s priorities could settle the issue once and for all whether early childhood ed is just Head Start or a broader academic preparedness scope.
* Office of Elementary Education — For quite some time, we have had an Office of Elementary and Secondary Education.  It is time to separate the two.  The Office of Elementary Education would focus on the foundations of education success, particularly reading and math.  With a K-8 focus, this office would emphasize the early building blocks of successful learning (reflecting much of the research we now know), while providing some new-found emphasis on the middle grades.  We at least need someone who will continue to promote the National Math Panel findings, particularly if we expect STEM to drive secondary ed policy.
* Office of Secondary Education and 21st Century Skills — Nationally, we have made a major investment in improving high schools, making them more rigorous, and providing all students the pathways to educational and life successes.  This office would focus on high school improvement, early colleges, and the transition from secondary to postsecondary.  Bolder still would be a deputy assistant secretary for STEM education, to ensure science-tech-engineering-math instruction is embedded in all our secondary school improvements.
* Office of Teacher Advancement — Obama has made a major commitment to recruiting, retaining, and rewarding teachers.  We should focus an office on the teacher, including teacher training and pre-service education, in-service professional development, teacher incentives, alternative routes for teachers, and overall educator quality.
* Office of Assessment and Accountability — Yes, I know we have an Institute of Education Sciences.  We’ll address that later.  ED needs an office that works directly with SEAs and LEAs on assessment issues, how we measure student achievement, how we address the issue of multiple measures, and how we ensure our schools and our government are accountable and focusing on the instruction and the supports that make a true difference.  And I wouldn’t mind if this office took a close look at the notion of national education standards.
* Office of School Options — During Obama’s time in Chicago, he was involved in the charter school movement.  He has also acknowledged charters as a piece of the education improvement puzzle.  This office would seek to de-politicize the issue, focusing on effective infrastructure, supports, and accountability in school options, particularly charter schools and virtual schools.  Within this office, ED should also include after-school, or out-of-school-time, programs, as such OST efforts are now a bastion for academic supports, social supports, the arts and other opportunities designed to fill the current learning gaps.
* Office of Family and Community Engagement — As I detailed in my open letter to the President-Elect earlier this week, there is a need and a hunger for an office focused on better involving parents and families in the education improvement process.  We need to better inform families, better encourage families to pursue options, and better prepare families to be a part of the solution. (http://blog.eduflack.com/2008/11/05/an-open-letter-to-presidentelect-barack-obama.aspx)
* Office of Educational Entrepreneurship and Innovation — I’ll admit it, I’ll buy into Andy Rotherham’s vision for converting OII into an incubator for new ideas and new opportunities.  Call it entrepreneurship, call it venture capitalism, even call it pubic/private partnerships if it feels easier, but it is a needed component to education improvement in the 21st century.
Not all of these may be (or should be) assistant secretary-level offices, but they should merit consideration somewhere in the grid.
The Conversions  
In addition to these new approaches, there are also a number of current offices that could use some assistance and  fresh outlook on the education landscape:
* Office of Communication and Outreach — This is obviously an office near and dear to Eduflack’s heart.  For too long, OCO has been viewed as a reactive office, one that regularly issues press releases, fields FOIA requests, and decides which media calls will be returned by whom.  Moving forward, the office needs to jump on the latter part of its name, and transform into an office of public engagement.  Utilize the vast social network built by the Obama campaign.  Broaden the reach to stakeholders.  Be proactive in pushing policy issues and promoting successes.  Set the terms and drive the story.  Doesn’t get more simple than that.
* Institute of Education Sciences — IES was created to be our nation’s home for education R&D.  Unfortunately,
there is still a great deal of work that needs to be done to meet that goal.  IES needs to broaden its mission beyond the WWC and become a true clearinghouse for quality research and a Good Housekeeping seal of approval for what works.  More importantly, it needs to expand the dialogue beyond the researchers and effectively communicate the education sciences to practitioners, advocates, and others in the field.
* Office of English Language Acquisition — OELA, and its previous personalities, has almost been a red-headed stepchild in ED for quite some time.  But as our nation’s demographics continue to shift, ELL and ESL issues become more and more important to closing the achievement gap and providing opportunity to all students.  Focusing on inclusiveness, partnership development, stakeholder engagement, and integration with other offices (particularly elementary ed), OELA can be the lever for improvement many want it to be.
* Office of Federal Student Aid — I’ll admit, I am a little out of my element here.  But with the economic issues we are facing as a nation, ED is going to have to spend more time and intellectual capital on helping students and their families better understand the funding options for postsecondary education.  Simplifying the FASA, ensuring students understand accreditation, articulation of credits between institutions (and between high schools and colleges), and other issues that factor into our ability to pay for college.
I can go on, but I will leave it at that.  Obviously, many core offices will likely remain in place — General Counsel, Inspector General, Civil Rights, Leg Affairs, etc.  Some will say the Office of Planning, Evaluation, and Policy Development — could be folded into the core responsibilities of a top three ED official.  And offices like Vocational and Adult Education may be past rescuing and just need to be left alone.  Regardless, there are clearly a lot of options for those thinking the big thoughts in the transition.
Yes, the Obama campaign was based on hope and change.  When it comes to the U.S. Department of Education, it may also be a time for similar hope and change.  Clearly, our educational priorities and needs have shifted over the last decade, as we focus on teacher development, 21st century skills, STEM, and the P-20 education continuum.  A new approach, with new foci, serves as a strong rhetorical tool to make clear what the Obama Administration will hold dear.  And such rhetoric is all the more important when current economic concerns make it difficult to fund new policy ideas straight out of the gate.  

An Open Letter to President-Elect Barack Obama

Eduflack charter schools, IES, NCLB, Obama, Parents, Presidential campaign, SBRR, STEM

Dear President-Elect Obama,

Congratulations on your impressive victory last evening.  For the past two years, you have spoken to the nation about the need for hope, the need to dream, and the need to do things differently.  Your message of change is not only one that should take hold of government itself, but it is also one that should serve as the cornerstone of your education policy.  You now have a mandate for real change, with the Congress and the national will to support it.
Throughout the campaign, you focused on five key education issues: 1) early childhood education; 2) general K-12, 3) teacher recruitment and training; 4) affordability of higher education; and 5) parental involvement. These issues now serve as the tent posts of your federal education policy.  And they play an equally important role in shaping your U.S. Department of Education.
Now is not the time to retreat to the educational status quo of a Democratic president.  Now is not the time to put power in the hands of those seeking to protect and conserve what was, or those who are troubled by the notion of innovation or new approaches.  And now is certainly not the time to refight the NCLB fight, throwing punches that should have been thrown six years ago.
Instead, now is the time to be bold and audacious, as you have called for so many times before.  Now is the time to be innovative and offer new ideas for the problems that have ailed our public schools for decades now. Now is the time to build a non-partisan approach based on what is needed, what is sought, and what works.  Now is indeed a time for change, and you need to use education to drive that change.  The status quoers or the defenders of policies part don’t fit with your message.  This is time for powerful rhetoric, deep thinking, and meaningful change and innovation.
I will leave it to you and your transition team to determine who the next EdSec will be.  If recent history is any indication, the Clinton model works well.  Find a strong administrator — a governor type — who understands the issues and knows how to effectively use knowledgeable staff.  The Mike Easleys or the Janet Napolitanos or even the Phil Bredensens of the world deserve a close look.  Sure, your selection will be based in part on who is selected for other Cabinet posts, as you seek the right racial, gender, and geographic balance of the Cabinet.  But these sorts of governors have the political experience, management background, and general understanding needed to move the issue forward.
Those jobs further down the line in the Department of Education are the jobs that are essential.  Who will be driving policy?  Who will implement the policy?  Who will collect the data?  Who will analyze it?  Who will market and sell all of it to the stakeholders that are needed to move change?  The assistant secretaries you appoint will be the linchpins of your education policy success. Don’t make these patronage jobs.  Don’t use these to reward friends or organizational friends of the campaign.  Get out into the field and find the best people for the jobs.  Of particular importance, at least in Eduflack’s eyes, is finding the right people to head the Office of Elementary and Secondary Education, the Office of Innovation and Improvement, the Office of English Language Acquisition (particularly since the Hispanic community was such an important demographic in your victory), and the Institute of Education Sciences.  Find the true leaders, the true innovators, and the true thinkers to head these offices and drive policy.
Now that we’ve gotten the administrative piece out of the way, let’s focus for a second on actual policies.  In your policy platforms, you’ve identified a number of issues and areas that you want to focus on, both in terms of rhetorical and financial muscle. Many of these are specific programs, whether they be the continuation of the old or the creation of the new.  These are good ideas — some great, but as your education transition team moves forward, I ask that you make sure a number of issues get their fair shake:
* STEM — We all know that science-tech-engineering-math is a hot topic these days.  But it is also a substantive topic.  Education doesn’t happen in a vacuum.  STEM provides you a tangible program to effectively link instruction to our future economic needs.  It tells kids they are career ready.  It tells employers we have a viable pipeline in the workforce.  And it tells the nation we are doing what it takes to align education with the economy.  STEM is your low-hanging fruit, and you can make some immediate gains by focusing on this policy priority, using education as an economic driver in all states.
* Reading — I have reluctantly accepted that Reading First is dead.  But for decades, the federal government has funded programs to boost reading achievement, particularly among minority and low-income populations.  We need to continue that commitment, and Title I doesn’t get the job done.  For all of its flaws, RF has left a legacy of evidence-based instruction and ensuring we are doing what is proven effective.  Let’s use that to build a new, better reading approach.  Scientifically based reading is in place in every Title I district across the country.  Now is not the time to change horses.  Now is the time to build on successes, showing all families — from those in our urban centers to those in our most rural of communities — that we are committed to making sure every child is reading proficient and reading successful.
* Education Research — Staying on the topic, we need to continue federal efforts to support high-quality K-12 research.  We need to do a better job of collecting long-term measurements of student achievement, teacher effectiveness, and the like.  And we need to do a better job of analyzing the data we collect. Now is the time to use IES to further shape education R&D in the United States.  That shaping requires a true innovator at the helm, with a good sense of research and a better sense of innovation and experimenting on what is new and possible.  Few see it, but the IES appointment will speak volumes as to the possibility of new ideas and new educational exploration for the next four years.
* Teachers — Supporting teachers is more than just supporting the teachers unions.  You’ve demonstrated that understanding in your support to merit pay.  Continue to display that independence.  Merit pay, for instance, is a terrific tool to implementing best practice in the schools, sharing best practices among educators, and incentivizing closing the achievement gap and boosting student achievement without the strict use of the ED stick.  If you need help with this, just give a ring over to your advisor Jon Schnur and ask him about New Leaders for New Schools’ lessons learned through the EPIC program.
* Innovation — All of the great education ideas have not been thought of yet.  You need to find ways to invest in experimentation and invest in what is possible and what is promising.  That is why OII was originally conceived. Take a look at advisor Andy Rotherham’s (and Sara Mead’s) study for Brookings on the future of education innovation, and start exploring the ways to use OII as a venture capital fund for new ideas and as an incubator for promising practices.  We should even elevate OII to full assistant secretary status.
* Accountability — Some think you will throw accountability out the window when you take office.  Eduflack knows better.  From your work in Chicago, you understand the importance of measuring the effectiveness of our reforms.  You know we need to see real results if we are to continue real work.  We not only need to keep measuring student achievement, but we need to do a better job of applying the data to policy dec
isions, spending decisions, and instructional decisions.  More importantly, we just need to plain know that what we are doing works, and it works in schools like mine, in classes like mine, with kids like mine.  There is nothing wrong with accountability if it is a shared responsibility, shared by government, schools, teachers, parents, and the students themselves.
* Choice — Forget about vouchers, the future of education choice is charters and virtual education.  There is a fine line between offering choices to families in need and stripping resources from the public schools.  You need to find it. Charter enrollment in our urban centers is at all time highs.  Find ways to further encourage it, while requiring higher quality and greater oversight.  Virtual education, such as that mandated by Florida, is the future, and needs to be further explored to expand learning opportunities, particularly in our urban and rural schools.  Options are key if we are to give every child a chance at opportunity.
* Parental Involvement — Now for my big idea.  I propose you actually establish an Office of Family and Community Engagement, an authorized body at the Assistant Secretary level that can get information into the hands of those who need it most.  The most recent regs from ED show that the current infrastructure isn’t getting it done.  If you’re serious about greater family involvement, turning off the TVs, and such, make the commitment to Family Engagement (and we do have to think beyond the traditional mother/father nuclear parent family structure). EdTrust has today’s student attaining education at lower rates than their parents. That is a travesty.  And the responsibility falls on the family.  Parents are our first, and most durable, of teachers.  Equip them with information, help them build the paths and help them paint the picture of the value and need for education.  Create this new office, have it collaborate with OESE, OCO, and others, and see the impact of effectively collaborating with families and the community at large on education improvement.
Throughout the campaign, you demonstrated a keen understanding for the intersection between policy and communication.   That understanding must be applied to your education work as well.  On the whole, your predecessor did a poor job when it came to communicating, even with regard to some good policies.  Their thinking seemed to be people will realize this is good policy, and if they don’t we’ll make them because we are the federal government. That won’t work for you.  You need to effectively sell your policies, and you need to sell them to a broad cross-section of audiences.  You need stakeholder buy-in from the beginning, and that buy-in comes from more than just the usual suspects.  Through a well-though-out, sustained public engagement plan, you can not only educate Americans on why education is important, you can actually change their thoughts and behaviors when it comes to the above issues and so many others.  And if you aren’t sure how, just give me a call.
I realize, from recent media interviews, that education is not going to be a top three issue for your Administration.  That is understandable.  I was heartened to see it comes into the top five.  That just means there is more heavy lifting for your Department of Education and for those inside it to do more and make more change with less of the presidential bully pulpit.  We share a common goal — a high quality education for all children.  Now we just need to build the team and execute the plan to move that goal into reality.  You have that chance.  Please take full advantage of it.  Yes, you — and we — can.
Best,
Patrick R. Riccards (aka Eduflack)

What Works for the WWC

Eduflack IES, Reading First, SBRR

Last week, This Week in Education revealed that Russ Whitehurst was leaving the Institute of Education Sciences.  That should come as no surprise, as Whitehurst’s congressionally appointed term expires in November 2008, and he has made clear he was not seeking reappointment.  TWIE’s announcement was followed by Fordham Flypaper’s news that Whitehurst was moving over to Brookings’ Brown Center for Education Policy, presumably to fill the very capable shoes of the departing Tom Loveless.

Interestingly, no one in DC education policy circles seems to be talking about what comes next for IES.  Who will serve as the next director?  What will the priorities be?  Heck, we don’t even know where an Obama or McCain administration stands on IES, its mission, and its programs.  One thing seems certain, though, the future of the Institute won’t be determined until well after the next president is sworn into office. 
The defining experiment of IES has been the What Works Clearinghouse.  Released six years ago with much fanfare, the WWC was intended to be a Consumer’s Reports of sorts for education practitioners.  It was to sort through all of the education research data, determine what works, and provide guidance to school district officials, building leaders, and teachers on what was most effective and how education dollars should be spent.
A noble goal, and a much needed role in today’s education universe. Unfortunately, a funny thing happened on the way to implementation.  WWC became a methodological monster.  In its zeal to distinguish itself from the past work of the U.S. Department of Education’s former Office of Education Research and Improvement, WWC laid out strict and complicated criteria for every piece of research it would examine.  The result?  Most research was kicked out during the initial stages, found to lack the methodological rigor WWC called for.  For these studies, we never got to the issue of effectiveness or impact because they lacked effective control groups or didn’t form the proper study structures, as called for by WWC and its advisors.
Over the years, we have accepted the WWC process as fact.  Some organizations have pleaded mea culpa, asking WWC for advice on what to do.  Others have carefully constructed a single study to meet WWC criteria so they could claim approval (while claiming their competitors were rejected).  And still others refused to acknowledge the value and authority of WWC, and continued to do things their way, stating that their research was true, clear, and effective.
In recent months, WWC has drawn some real attention, mostly for its negative findings on the instructional impact of reading programs Open Court and Reading Mastery. For those watching WWC all these years, we just chalked it up to WWC being WWC.  Eduflack was of similar mind, reminding individuals that, at the end of the day, decisionmakers at the district or school level are simply not making decisions based on WWC findings.  While it may have been the intention, WWC is now merely a forum for researchers to try and outresearch each other, for methodological masters to do their jousting and determine who the true jedi was.  Despite the best of intentions and the most noblest of needs, WWC had become irrelevant.
Then an interesting document crossed my desk.  Under the headline “Machinations of the What Works Clearinghouse,” Zig Englemann provides a terrific analysis on the WWC’s decisions regarding Reading Mastery.  The headline?  “What Works Clearinghouse is so irreparably biased that it would have to be thoroughly reoriented and reorganized under different management rules to perform the function of providing reliable, accurate information about what works.”
With regard to Reading Mastery, Englemann points to the fact that there are more than 90 research studies on the program (and its predecessor DISTAR Reading), with most of these studies appearing in “refereed journals.”  Yet WWC found that “no studies of Reading Mastery that fell within the scope of the Beginning Reading review meet WWC evidence standards.”
Honestly, Eduflack is just plain tired of hearing that no studies fell within the scope.  For years, I defended WWC and IES, seeing both as necessary components to strengthening the research base of the education field.  When Mathematica announced that it was reopening and re-examining the Beginning Reading field — a prior review that had caused much heartburn across the reading community — I was heartened that change may be in the works.  But my atypical optimism has quickly been replaced by my real and necessary skepticism.  Englemann is right, we need to build a new, better, and more effective IES before it dies from within.
Englemann does a terrific job in looking at the WWC methodology, identifying the problems in techniques, approaches, and analyses undertaken by WWC (who knew that research conducted prior to 1985 didn’t count, even if it met every methodological standard laid out by WWC and its methodological advisors).  I would highly recommend the study for all those seeking to better understand how good research has been deemed outside the federal scope these past few years.
But yelling into the wind only gets us so far.  For the record, I still believe in the mission and goals of WWC, and I particularly believe in the mission, goals, and objectives of IES.  We need a strong, independent voice in the federal government that can tell us what works and can point us to valuable research on improving practice and instruction.  We need a voice that ensures what is proven effective is what is being practiced.  We need the evidence, plain and simple.
Now is a unique time for IES and the Board that oversees it.  Currently, a number of board member nominations are pending before Congress.  Clearly, a search for a new IES director must be underway.  So what’s the message IES and its Board should be spending in the near future?
* The voice of the practitioner is just as important as the voice of the researcher.  Methodology is important, but it is no more important that ensuring an intervention works in a school like mine, in a class like mine, with kids like mine.
* We cannot exclusively value process over results.  Again, the methodology is important, but so are outcomes.  Which is more valuable to our schools — methodologically strong programs with middling results or middling methodology with extremely strong results?  Yes, we want strong methodology and results, but if I have to choose, I want to invest in what works and what’s working on a large scale.
* We need to broaden the audience.  IES should be playing to a wide range of stakeholders, and not merely the research community and the education publishers it happens to review at a given time.  It should be a resource for teachers, school leaders, and policymakers.  To do that, we need to simplify the message and broaden the appeal.
* IES can become a Consumer Reports for the education sector.  But to do so, it needs to change its thinking and approach.  It needs to be user friendly.  It needs to be collaborative.  And it needs to think more broadly about the impact it is having, not just on research but on education and the community at large.
* IES needs to inspire and lead.  This isn’t just about being the stick to ED’s funding carrot.  It isn’t just about good, methodologically sound research.  IES needs to invest the time and effort into educating stakeholders as to why scientifically-sound education is important.  Why do we care about methodology?  What are the benefits to scientifically based instruction?  And what can I do to get the science into my kids’ schools?  It needs to demonstrate how education sciences improves student achievement and boost student options for both school and
life.
Recent attempts by IES and WWC to quickly turn around focused research reports are a good first step. But if the Institute is going to build its legacy, it needs to focus on building both public awareness for its mission and objectives and establishing public support and enthusiasm for achieving those goals.  It’s not a simple process, but it is doable.  And it is necessary for IES to survive and thrive.
Regardless of who is EdSec come January, IES should be a priority for either administration.  It possesses enormous potential and opportunity.  Hopefully, someone will remember that and we can build on its best qualities, learn from its worst, and do what is right by states, districts, schools, and students.

The Neverending Saga of RF Data

Eduflack IES, Reading First, SBRR

Even the most zealous of Reading First advocates/agitators (yours truly included) recognize that the headstone for the federal program has been carved.  At this point, we’re all just waiting to see if RF will officially be laid to rest on October 1, 2009, when a new fiscal year takes affect or in March 2009 or so, when a new Congress decides to abandon a continuing resolution for the federal budget and actually passes a Labor/HHS/Education appropriations bill (and as former appropriations folk, Eduflack would be shocked if anything new happens with the budget this spring, regardless of who is president).

Earlier this year, we heard much media trumpeting of the failure of RF.  Flying a banner of an IES interim study of RF effectiveness, RF opponents and many members of the media flatly stated that, after billions of dollars, Reading First just didn’t work.  The proof?  While reading scores have improved since RF’s passage, the initial differences in proficiency increases in RF schools and non-RF schools weren’t that much different.
As we’ve stated here before (and as others have more prominently stated in other more prominent forums) that interim study was significantly flawed, methodologically.  More importantly, it never took into account the effect that RF had on reading instruction throughout the nation.  What does that mean?  Publishing houses changed their textbooks and their support materials to meet the RF research standards.  Teacher training and PD programs evolved to meet the research standards laid out in RF.  Whether a school received specific RF funding or not, all schools were and are benefiting from the policy, mandates, and expectations of the Reading First law.
Late last week, the U.S. Department of Education released a new study on RF.  This study, prepared by Abt Associates and meant as a follow-up to a similar 2006 study, declared “limited benefits.”  This study did look at the RF schools versus the non-RF schools, noting that all are likely affected by the law, but that RF schools adopted the law with more fidelity.
At the end of the day, the ED study found limited gains for third-grade reading proficiency in half of the 24 states surveyed,  For fourth graders, six of 17 states surveyed saw improvements in reading proficiency.  Data came from 1,000 RF schools and from 500 Title I schools that did not receive RF funding.  The full story can be found at Education Week, courtesy of Kathleen Manzo — <div><br></div><div>While”>www.edweek.org/ew/articles/2008/10/15/08reading.h28.html?tmp=326752502.

<div><br></div><div>While”>

<div><br></div><div>While”>While I like to pretend I am, I am not a researcher.  I’ll leave it to the real experts to tell us what’s under the topline data and what these findings really mean, both for Reading First and for schools throughout the nation.  There is still a lot to be written about what this study truly finds, just as there is a lot to be said in what IES will report in its final study, hopefully cleaning up the problems of this year’s interim study.
What troubles Eduflack, though, is how little attention this study has received.  I recognize it was only released late last week.  To date, I have only seen Education Week take the time and consideration to cover this issue.  In the past, publications tripped over themselves to report on the failures of RF or the finding of no findings.  Where is the media coverage of this study?  Here, we have data that demonstrates benefit, and real benefit in half those states surveyed.  Sure, I wish the results were stronger.  But this data — hopefully one of many studies to come in future years, demonstrates there is benefit to Reading First.  And that means we have something to learn from.
RF’s research base, instructional goals, priorities, and potential impact on quality instruction are all positives for our education system.  Hopefully, once that headstone is erected, we can take these parts — a multiple organ donation if you will — and use them to add real life to a strong research-based reading policy.  As a federal government, we’ve invested in reading instruction for decades now.  That won’t stop when RF stops.  With luck, though, we will build and evolve, and not simply scrap and start new.

Is Reading First Dead or Not?

Eduflack IES, NCLB, Reading First, SBRR, Spellings

Not much more than a month ago, it seemed the entire education community had written Reading First off for dead.  Congress has zero-funded the law.  The U.S. Department of Education was doing little, if anything, to do something about it.  IES had released an interim study questioning the program’s effectiveness.  All seemed relatively lost.

Yes, there was a small chorus of sane voices out there, trying to save this important program.  Sol Stern led a charge.  USA Today strongly weighed in.  Fordham Foundation provided intellectual heft.  Even little ol’ Eduflack got in more than its cent and a half.
Yet most have been planning for RF’s funeral.  Facts are facts, and the facts for RF were just not looking good.  Despite the need for scientifically based reading, despite the impact it has had on student achievement over the past five years, the simple fact was that RF was being zeroed out.  Those schools looking to implement SBRR would need to do so on their own, finding the necessary resources to fund programs that work (without the help from the feds).
The start of the school year may have shifted a little bit of thinking, though.  Tomorrow, EdSec Margaret Spellings will be in Des Moines, Iowa for a day o’ Reading First.  She’ll be touring RF classrooms at George Washington Carver Community School, and then will participate in a roundtable discussion with the superintendent and RF teachers.
More important, though, was the report issued late last month by the Reading First Federal Advisory Committee.  This advisory committee — led by Katherine Mitchell, the former Assistant State Superintendent in Alabama — issued its report as a direct response to the RF interim study released earlier this year by IES.  In their report, the Advisory Committee points to the interim study’s fundamental flaws (most of them methodological, which should be a surprise coming from IES).  More importantly, the committee states that the data found in the IES study is insufficient to make the claim that RF is ineffective.  The advisory committee’s ultimate conclusion — the Congress and ED should not make any long-term decisions on RF until better, more comprehensive data is collected.  They aren’t saying the IES study is wrong, they are just saying the data is insufficient to make any meaningful conclusions.
Of course, this study has gotten little (just about NO) attention from the media.  IES’s interim study was a dagger into RF’s heart, offering the media an entertaining Shakespearean education reform tragedy.  It made from great news, as IES (the office created, in many eyes, to build up SBRR and RF) was ultimately inflicting the wound.  It fell to alternate media, such as the blogosphere, to identify the flaws in the interim study.  It will likely fall to them once more.
So what comes next?  Despite the wishes of the chattering class, RF is likely to get level-funded for one more year.  As Congress fails to pass a new Labor/HHS/Education appropriations bill before the end of the month, Congress will simply move into CR mode, meaning that the new budget will simply be a carbon copy of the old budget. So RF programs will collect another year of federal funding, some $350 million or so.  One more year of life.  One more year of opportunity.
Why is this important?  It gives RF (and more importantly, SBRR) supporters a final year to ensure that the legacy of RF is not abandoned when the federal implementation funding dries up.  In a year when the White House, ED, and a number of state departments of education will change hands, those who have benefited from RF’s beacon will need to figure some things out.  How do we keep what works in the classroom?  How do we ensure our schools continue to prioritize scientifically based reading research?  How do we distinguish between good and bad research?  How do we empower teachers with research-based instruction?  How do we get all kids reading?
A lot of questions, yes.  But a lot of questions with clear answers.  We may need a change of vocabulary, but the core principles on which RF was built remain more important than ever.

What Reading Program Works

Eduflack IES, LAUSD, Reading First, SBRR

Earlier this week, the What Works Clearinghouse released its analysis on the research base for the Open Court and Reading Mastery programs.  To the surprise of many (or at least many of those who are paying attention to the WWC these days), both programs were found to lack the research umph that WWC and the Institute of Education Sciences demands under the “scientifically based” definition.

EdWeek’s Kathleen Manzo has the full story here — http://www.edweek.org/ew/articles/2008/08/13/01whatworks.h28.html?tmp=1851512060.

The reports are particularly interesting because most believed Open Court and Reading Mastery were two of the leading programs for which Reading First and SBRR were intended.  Open Court is the program of choice in Los Angeles, for instance, and both programs have been credited with boosting student reading achievement in the classroom.

Critics of RF will use this as yet another “I told you so” moment, that such golden list programs lack the research merit to warrant inclusion.  And while it might make good AERA chatter, there is a much larger issue we should be discussing.

What is the true impact of the What Works Clearinghouse?  Based on these reports, does anyone expect LAUSD to drop its contract with Open Court?  Of course not.  LAUSD has long believed the program has helped students in LA, and they’ll point to their own student achievement numbers to prove it.  Same goes for most of the schools using both Open Court and Reading Mastery.  It is in those schools because administrators, teachers, or both have found it effective with their kids. 

As with much of the federal education reforms of the past decade, WWC is in a time of transition.  Now is the time for the Clearinghouse to figure out what it really wants to be, and what role it is to play in P-12 education.  Is it an evaluator of commercial programs?  Is it an arbiter of scientifically based research?  Is it a Consumer Reports for education?  Or is it a tool to help education decisionmakers make intelligent decisions about instructional practice?

We need to start shifting from an “all or nothing” thinking and start determining how WWC fits into the larger framework.  Otherwise, it could be another story of unfulfilled potential.

The Neverending Quest for Good Data

Eduflack AYP, IES, national standards, NCLB, Reading First, SBRR

Why is it so hard to find good, meaningful scientific data to prove the efficacy of an education reform?  Do we know what good data is?  Is it too expensive to capture?  Is it deemed unnecessary in the current environment?  Is it out-of-whack with the thinking of the status quoers?

EdWeek’s Kathleen Manzo has been raising some of these issues over on her blog — Curriculum Matters.  (http://blogs.edweek.org/edweek/curriculum/)  And no, Eduflack has no qualms whatsoever with her taking me to task on whether the proof points I use to demonstrate Reading First is working are truly scientifically based proof points.  To the contrary, I appreciate the demand to “show me” and have greatly enjoyed the offline conversations with Manzo on what research is out there and whether that research — the good, the bad, and the ugly — meets the hard standards we expect.

For the record, I am not a methodologist, a neuropsychologist, nor an academic to the nth degree.  I learned about research methodology and standards and expected outcomes from NRPers like Tim Shanahan and Sally Shaywitz and from NICHDers such as Reid Lyon and Peggy McCardle.  My knowledge was gained on the streets, so take it for what it is worth.

When NCLB and RF were passed into law, the education community took a collective gasp of concern over the new definition of education research.  The era of squishy research was over.  The time for passing action research or customer satisfaction surveys as scientific proofs of effectiveness had met its end.  Folks starting scratching their heads, wondering how they would implement (and fund) the longitudinal, double-blind, control-grouped studies defined as scientifically based education research.

The common line in 2002 and 2003 was that only two reading programs, for instance, met the research standards in SBRR.  Those two?  Direct Instruction and Success for All.  Not Open Court.  Not Reading Recovery.  Not Voyager.  Only DI and SFA.

So what has happened over the years?  In 2002, the fear was that every educational publisher would have to adopt a medical model-style research network a la NICHD.  Millions upon millions of dollars would need to be spent by the basals to prove efficacy.  It was to be a new world order in educational research.

Where are we today?  As Manzo correctly points out, five years later there is little (if any) research out there that is now really meeting the standard.  Even the large IES interim study of RF effectiveness — that $31 million study of our RF districts — fails to meet our standards for high-quality, scientific research (if you listen to the researchers who know best).  Why?  Why is it so difficult for us to gather research that is so important?

First, we have interpreted the law the way we want to interpret the law … and not the way it was written or intended.  Those being asked to implement the research models simply didn’t want to believe that Reid Lyon and Bob Sweet really wanted them to pursue such zealous and comprehensive research.  So it was interpreted differently.  Neither consumers (school districts, teachers, and parents) nor suppliers (basals, SES providers, etc.) saw the necessity of longitudinal, control-grouped, double-blind, peer-reviewed research.  We settled for what we could get.  We knew that documents such as the NRP report of the previous National Research Council study met the requirements.  So instead of doing our own research, in the early years of RF we simply attached the NRP study as our “research base” to demonstrate efficacy.  Forget that the ink on the instructional program wasn’t dry, it was “scientifically based.”  And there were no checks or review process to prove otherwise.

Second, we are an impatient people, particularly in the education reform community.  Take a look at the NICHD reading research network, and you’ll see it takes a minimum of five years to see meaningful, long-term impact of a particular intervention.  RF grants were first awarded in 2002, with most early funders using the money for the 2003-04 school year to start.  That means just now — for the 2008-09 school year — would we truly be able to see the impact of RF interventions.  But have we waited?  Of course not.  We declared victory (or defeat) within a year or two of funding.  If test scores didn’t increase after the first full academic year, the nattering nabobs of the status quo immediately declared RF a failure, simultaneously condemning the need for “good” research.

We need to see results.  If our second grader isn’t reading, we want her reading by third grade, tops.  We don’t have the patience or the attention span to wait five to seven years to see the true efficacy of the instruction.  We need a research model that provides short-term rewards, instead of measuring the long-term effects we need.  A shame, yes, but a reality nonetheless.


The final side to our research problem triangle is the notion of control groups.  In good science, we need control groups to properly measure the effects of intervention.  How else do we know if the intervention, and not just a change in environment or a better pool of students, should be credited or student gains?  That is one of the great problems with the IES interim study.  We are measuring the impact of RF funding, but were unable to establish control groups that did not benefit from RF materials, instruction, and PD (even if they didn’t receive any hard RF dollars).

But in our real-life classroom environment, who wants their kid to be in that control group?  We all want the best for our children; we don’t want them to get the sugar pill while all the other students are getting scientifically based reading and a real leg up on life.  How do you say to teachers — in our age of collective bargaining — that these teachers on my right will get scientifically based professional development, but these two on my left will get nothing?  How do we say these students on this side of the district will get research-based instruction and materials, but this cluster here will get instruction we know to be ineffective.  Politically, our schools and their leaders can’t let real scientifically based research happen in their schools.  Too much grief.  Too many problems.  Too little perceived impact.

So where does this all leave us?  At the end of the day, we all seem to be making do with the research we can get, hoping it can be held to some standard when it comes to both methodology and outcomes.  We expect it to have enough students in the study so we can disaggregate the data and make some assumptions.  We expect to do the best we can with the info we can get.

Today, we see that most “scientifically based” research is cut from the same cloth.  No, we aren’t following the medical model established by NICHD’s reading network, nor are we following the letter of the law as called for under NCLB and RF.  Some come close, and I would again refer folks to the recent RF impact studies conducted in states such as Idaho and Ohio.  The methodology is strong, the data is meaningful.  And it shows RF is working.

What we are mostly seeing, though, is outcomes-based data.  School X scores XX% on the state reading assessment last year.  This year they introduced Y intervention, and scores increased XX%.  Is it ideal?  No.  But it is a definite start.  We are a better education community when we are collecting, analyzing, understanding, and applying data.  Looking at year-on-year improvement helps us start that learning process and helps us improve our classrooms.  It isn’t the solution, but it is an important step to getting there (particularly if we are holding all schools and students to a strong, singular learning standard).

Yes, Kathleen, we do need better research.  We know what we need, we know how to get there.  But until we demonstrate a need and a sense of urgency for the type of research NCLB and IES are hoping for, we need to take the incremental steps to get us there.  Let’s leave the squishy research of days of old dead and buried.  We’ve made progress on education research over the past five years. We need to build on it, not destroy it. 


 

Is Opinion Research?

Eduflack DCPS, IES, NCLB, Spellings, vouchers

For nearly a decade now, “research” has been the buzz word in education reform.  It comes in many flavors, and it usually comes with a number of adjectives — scientifically based, high quality, effective, squishy, and such.  And by now we all know that “scientifically based research” is in the NCLB law more than 100 times.

With all of the talk about research, we know there is good research and there is not so good research.  We have action research passed off as longitudinal.  We have customer satisfaction studies passed off as randomized trials. We have people mis-using, mis-appropriating, and downright abusing the word “research.”

Through it all (at least for the past seven years or so), the U.S. Department of Education was supposed to be the arbiter between good and bad research.  IES was founded to serve as the final, most official word on what constitutes good education research.  Dollars have been realigned.  Programs have been thoroughly examined.  Priorities have been shaken up.

So where does it all leave us?  In this morning’s Washington Post, EdSec Margaret Spellings launches a passionate defense of the DC voucher program.  http://www.washingtonpost.com/wp-dyn/content/article/2008/07/07/AR2008070702216.html  (Personally, I’m still waiting for such a defense of Reading First, a program helping millions upon millions of more students in schools beyond our nation’s capital, but what can you do?)

It should come as no surprise that Spellings sought to use research to demonstrate the effectiveness and the need for the DC voucher program.  Without doubt, vouchers have had a real impact on the District of Columbia.  It has reinforced the importance of education with many families.  It has opened doors of schools previously closed off to DC residents.  It has forced DC public schools and charters to do a better job, as they seek to keep DC students (and the dollars associated with their enrollment) in the DCPS coffers.  And, of course, we are starting to see the impact vouchers are having on student achievement among students who previously attended the most struggling of struggling schools.

Spellings points out all of this in her detailing of the research validating the voucher program.  But there is one “research” point Spellings uses that just has Eduflack scratching his head.  From the EdSec’s piece — “The Institute of Education Sciences (IES) found that parents of scholarship children express confidence that they will be better educated and even safer in their new schools.”
 
Such a statement is downright funny, and quite a bit concerning.  In all of the discussions about scientifically based research, high-quality research, the medical model, double-blind studies, control groups, and the like, I don’t remember public opinion surveys meeting the IES standard for high-quality research.  Parents feel better about their children because of vouchers?  That’s a reason to direct millions in federal funding to the program? 

Don’t get me wrong.  I’m all for public opinion polling and the value of such surveys (along with the focus groups and other qualitative research that helps educate them).  But it is one of the last things that should be used to validate a program or drive government spending on educational priorities.

If DC is to keep vouchers, it should keep them because it is driving improvement in student performance and is giving a real chance to kids previously in hopeless situations.  It should be saved with real data that bears a resemblance to the scientifically based research we demand of the our programs and that we expect our SEAs and LEAs to use in decisionmaking.  It should be actionable research, with a clear methodology that can be replicated.
 
Otherwise, we’re just wrapping up opinion in a research wrapper.  That may be good enough for some for-profit education companies and others trying to turn a quick buck on available federal resources, but it shouldn’t make the cut for the government — particularly the branch of ED that is in charge of high-quality research.  Ed reform should be more than a finger-in-the-wind experiment.  And Spellings and IES should know that by now.


Swingin’ at an RF Pitch

Eduflack IES, Reading First, SBRR

I know, I know, I promised my Quiotic quest over the IES Reading First implementation study was headed for the bench for a little bit.  But after watching so many swing and miss at this RF pitch, Eduflack just has to offer plaudits when someone else makes solid contact and raises some great issues on this study.

Kudos go to Kathleen Kennedy Manzo over at Education Week.  Manzo is one of the original RF reporters (along with Greg Toppo), having covered it from the early stages to today.  It’s meant that she’s likely been flooded with information, data, research, opinion, and spin over these past six or so years.  It’s meant a continuous learning process.  And it’s meant having to sort through it all, avoiding the pitches in the dirt and waiting for the good pitch to hit.

Hit it she did.  In this week’s Education Week, Manzo’s got a great piece on the IES study.  http://www.edweek.org/ew/articles/2008/06/04/39read.h27.html?tmp=1914927477  She explores many of the quality issues that have been raised to date.  More importantly, though, she gets Russ Whitehurst to state that no conclusions should be made based on the interim report.  Instead, we need to wait for the final.

I, for one, am hoping that means there’s a whole lot of fixing coming in the final report.  Of course, I’ve been disappointed before.  Regardless, EdWeek and Manzo deserve credit for taking a complicated and growing issue, and reporting on it so that the average educator or the average policymaker understands the issues and knows the tough questions to ask.

Gold stars all around.