eddata

Injecting Some Moneyball into Student Testing

Eduflack accountability, Achievement gap, assessment, AYP, eddata, reform

I’ve always been one to find love in both the art and science of a given subject. As a lifelong baseball fan – and a pretty poor baseball player through high school – I quickly embraced Ted Williams’ The Science of Hitting, believing that the charts and graphs explaining strike zones and such would somehow transform me from a doubles hitter into a homerun machine. Sadly, it never did.

I’m also an unabashed fan of the New York Mets, and have been since the early 1980s. For more than three decades, I have endured the highs and lows (mostly lows) of rooting for the Metropolitans and in believing this might just be the year.

Sadly, the 2018 season wasn’t that year for the Mets. But it was such a year for Mets ace Jacob deGrom. Last week, the All-Star received the Cy Young, recognizing the best pitcher in the National League. It was a well-deserved honor, recognizing one of the best seasons a starting pitcher has ever had, including an earned run average of only 1.70, a WHIP of 0.912, and 269 strikeouts in 217 innings pitched. DeGrom secured the first place position on all but one of the ballots cast this year, offering a rare highlight in another tough Mets season.

Leading up to the award, there were some analysts who wondered if deGrom would win the Cy Young, despite those impressive numbers. The major ding against him was that he was pitching for the Mets, and as a result posted only a 10-9 record, getting almost no run support at all all season from his team. DeGrom’s top competition in the NL had 18 wins. The Cy Young winner in the American league posted 21 victories. So when a 10-9 record won the Cy Young, some critics pounced, accusing sabermetrics and “moneyball” taking over the awards. The thinking was that one of the chief attributes of a top starting pitcher is how many wins he has. If you aren’t winning, how can you possibly be the best?

All the discussions about how sabermetrics has ruined baseball – or at least baseball awards – soon had me thinking about education and education testing. For well over a decade, we have insisted that student achievement, and the quality of our schools, is based on a single metric. Student performance on the state test is king. It was the single determinant during the NCLB era, and it remains the same during the PARCC/Smarted Balanced reign.

Sure, some have led Quixotic fights against “high-stakes testing” in general, but we all know that testing isn’t going anywhere. While PARCC may ultimately be replaced by a new state test (as my state of New Jersey is looking to do) or whether the consortium may one day be replaced by the latest and greatest, testing is here to stay. The calls for accountability are so great and the dollars spent on K-12 education so high, that not placing some sort of testing metric on schools, and kids, is fairy tale. Testing is here to stay. The only question we should be asking is whether we are administering and analyzing the right tests.

I’ve long been a believer in education data and the importance of quantifiable research, particularly when it comes to demonstrating excellence or improvement. But I still remember the moment when I realized that data was fallible. While serving on a local school board in Virginia, overseeing one of the top school districts in the nation, we were told that our nationally ranked high school had failed to make AYP. At first I couldn’t understand how this was possible. Then I realized we were the victims of a small N size. The impact of a handful of students in special education and ELL dinged up in the AYP evaluation. The same handful of students in both groups. It didn’t make our high school lesser than it was. It didn’t reduce our desire to address the learning needs of those specific students. But the state test declared we weren’t making adequate progress. The data had failed us.

The same can be said about the use of value-added measures (VAM scores) in evaluating teachers and schools. VAM may indeed remain the best method for evaluating teachers based on student academic performance. But it is a badly flawed method, at best. A method that doesn’t take into account the limitations on the subjects that are assessed on state tests, small class sizes (particularly in rural communities or in subjects like STEM), and the transience of the teaching profession, even in a given school year. Despite these flaws, we still use VAM scores because we just don’t have any better alternatives.

Which gets me back to Jake deGrom and moneyball. Maybe it is time that we look at school and student success through a sabermetric lens. Sure, some years success can be measured based on performance on the PARCC, just like many years the best pitcher in baseball has the most victories that season. But maybe, just maybe, there are other outcomes metrics we can and should be using to determine achievement and progress.

This means more than just injecting the MAP test or other interim assessments into the process. It means finding other quantifiable metrics that can be used to determine student progress. It means identifying the shortcomings of a school – or even a student – and then measuring teaching and learning based on that. It means understanding that outcomes can be measured in multiple ways, through multiple tools, including but not limited to an online adaptive test. And it means applying all we know about cognitive learning to establish evaluative tools that live and breathe and adapt based on everything we know about teaching, learning, and the human brain.

DeGrom won the Cy Young because teams feared him every time his turn in the rotation came up. We knew he had a history-making season because of traditional metrics like strikeouts and innings pitched, but also because of moneyball metrics like “wins above replacement,” or WAR, and “walks and hits per innings pitched,” or WHIP. Had he not won that 10th game the last week of the season, thus giving him a winning record, deGrom would have had no less a stellar season. In fact, a losing record would have indicated his personal successes and impact despite what others around him were able to do.

Maybe it finally is a time a little moneyball thinking works its way into student assessment. Hopefully, this discussion will come before the Mets reach their next World Series.

 

 

A Win for Education Research

Eduflack accountability, eddata, IES, reform, TrumpED

For nearly a year now, the education community has been waiting for key nominees for President Donald Trump’s Education Department. Some have been holding their collective breaths to see who get the nods for some of the “sexy” posts, including assistant secretary of elementary and secondary education.

Last night, President Trump announced the nomination of Mark Schneider to head the Institute of Education Sciences (IES). No, IES is hardly considered a sexy post by most in education. But it is an incredibly important nomination … and job.

If one believes in the identification, understanding, and use of education research, then IES is important. If one believes in scientifically based education, then IES is important. If one believes our schools — both K-12 and higher ed — should be focused on what is proven effective, then IES is important. If one believes data should trump philosophy when it comes to education, then IES is important.

It’s equally important that someone like Dr. Schneider is getting the nod for this job. Mark is both a terrific education researcher and a keen education policy person. He served as commissioner for the National Center for Education Statistics (NCES) in the George W. Bush Administration. Perhaps just as important, he is the nation’s foremost expert on the data surrounding the cost of college — the true cost of college — at least as it applies to students and their families.

I don’t just offer this hearty endorsement based on Dr. Schneider’s reputation. Eduflack has had the opportunity to work closely with the new head of IES, particularly in helping him launch College Measures, an American Institutes of Research center focused on improving higher education outcomes by better understanding higher education data.

At Dr. Schneider’s side, I gained a far greater appreciation for both the data and for its true meaning. I was able to explore how costly some community colleges truly were, when one looked at the cost of actually earning a degree. I had to do rhetorical battle with a university president who thought his eight-year graduation rate (for four-year undergraduate program!) of less than 30 percent was something to be applauded, not concerned about. And I came to appreciate the costs and benefits of college are best looked at through the lens of the consumer, not necessarily the provider, that we need to look at the cost of degree for a student to obtain, not for the university to offer.

I’m fortunate to call Mark Schneider a friend and a mentor in the ed data space. And from nearly two decades experience working with IES — working with the Institute since its inception in 2001 — I’m grateful we will have an experienced, knowledgeable, results-focused leader at the helm.

We may not always know who is leading an agency like IES, but when it isn’t someone of the caliber of Mark Schneider, we feel the impact.

 

A Flashlight, Not a Hammer, When it Comes to #EdData

Eduflack accountability, eddata, teachers

For years now, the education community has debated the proper role of “education data” in the process. What started off as important information to help teachers tailor and improve their instruction quickly became a blunt instrument to punish students, teachers, classrooms, and schools. As with most things, the abuses of a valuable tool became the focus.

With a greater emphasis on testing and the use of testing information, ed data has gained even greater scrutiny. In Eduflack’s own school district, there is a growing call for eliminating all technology from the schools out of fear of education data and its impact on student privacy (among other things).

And that’s just a crying’ shame. And it doesn’t have to be that way.

Last year, the Data Quality Campaign hosted a national summit focused on the importance of education data. In it, DQC CEO Aimee Guidera spoke of the importance of using education data as a flashlight — to illuminate the way to improvement and success — rather than as a hammer to strike those who are struggling.

This week, DQC released From Hammer to Flashlight: A Decade of Data in Education. Noting that “although much work remains before education becomes a truly evidence-based field,” great work has already been undertaken to use data to better inform both inputs and outcomes in the classroom.

Education data isn’t going away. If anything, we need to become more savvy in its application to learning. That’s why From Hammer to Flashlight is such an important read.

 

On #EdData, It Is Time to Act

Eduflack accountability, Achievement gap, assessment, eddata, teachers

I’ll proudly admit it. Eduflack is a strong advocate for data in the learning process. I make no apologies for believing that we need good research and data to determine how effective teaching and learning actually is. Without such data, we are just wishin’, hopin’, and prayin’ that we are getting it right.

Our kids deserve more than just a wish and a prayer. When it comes to their educations, we need to both trust and verify. And education data is the only way to do that.

Of course, that means educators, parents, policymakers, and others in the mix must be able to distinguish between good data collection and lousy. It means ensuring that, if data is collected, it is promptly provided to teachers so they can use it to improve teaching and learning in their classrooms. And it means eliminating those tests that are simply done to check a box, without linking back to our students and their demonstrated knowledge.

The topic of #eddata is usually (one of) the third rails of education policy. For many, assessment is the third circle of Dante’s inferno. They preach of how testing (and the data coming from it) is destroying our schools, stressing out our kids, and stripping the joy of learning from our classrooms. It is also seen as the guillotine in a results-oriented system, used to punish schools, teachers, and kids who are not where we want them to be.

In the battle for edu-hearts and minds, such a false narrative can be incredibly powerful. It can lead parents and educators to forget that data has always been an important part of our K-12 system, and that while specific tests may come and go, not testing our kids just isn’t an option. And it ignores the truth that teachers speak each and every day, that they need data to effectively lead their classrooms, and the question should be what data is collected and how are we assessing student learning (not if we assess it at all).

For the past decade, the Data Quality Campaign has worked across the country to address the false narrative and help policymakers and educators see the importance of educational data quality (thus the name). Because of their work, we see states that have created incredibly strong data systems focused on the learner. We are seeing policymakers and leaders asking the right questions when it comes to data and accountability. And we are seeing where good data is improving instruction, particularly for those communities that need it the most.

Earlier this week, DQC launched an important new report and all to action, Time to Act: Making Data Work for Students. In Time to Act, DQC lays out four key priorities when it comes to ed data:

  • Measure What Matters: Be clear about what students must achieve and have the data to ensure that all students are on track to succeed.
  • „ Make Data Use Possible: Provide teachers and leaders the flexibility, training, and support they need to answer their questions and take action.
  • „ Be Transparent and Earn Trust: Ensure that every community understands how its schools and students are doing, why data is valuable, and how it is protected and used.
  • „ Guarantee Access and Protect Privacy: Provide teachers and parents timely information on their students and make sure it is kept safe.

In issuing this important call, DQC President Aimee Guidera noted that education data should be seen as a flashlight to guide teaching and learning, not as a punitive hammer for teachers and students. As correct as Guidera is, I propose we take it a step further. Ultimately, ed data needs to be our Bat signal (of course the good, Michael Keaton kind, not the Ben Affleck model), one that we signals when we need help and one that makes clear to the community at large that we are watching over those who need our help. That even in dark hours, there is an incredible collection of tools and data and commitment available on the edu-utility belt to help all those who need it.

No, are schools are no longer waiting for Superman. Batman has taught us the right tools can turn a regular guy into a superhero. When that ed data light is shown in the right way, it can illuminate the path to help all educators and kids succeed. We need to make sure that that light is as strong as possible, equipping all educators with the tools to be the superheroes they are.

 

 

For Students’ Sake, Let’s Look to Student Data

Eduflack 21CS, assessment, eddata, Parents, Students

Over at Education World, I have a new piece that looks at the important role student research and data can play when gathered and utilized properly. From Data Quality Campaign to ACT to NWEA to Project Tomorrow to ERCA, we can see the value data plays, particularly in understanding student perceptions on key issues, including their college and career aspirations.

As I wrote:

For years, we have grappled with the notion of “assessment literacy,” where educators, families, and policymakers can better learn the importance of data collection and the ability to distinguish a valuable data-gathering tool from a lousy one. At the same time, educators have demanded that any student data collected needs to be used to help the student, and not just as the impetus for punitive action.

Teachers are right. We probably don’t focus on what is helpful to the student nearly as often as we should, particularly when it comes to student data. And that’s a cryin’ shame. The student perception information coming from organizations like Project Tomorrow and NWEA is incredibly important. It provides a glimpse at how what is taught in the classroom aligns with student interests and passions. It helps us better understand the path today’s young learners are on as we encourage them toward college and career success.

I hope you’ll give it a read.

 

 

#Studentdata and #highered

Eduflack accountability, assessment, eddata, higher ed, technology

We spend so much time talking about the reauthorization of the Elementary and Secondary Education Act (or the replacement of NCLB, whichever term you prefer), that we can forget that reauthorization of the Higher Education Act is waiting in the wings as well.

Earlier this spring, Sen. Lamar Alexander, chairman of the Senate HELP Committee, issued a series of white papers on some of the top issues the Senate would consider as it began to dig into HEA reauth. One of those topics was consumer information, what many of us better know as student data.

Last week, I submitted a formal response to the Senate’s higher education student data call. In doing so, I noted: “As a nation, we have long said that information is power, using the call for greater knowledge to rally support for education. But our educational infrastructure itself has not provided the powerful information we need. Higher education has fallen short in its ability to both capture and apply data that can be used to improve how students learn, how they are taught, and how we measure it.”

This should come as no surprise. In talking about the work of the Woodrow Wilson Foundation and its focus on linking student outcome data to determine the effectiveness of its own programs I stated:

All have a right to know the difference between a successful school of education and a not-so-successful one. That difference really can only be revealed through the collection, analysis, and utilization of outcome data. It is not enough to know that future teachers entering schools of education bring a certain high school class rank, GPA, or SAT/ACT score into the process. Yes, the inputs are important. But far more important is what they do with those tools. And we cannot measure that impact based simply on academic performance leading to the award of a college degree. It requires post-graduation data that can be tracked back to the degree-granting institution.

My full statement, including responses to a number of specific queries from Senator Alexander’s staff can be found here. The initial white paper from the Senate HELP Committee on consumer information (and other topics like accreditation) can be found here.