There was an error in this gadget

Wednesday, September 21, 2016

Ohio Report Card Drives Down Performance. So What?

Amid all the hue and cry over Ohio's new, obtuse and downright head scratching Report Card results, it might be helpful to look at how performance has changed over the four years that Ohio has implemented its much-praised A-F grading system.

What is clear is that each year, report card grades stay the same, or get worse for all school and district types. However, the biggest performance drop -- by far -- has been for Ohio's school districts, which saw their percentage of Ds and Fs more than double over the last four years.

Meanwhile, Ohio's charter schools have had a nearly 40% greater jump in their failing grades than Ohio's much-maligned Big 8 Urban school buildings (the Big 8 school districts are Akron, Canton, Cincinnati, Cleveland, Columbus, Dayton, Toledo and Youngstown). Four years ago, Ohio's charters and urban buildings had almost exactly the same percentage of D and F grades and today, charters have 5 percentage points more D and F grades.

The charts below show the percentage of As, Bs and Cs each group of schools and districts received each year since 2012-2013 and their percentage of Ds and Fs.

Today, about 7 out of 10 Big 8 Urban building grades are D or F, which is not good.  However, 3 out of 4 charter grades are D or F, which is worse.

Meanwhile, even though charters did the best they had ever done in the 2012-2013 school year, with 60% of their grades being D or F, that result is still the mirror opposite of school districts' worst performance.

What's that mean? It means the best charters have ever done is having 6 out of 10 grades be D or F. The worst districts have ever done is have 6 out of 10 grades be A, B, or C.

The only constant between the 2012-2013 report cards and the 2015-2016 version is Ohio charter schools continue to be vastly outperformed by their school district counterparts (remember that all but 2 Ohio school districts lost students and funding to charters last school year). And now, unlike four years ago, we can say that Big 8 Urban buildings -- overall -- outperform charters in a meaningful way.

Regardless of school type results, what these historic data show is just how illogical the report card has become. Does common sense tell you that school districts today are twice as ineffective as four years ago, or even that 3 out of 4 times, charters are failing?

To hear the report card defenders, all this indicates is that our standards are getting tougher.

Never mind we've had three different state assessments the last three years. Don't think that matters? In 2012-2013 and 2013-2014 we had the same tests. Is it a surprise that the results are nearly identical each year? Now they are wildly different.

Never mind that it is illogical to assume that worse grades mean standards are tougher or results are more accurate -- a narrative thread I strongly oppose.

Never mind that if I developed a test in my class that I knew significant numbers of students would fail (like we were told these new Ohio assessments would do), I would be disciplined for developing a bad, unfair test.

What matters most to Ohio policymakers, it appears, is that we are finally getting some data that indicate Ohio school districts are failing kids almost as often as politicians have been telling us they have been failing for years. If the report card is meant to support a political narrative that public schools are failing kids and we need more choices, regardless of those choices' quality, then the report cards are doing that job better than ever.

But if they're meant to actually give the public a better, more accurate notion of what's happening in Ohio classrooms? Well, I'm afraid that on that account, the report cards would get a big, fat F.

Wednesday, September 14, 2016

USDOE Calls Ohio Charter Program "High Risk", puts Special Conditions on Nation's Largest Federal Charter School Grant

It what appears to be a first for the U.S. Department of Education, Ohio has been designated a "high risk" state for charter school oversight, which means the federal government will assert far more authority over how Ohio distributes the $71 million it received last year through the federal Charter School Program grant, which was the largest given last year.

When the grant was given, many immediately questioned how it could have happened, given the fact that Ohio's well-told story of struggling charter schools was well known. The person who sent in the application, David Hansen (husband of Gov. John Kasich's chief of staff and presidential campaign manager Beth Hansen) was forced to resign for illegally doctoring a new charter school accountability regime to benefit politically powerful online charter school operators. It was this regime that was the foundation for the USDOE grant.

There were other sleights of hand that many people identified in the application. However, while USDOE found that there were no "significant inaccuracies" in the state's grant (leaving open the possibility that inaccuracies did exist), there were several problems with Ohio's history with charter schools in general that forced the Department's hand. Specifically, the letter cites the following concerns:

  1. The Hansen affair
  2. Ohio congressional leader concerns
  3. Issues found in previous audits of ODE's capacity to implement federal charter school programs (the only available audit I wrote about here in some detail)
  4. Implementation issues of the additional oversight functions prescribed by House Bill 2 that could impact the state's ability to administer the grant
In addition, the feds urged ODE to "put into place additional mechanisms to help earn the public's confidence in its ability to act as a proper steward of its Federal grant funds on behalf of Ohio's families and students."

The feds also went out of their way to "strongly encourage" the state to put special mechanisms in place to increase transparency between non-profit charter schools and the for-profit entities with which they contract. The feds also made explicit that no online schools can receive the funding. The feds also expressed so much concern about Ohio's miserably performing dropout recovery schools that they told ODE that the feds themselves have to approve all grants given to dropout recovery schools.

The feds wrapped up by expressing confidence that ODE will be able to administer the grant. But they warned that if they fail to meet these special conditions, the state will be subject to strict penalties. They especially pushed ODE to work toward creating "the rigorous oversight of authorizers that was described in ODE's approved grant application."

While I'm glad Ohio will have the opportunity to use this federal money to expand higher performing charters, it is once again a reminder of just what a backwater we have been on charters and how far we have to go.

To add insult to injury, a charter school chain that has some of the highest performing charters in Ohio was rejected for sponsorship in Mississippi -- yes, THAT Mississippi -- this week for not being high performing enough for that state to sponsor.

When the best performing schools in your state aren't high performing enough for Mississippi, you've got problems.

Monday, September 12, 2016

Youngstown Plan Steers Clear of Privatization. For Now.

The recently released plan for Youngstown City Schools had some encouraging things in it. A commitment to wraparound services. An idea to create a high school transition school for students entering high school. A professionalization of the workforce. A serious attempt to engage parents in their children's learning.

And, importantly, no obvious expansion of the mostly failing school choice experiment in Youngstown. Half the kids in Youngstown charters go to schools that perform worse overall on state metrics than Youngstown.

As we at Innovation Ohio suggested last month, Youngstown CEO Krish Mohip should follow evidence-based practices turning around Youngstown schools while trying to avoid the non-evidence-based practices, like relying more on school choice, which in Ohio, has not worked out as promised.

And, it looks like, for the most part, Mohip did stick to those principles. And while some of the stated plan goals are simply unrealistic (which I'll get to later), overall there is much to like with the initial plan.

Now to the details. The most glaring omission is one that Cleveland also neglected in its transformation plan: ask the state for more money to implement the plan. The problem in Youngstown is that I don't see the citizens of Youngstown passing a 15-mill property tax levy to make up for the state's lack of investment the way Cleveland's residents did.

So, for example, how will Mohip possibly be able to provide quarterly performance reviews of 100% of the district's teachers in two years -- a laudable goal -- (let alone weekly instructional feedback) with no additional funding to hire the necessary supervisors to conduct these reviews? Or, how will Mohip ensure that 85% of parents attend open houses in two years when there's no additional funding to pay for outreach coordinators who can be active in the city's neighborhoods? Or how will Mohip ensure all Youngstown children have a personalized learning plan in two years without any additional funding for the coordinators to organize such a massive undertaking? Or how will he pay for his proposed 9th Grade Academy to help kids transition to high school?

Again, all of those goals are laudable. They should be tried. But without additional state resources, it's going to be a struggle for Youngstown to meet them without radically altering job scopes for scores of district employees -- adding new responsibilities to already overloaded staff. Youngstown simply doesn't have the community resources of Cleveland, which was able to utilize substantial foundation and non-profit assistance to do many of the things Youngstown now wants to do.

But that doesn't mean the district shouldn't try.

It is also interesting that currently, 87.7% of the district's teachers are considered to be high performing and the plan's stated goal is to get that number up to 95% in two years. So while much of the plan entails doing more effective teacher evaluations (modeled after what Baltimore has been using for a few years), honoring exceptional performance, and requiring weekly, quarterly and annual teacher/leader feedback, the data suggest it's not teaching, for the most part, that's letting down Youngstown's kids.

In other words, if you were going to pour resources into Youngstown City Schools, the data suggest pouring tons of resources in a new teacher evaluation and tracking system may not be as critical, especially with no apparently new funding coming down the pike.

This brings me to another concern: the plan's call to review employee compensation to ensure "regional competitiveness." What if the review finds that Youngstown teachers make more than comparably prepared professionals? Does than mean Mohip will start ripping up contracts and paying teachers and other professionals less as he begins expecting more from them? Again, no indication one way or another yet.

But given this plan's origin in trying to primarily expand charter school options, it is indeed encouraging that there is no mention of "school choice" or "vouchers" or "charter school" in the 22-page document. That could be because a $71 million federal grant, which was largely intended to expand charters in Youngstown, has been held up by the U.S. Department of Education for several reasons. And once that money's freed up, it will be invested in Youngstown, as originally intended.

Or it could be that Mohip looked at the data and found that, unlike Chicago where he came from, there simply is not a critical mass of high performing charters in Youngstown. And maybe he is thinking that the best hope for parents and their children is to improve the Youngstown City Schools rather than hoping Ohio's nationally ridiculed charter school sector can ride in on a white horse (or White Hat, as the case may be).

That's a good sign that Mohip is actually paying attention to the realities on the ground in Youngstown. That's not to say there aren't some pretty unrealistic goals, even with unlimited funding, which Youngstown does not have.

While the Youngstown Vindicator's coverage called the plan "realistic," here are a few examples that left me scratching my head:

  1. The plan calls for 90% of district buses to pass State Highway Patrol inspection the first time. This year. Last year, 0% did.
  2. This year, the plan wants half of the district's employees' performance to be monitored through a data management system. Last year, no one was. As a City Council member in a community that is shifting to a new data management system for human resources for just a few score people, I can tell you this won't be an easy lift.
  3. Have 1/2 of all district teachers this year provide effective instruction as per the Baltimore system, which has yet to be adopted by the Academic Distress Commission. Again, adopting a new system during the year and expecting 1/2 of teachers (again, 87.7% of whom are already high performing, according to Mohip's report) to understand, adopt and excel with it is quite a lofty expectation.
  4. Ensuring that all kids' test scores and growth are measured every year from K-8, even in grades that aren't currently tested starting this school year seems like quite a lift. It costs the state about $760,000 a year to administer the tests it already does give to Youngstown students. Would the several additional batteries of tests the plan calls for be paid for by the state, or would Youngstown have to pay for them? Stay tuned.
  5. Going from 0 kids taking the PSAT to all sophomores taking it this year seems like a tough thing to achieve, if for no other reason than it costs $15 a student, which could prove a tough fee for parents to handle.
  6. There are several aggressive test score improvement goals throughout the plan that seem very challenging, such as increasing some proficiency rates by nearly 2-3 times in just a few years. Given test scores' nearly linear relationship with poverty, and Youngstown's distressingly high poverty level, it's tough to see these improvements happening, regardless of effort.
Although these goals seem destined to fail, it's good to shoot for the stars. Why not? 

But I have a couple long-term concerns. First of all is the legislation that took control of Youngstown from the people of that city and gave it to a state-appointed CEO requires great report card improvement in order for control to return to the people. And while the goals outlined by Mohip could lead to that improvement happening in a few short years, chances are without the necessary funding, the results will fall short -- sometimes far short -- of the goal. 

If these goals aren't met, then what?

Will Mohip turn to the ideologues? Will the test score pressure become too much for Mohip to resist the call of our state's mostly failing school choice options? Will whole swaths of teachers and principals get fired? Will cooler heads prevail?

We'll see.

But for now, it looks like the Youngstown CEO is moving in the right direction. And given where he could have gone (firing teachers, ripping up contracts, turning the keys over to Ohio's many failing charter school operators), that's a positive development.

Friday, September 9, 2016

When an F isn't an F

Yesterday, I posted about how Donald Trump was visiting a Cleveland charter school that received an F on the state report card for student growth -- a grade I called "failing" in several press accounts. And, in fact, I found it curious that Trump would visit a Cleveland charter with such a poor student growth grade given that Cleveland is the only area of the state where there are several high-performing charters.

I suggested that it was because Ron Packard -- a notorious political operative in the education space -- ran the for-profit company that operates the school. And, right on cue, Trump shouted out Packard at the beginning of his speech. I'm willing to bet that a Packard donation will show up in Trump's next campaign finance report.

Anyway, some charter advocates, especially Aaron Churchill at the Fordham Institute, pushed back against my characterization of the Cleveland charter as "failing" because it's only based on one year of data. And the previous year, the school performed well.

As I've said before, I really respect Aaron, but he's trying to do a do-over here on Ohio's accountability system -- a system he and Fordham pushed to have in place.

Fordham and Aaron championed our state's switch to an A-F report card system because "The A-F grades provide a clear and transparent way of reporting whether a school is academically strong, weak, or somewhere in between." This is also the same group that announced that the drop in state scores last year for many schools due to the PARCC exams and tougher Common Core standards was a better indication of how the students (and schools) "actually performed." (I vehemently disagreed with that assumption, by the way.)

Unless, it appears, Donald Trump decides to visit a charter school with an F in student growth. Then they'll make excuses.

Look, you're either for tougher standards and what the fallout from those standards entail, or you're not. You can't be saying that the tougher standards show us how kids are really doing, then claim they really don't in a school you'd like to see perform well. You can't say the A-F report card gives Ohioans a more transparent way to understand how schools are serving kids, then say the report cards don't in a school you'd like to see perform well.

I've said repeatedly that basing performance solely on test scores, as Ohio (and the country) currently does, is folly and wholly unfair to schools whose performance shouldn't be judged on how kids do during a few hours of testing when they spend more than 1,000 hours in school. But we have this high-stakes system in place. And it is there in no small measure due to the shaming of our public schools, which has been perpetrated by Fordham and others over the years.

And under the system Fordham and other reformers have trumpeted for years, the charter Trump visited yesterday failed to grow student learning as effectively as the Cleveland Municipal School District. And, in fact, the school got an F, which is a failing grade in anyone's book.

Fordham and others in their camp don't get to suddenly adopt a nuanced approach to school performance just because the schools they like failed.

I welcome the more nuanced approach and understanding of school performance. And I'd be less skeptical if Fordham and other education reformers hadn't spent the last 20 years trying to get the no excuses, un-nuanced system we have in place today.

In the words of Mike Brady from A Very Brady Sequel, "Caveat Emptor."

Thursday, September 8, 2016

Trump Makes Point that Charters are Better Options by Visiting Poor Performer

Donald Trump has told the African-American community that "your schools are a disaster", so he has said Charter Schools are the answer. He's trying to demonstrate that today when he visits the Cleveland Arts & Social Sciences Academy in Cleveland.

Here's the thing. The school Trump is visiting received an F on the state report card for student growth -- the most important measure to most charter school advocates. Cleveland -- an urban district Trump has derided collectively -- received a C on the same measure. Visit to find all of the school's performance metrics.

Oh, and the state pays the charter school more money per pupil than Cleveland ... for worse student growth measures.

What's sad is that Cleveland is the only place in Ohio where a case can credibly be made that charters are in any significant way living up to their promise. The Breakthrough Schools and ICAN schools are performing very well. As charter school advocates have noted, that's in stark contrast to the rest of Ohio where 40% of its charter schools are in "urgent need of improvement."

So why would Trump visit a school with an F in the most important performance metric when he had plenty of much higher performing options? Perhaps it's because the school he's visiting is run by a for-profit company called Accel Schools Ohio. Accel is an imprint of Pansophic -- a charter school firm started by K12, Inc. founder Ron Packard. Packard has an infamous reputation for political gamesmanship.

If I wanted to make the point that charters can give kids hope where none before existed, I wouldn't go visit an Accel School. I'd visit a Breakthrough or ICAN school. However, if I wanted to make hay with a potential political contributor with experience in education politics, I would visit Ron Packard's school.

However, this choice doesn't surprise me given Mr. Trump's clear ignorance of the state of education policy.

Tuesday, August 23, 2016

NEW Cleveland Transformation Plan Report: Slow, Non-Uniform Progress, but Many Challenges Ahead

Today, we at Innovation Ohio released a comprehensive look at the so-called "Cleveland Plan". The initial look we took at the plan in 2012 led to a report that caused some much needed changes to the legislation. Our newest follow up was meant to re-examine the plan, how it's working, how it needs improved and, ultimately, some ways to fix it.

Complicating this whole thing is the specter of a teacher strike. The tensions around the union negotiations will, inevitably, color how the sides view our report. But I want to make something clear: This report was done to examine how the plan is working so that the people of Cleveland understand how important it is to pass the district's levy renewal this November. One of the major flaws with the Plan was the state never put additional investment into its components' success. So the local community has had to step up to a much greater degree than it should have had to. And it will still need to do that.

It is a credit to the people, businesses and philanthropies in Cleveland that they have stepped up for Cleveland's kids thus far. And while the labor issues pose a real threat to the Plan's future, the potential for the Plan to usher in a new era of better, more accurate teacher evaluation and pay structures should not be understated. There is great hope here, but the district and its teachers need to work this out. Quick. Because there's little question that if the levy fails in November, even the modest improvements the district has seen will quickly fall away.

We went out of our way to not use district or union data in the report (and we got plenty of it), sticking rather the state data so that we couldn't be accused of siding one way or another. While I certainly spoke with and interviewed both sides, I chose to use state-reported (and national) data to do the analysis. There will be things in the report both sides will like and not like. But I did my best to, on the data, stick with a neutral party. Though I know some will suggest otherwise. That comes with the territory.

Here's our summary of the report:

"For the first time in decades enrollment in Cleveland public schools has increased. Graduation rates have also increased and proficiency test scores have improved relative to other large urban school districts.
These are a few of the positive developments in the Cleveland schools, but many challenges still remain. Fourth grade reading and math scores have seen a slight uptick, but are still very low nationally. Excessive state funding cuts and prolonged tension between the districts' teachers and administrators have made the path forward difficult. 
Our latest report examines these successes and challenges of the Cleveland Plan for Transforming Schools."

Monday, August 22, 2016

Testing the Boundaries: Part II - What Can Outliers Teach Us?

We've spent 12 years obsessing over two things: tests and achievement gaps. Yet it appeared that the efforts have been for naught, as the achievement gap has continued to grow.

However, there is some good news. The state's poorest districts saw the biggest percentage improvements in their test scores over those 12 years. The bad news is that improvement, while slightly narrowing the raw score performance gap between rich and poor districts, widened the state's relative performance gap. Meaning, it wasn't big enough to make a difference, relatively speaking, because high performing districts also significantly improved their scores.

You can see the problem in the following charts. Pay attention to what happens between the Suburban, very low student poverty and Urban, very high student poverty categories. You'll see impressive improvement among the poor category, which includes Ohio's so-called "Big 8" urban districts -- Akron, Canton, Cincinnati, Cleveland, Columbus, Dayton, Toledo and Youngstown. These are the districts state lawmakers have targeted for years as "failing" kids. Though, looking at the raw score improvement, one can't be anything but impressed by the 12-year jump.

Here's the problem: All districts improved, even the wealthiest districts. So despite the state's poorest districts' impressive improvement, all the other districts' improvement was enough to widen the relative performance gap so that the wealthiest districts made up a significantly higher proportion of top 10% mini-PI scores in 2014-2015, and the poorest districts made up a significantly higher proportion of the bottom 10% mini-PI scores.

But it cannot be ignored that Ohio's maligned, urban districts improved their math scores by more than 21%, and the improvement among all districts with high or very high poverty saw scores jump by 20% or more. That's impressive. But it wasn't enough to improve their status among Ohio's districts. And, in fact, they dropped in comparison with their wealthier brethren.

So what gives? Why did a 20% improvement lead to the widening of the achievement gap? A major reason is the upper limit of the mini-PI score. The upper limit of the PI score is 120. That's the maximum score any district could receive. And while some districts get close, getting a perfect 120 is probably impossible. So in the 2003-2004 school year, the average PI score for the OGT math score in the state's wealthiest, suburban districts was 102.84 -- a mere 18.16 points from perfection. So the greatest percentage improvement they could have was a bit more than 17%. The average for the state's poorest, urban districts was 68.26 -- 51.84 points from perfection. So the greatest percentage improvement they could have was 76%. 

In the intervening years, the wealthiest districts went up to an average of 112.25 -- an amazingly close 7.75 points from perfection, or more than 1/2 of their maximum percentage improvement. And the state's urban districts went to 82.26 -- less than 1/3 of their maximum percentage improvement. 

So what this meant is more suburban districts performed at an elite level, while the state's urban districts, which performed much better, remained mired in the bottom 10% of mini-PI Scores.

But this doesn't mean there aren't outliers. Take Lakota in Sandusky County -- classified by the state as a rural, poor district. In 2004, their OGT Math score placed them 511 of 608 districts. Not good. But in 2015, their score jumped so much that they are now in the top 1/3 of districts. Or Maplewood in Trumbull County. In 2004, they were in the bottom 1/2 of Ohio districts on OGT math scores. Now, they're in the top 5%.

In fact, the greatest rank jumpers in the state were the rural districts. They made up almost 1/2 of the greatest rank improvements. 

Here's the other problem: They also made up about 40% of the greatest rank fallers too.

Tellingly, no urban district was in the top 10% of climbers or fallers.

Again, is there something interesting going on in the districts with the greatest improvements? Or something awful happening in the fallers? Or is it just statistical noise -- a true outlier? One would have to go to the district and see what, if any, change has led to the relative improvement. But given the predictive force of poverty on these scores, normal statistical variation would seem to explain much of this difference.

What does all this mean from a policy perspective? Well, it means that if improvement or performance slippage can be explained in large part by statistical variation, should we be granting kudos or shame to districts that grow or slip? It'd be easy to devise a method to reward districts that show dramatic improvement on scores, even if their relative performance to other districts remains low. Likewise, it can be easy to punish districts that slip. But should we, given what we know about statistical variation in these cases? There will always be outliers -- districts that out- or under-perform their demographics. But if there's one thing variation means it's that over or under performance could, in fact, be nothing more than yet another expected statistical result of standardized testing. Every district has their strong or weak classes pass through the system.

In other words, a district's score may not be indicative of the actual quality of their educational program. Yet, under our current accountability system, it is the dominant determinant of a school or district's quality.

But that doesn't mean these score changes mean nothing. Figuring out what they mean is a great challenge, but one that must be unlocked in order to understand how, if at all, they should inform our nation's education policy.

Testing the Boundaries
Part III - Finding Excellence through the Noise