There was an error in this gadget

Tuesday, February 2, 2016

Proper Perspective: What Does Ohio's K-3 Literacy Data Mean?



Ohio’s K–3 literacy scores: Is the third-grade reading guarantee living up to its promise?

The first round of school report card data came out in January (expect the second batch February 25), shedding light on (among other things) how schools are doing in K–3 literacy. (Note that ninety-six schools have appealed their K–3 literacy grades, and data is under review for another seven schools, so take all of this with a cairn of salt.)

This year’s report cards are the first to include a letter grade for K–3 literacy, a metric that measures the improvement that schools and districts have made in moving students from ‘not-on-track’ to ‘on-track’ and eventually proficient, according to Ohio’s third-grade reading assessments. Reading ability by the end of the third grade is highly predictive of later schooling (and life) success. Recognizing this, Ohio adopted the third-grade reading guarantee in 2013 in order to ensure that students are proficient readers before moving onto the fourth grade.

Here’s what the 2014–15 data says:

Out of 609 school districts, 452 (74 percent) received a K–3 literacy grade. The rest were not rated (NR). It’s unclear why 157 districts didn’t receive a grade. Unlike school building rating data, most districts in Ohio serve students from kindergarten through third grade, so it’s not due to grade configurations. Because K–3 literacy only gauges kids already “off track” in reading, it’s possible that small districts have too few students to count and larger but higher-performing districts simply have very small numbers of kids not already reading on grade level (e.g., Beachwood City or Grandview Heights).

The grade breakdowns are as follows:

Out of 3,415 public schools (district and charter included), only 1,384 (41 percent) received a grade. While some buildings obviously received “NR” because they are high schools or middle schools, this doesn’t account for over two thousand schools that went unrated. Many of these buildings had too few struggling readers to receive a rating—though it’s impossible to distinguish whether this is because of sheer small size or because they simply don’t have struggling readers (as is likely the case with higher-performing schools).

The grades are broken out as follows:


Jamie's Take:

The K–3 literacy grade is new this year, so we can’t track progress over time to determine whether Ohio’s third-grade reading guarantee is having an impact on early readers. I will say, however, that I’m less impressed with the numbers than Stephen. He wrote on his blog the day the data came out that “only 668 third graders weren't deemed proficient in reading” and concluded that “having 99.5 percent of third graders be proficient seems like as close to a third-grade guarantee as you can get without getting to 100 percent.” If this is the case, then we may as well hang up our hats and go home. Ohio’s reading guarantee has worked after only two years.

Except it hasn’t. In parsing the data, 858 students statewide failed the third-grade reading exams and were not on a Reading Improvement and Monitoring Plan (RIMP). This means there were many more students who failed but were on a RIMP, and their failure rates aren’t yet being reported. In other words, some of the data reported is not all that useful, and we should be careful not to declare victory prematurely. According to ODE, the K–3 literacy measure “identifies students who were never on or removed from a Reading Improvement and Monitoring Plan but do not achieve proficiency by the end of third-grade.” So all that number really tells us is the number of students who were not placed or kept on an improvement plan but probably should have been. Reading proficiency scores released next month will better tell us whether Ohio is headed in the right direction.

Moreover (and as my colleague Aaron Churchill noted in his scorecard of Ohio’s report card), it seems problematic that the K–3 metric is premised on having readers (in any grades K–3) who are deemed “off track.” Schools without off-track young readers end up receiving a “not rated” label. K–3 literacy data is not calculated for a remarkable number of schools (not just those schools without grades K–3). Thus, we essentially have no data about how many schools’ kindergarten, first-, second-, and third-grade students are faring. Struggling readers who happen to attend an otherwise high-performing school (or a small one where there are too few students in each grade to calculate a rating) aren’t reflected in the K–3 literacy metric at all. It would be more useful to know how all kids in the early grades are progressing. In sum, while it’s still a good idea to focus on early literacy, the jury is still out on whether the third-grade reading guarantee is “working” or not—and we may want to question exactly whom it’s working for. 



Steve's Take: 

There is little question that starting when the Evidence Based Model required Ohio school districts to offer free All-Day Kindergarten, districts began adopting it more frequently. Story after story has noted how many more districts are offering ADK for their children, not just in Ohio, but nationwide. The reason is very simple: ADK works. It helps children learn more and catches kids up who may have started their Kindergarten year behind.

The overwhelming evidence supporting ADK’s positive effect on student achievement is the reason why, when I was the Chairman of the Primary and Secondary Education Subcommittee of the House Finance Committee, I stripped out all the mandates from the Evidence Based Model … except the ADK mandate. I did allow for waivers because I wanted districts to have to explain why they were doing something other than ADK.

And while some districts gave up ADK after Gov. Kasich and the General Assembly repealed the EBM in 2011, the state’s adoption of the Third-Grade Reading Guarantee in 2013 led to more districts adopting ADK. The thought was ADK would help kids catch up earlier so more costly, later interventions would be avoided.

New K-3 Literacy data released by the State of Ohio seems to validate districts’ investments in ADK. The new report card includes the number of children who are behind in reading in each of the K-3 grades and how many get caught up to grade level by the end of the year.

The two most significant improvements happen in Kindergarten and 3rd Grade. The 3rd Grade result shouldn’t be surprising – it’s the last time students have to be caught up for the Third-Grade Reading Guarantee, meaning they can move on and not be held back a year. So there’s a logical focus on 3rd Grade improvement.

However, the similarly huge improvement in Kindergarten is less intuitive, which tells me that districts’ recent investment in ADK programming has helped.

Buttressing this conclusion is the fact that in 1st and 2nd Grade, the improvement rates drop by more than ½.


The lesson is simple: Invest in staff and children and kids will thrive. There is no similar push to improve reading in 1st and 2nd Grade as there has been for ADK, and, not surprisingly, the improvement isn’t nearly as stark in the later grades. Let’s hope policymakers notice these data and step up investments in the intervening grades so kids aren’t forgotten during those equally important times. Their decisions have real impact for kids, as ADK and the Third-Grade Reading Guarantee demonstrate.

While I’m still reticent to read too much into the latest report card data because of the inordinate number of appeals over the grades, I think it’s helpful to look at the data trends. And one of the most significant ones I see is that ADK appears to be a sound investment.

3 comments:

  1. Steve and I are on common ground in supporting full-day kindergarten and high-quality early learning generally—especially targeted to low-income children. The latest rankings out about preschool are worrisome: according to the Dispatch’s coverage this morning only 4 percent of Ohio’s low-income four-year-olds are enrolled in pre-kindergarten. (Of course, ensuring that pre-K and K are high-quality is a whole separate conversation.)

    I think Steve is right in observing that students in certain grades will see greater improvements because of the consequences attached at that particular juncture. Ohio schools have every incentive to emphasize third-grade reading and even to staff accordingly with the strongest teachers in that grade. And he’s right that we can’t forget kids in the other grades.

    The kindergarten improvement seen in the latest data from ODE could be due to all-day kindergarten, or to the simple fact that kids have fewer skills to make up at such an early stage. Speaking anecdotally, I used to teach kindergarten and while it’s never easy to bring students up to speed when they come to school with missing skills, it was far easier to teach the alphabet, phonics, and basic decoding to a five-year-old than it was for my first- and second-grade teacher colleagues to do the same for seven- and eight-year-olds. Overall I’m glad to see the emphasis on K-3 literacy in Ohio but hope we can refine the metric so that it gauges learning outcomes for all children, not just those deemed “off track.”

    Jamie

    ReplyDelete
  2. This comment has been removed by the author.

    ReplyDelete
  3. While I appreciate Jamie’s concern about what the new K-3 Literacy report card data means for the Third-Grade Reading Guarantee, I want to address her critique of my prior blog post about the extraordinary low number of kids in Ohio who didn’t pass the third-grade reading proficiency test and don’t have a reading improvement plan.

    I chose to see that if 99.5% of Ohio’s third graders were either proficient or had a plan in place to meet that proficiency target, then Ohio’s school districts were doing a pretty good job catching up these kids and not letting them fall through the cracks. Yes, a few did. But out of 123,000 third-graders, not even 700 fell through the cracks statewide. That’s not bad. That most of those issues occurred in districts where we would typically expect this to happen is concerning, as is the fact that Ohio’s charter schools let a higher percentage of kids falls through the cracks than districts.

    But I think overall, these data are a good sign.

    Just under 10% of Ohio’s third graders didn’t pass the reading proficiency test last year (though, as I mentioned, only a relative handful didn’t have an intervention in place to fix that). And that’s too high. In education and medicine, a 10% failure rate isn’t good enough.

    As I said in my commentary, I’m reticent to read much into these results. And not just for Jamie’s concern that not many districts or schools actually received ratings under this K-3 Literacy rubric. But also because so many of these scores are being appealed. So we may be writing very different things depending on their outcome. However, I’m more encouraged by these pretty good Ohio results than Jamie seems to be.

    ReplyDelete