University of Arkansas Office for Education Policy

Play It Again Sam… Letter Grades and ACT Scores

In The View from the OEP on October 17, 2018 at 12:20 pm

It’s been a big week in Arkansas education, with A-F Letter Grades and ACT scores being released, but both scores are generally a repeat of last year’s results.

As expected, the majority of schools (67%) received the same letter grade as last year.  There was some movement up a level (16% of schools) or down a level (17%) of schools, but these were generally more reflective of schools crossing over a classification threshold than significant changes in performance.  ACT scores for Arkansas’ 2018 graduates were just the same as they were for the Class of 2017. The average composite score of 19.4 and the determination that 17% of Arkansas graduates met the readiness benchmarks in all four tested areas are both below the national average and exactly the same as last year’s results.

How are school leaders, parents, and policy makers to interpret the static results?  Here at the OEP, we hope to share some tools for interpreting the new data. We also want to get into the weeds about how the Achievement and Growth scores interact in the ESSA model.

Before we jump into the details, we have some good news to highlight!

First- we congratulate ADE on getting the school performance information out so quickly this year. This helps schools evaluate how their school is serving students and make changes in a timely manner to address areas where the numbers are low.

Second- we celebrate the transparency that ADE has built into the system, and we fully support Arkansas’ decision to provide all students the opportunity to take the ACT for free.  These are positive decisions for students!  We love the public availability of the school performance reports through myschoolinfo.arkansas.gov– especially the option to compare achievement and growth performance with similar schools.  This option is available within the Reports tab by selecting similar schools based on %FRL, racial percentages and/or geographic proximity (see images below). If you haven’t checked this out, you should!

School filter

apply filter

 

 


The data is a lot to take in, and it can be tough to try to figure out what’s important and what the patterns are. We have posted this interactive data viz of the Letter Grades and associated scores to help you see a statewide picture.  You can find your school, and filter by academic growth score and poverty rate to see how schools that are similar to yours have scored.

OEP’s Interactive 2017-18 ESSA Data Visualization

ESSA Viz

Even with all the data out there, however, we are concerned that it may be difficult for many stakeholders to understand how their school is performing. In addition, we think each indicator provides insight independently, and so we made these simple one-pagers for easy reference.  To ease interpretation for stakeholders, we assigned each school a percentile rank (within their assigned grade span) for the Achievement, Growth, and School Quality indicators.  While this is too superficial for school personnel to act upon, we hope you find it a helpful communication tool. You can get them from our website officeforeducation policy.org, or here:

OEP’s One-Page School Summary of 2017-18 ESSA Data

One Pager


Now we are going to dig into the school performance data, and consider the new letter grades. Don’t worry – there are pictures, and you can download the data we used here!

As expected, the majority of schools (67%) received the same letter grade as last year.  There was some movement up a level (16% of schools) or down a level (17%) of schools, but these were generally more reflective of schools crossing over a classification threshold than significant changes in performance. Overall, the percentage of schools receiving each letter grade was similar to last year.

Figure 1: School Letter Grade Percentages, 2016-17 and 2017-18.

2017-18 LetterGrade Chart


Certain types of schools are more likely to get A’s and B’s.

Once again this year, we find that schools serving a lower percentage of students who participate in Free/Reduced Lunch (FRL) generally get better grades than schools who serve a more disadvantaged population.  As you can see in Figure 2 below, there is a strong negative relationship between FRL and ESSA scores (r=-0.69) although schools with the same %FRL have a range of about 20 ESSA points. Take note that there are a lot of schools reporting 100% FRL, which is the result of districts participating in the community eligibility provision.  We like the CEP, but some of these schools reported much lower FRL rates in years past, so we should use caution when considering the performance of these schools since it may not be an apples-to-apples comparison.

Figure 2: School ESSA Index Score and %FRL, 2017-18.

ESSA FRL

As we’ve said before, the ESSA index is mainly driven by achievement, and two scores are almost perfectly correlated at r=.97.  This is frustrating because the model was supposed to consider growth more heavily than achievement, but in reality achievement scores overwhelm growth. In addition, you can see in Figure 3 that this year’s achievement is almost perfectly correlated with prior year achievement (r=.94). So high achieving schools (generally lower FRL schools) tend to get better grades both years.

Figure 3: Weighted Achievement, 2016-17 and 2017-18.

Achievement 2 year


What about GROWTH?

The good news is that academic growth (our favorite) is less associated with school FRL rates than achievement (see in Figure 4 (r=-.25)). Growth is the indicator that measures how students scored compared to how well we predicted they would score based on prior achievement.  We feel it is very important to examine this indicator carefully, as it is the best reflection of the learning occurring in our classrooms.

Figure 4: Academic Growth and %FRL, 2017-18.

growth FRL

Interestingly, as you can see in Figure 5, academic growth is strongly correlated with prior year growth (r=.80).  This means that schools with high growth in 2016-17 also had high growth in 2017-18. The reverse is also true, unfortunately, as schools where students performed lower than predicted in 2016-17 demonstrated the same pattern in 2017-18.

Figure 5: Academic Growth 2016-17 and 2017-18.

Growth 2 year

This is great news for Arkansas educators!  We have an indicator of student learning that is not very correlated with school poverty, but seems to be consistently identifying schools as high, average, or low growth. Keep an eye out for the OEP awards, which will celebrate the high growth school!


But does increased growth relate to increased achievement, as we would think it should? The figure below represents the change in growth score from 2016-17 to 2017-18 and the change in achievement score over those same two years. Although increased growth does not always correlate with positive changes in achievement (because students can be achieving at a higher level than predicted but not necessarily making it into another proficiency category), there are few schools in the lower right quadrant where growth decreased and achievement increased. Of greatest concern are the schools in the lower left quadrant, where growth and proficiency both decreased since the prior year.

Figure 6: Change in Academic Growth and Achievement, from 2016-17 to 2017-18.

Growth Achievement


So what should school leaders, parents, and policy makers do?

  1.  Get into the data!  Understand and communicate how your school is performing on each indicator compared to similar schools.  If there is an indicator where scores are relatively high, build on that success.  If there is an indicator where scores are relatively low, consider what might be contributing to that score and work to change students’ school experiences. Stakeholders and policy makers should celebrate school successes, and work to support development in low-performing indicators.
  2. Focus on the learning every day! The ESSA Index, the ACT Aspire, and the ACT tests are once-a-year snapshots of performance.  Teachers should be using high quality ongoing formative assessments to understand where their students are and work to help them move them forward in their learning. Stakeholders and policy makers should support schools as they work to make an opportunity for growth for their students.
  3. Stay the course! Change takes time. To move the needle on student achievement and success in Arkansas, we need to put in the work and give the work time.  Changing assessments or backing away from rigorous and high-quality school performance analyses will only add instability into the system.

Arkansas education is on the right track, so let’s keep playing that tune.

 

 

 

Advertisements
  1. I love the data visualizations and the filter for poverty rate shows you some of the great news we are seeing for schools. For Arkansas, the best news is how all of the indicators work together and capture changes happening at the student level.

    1- The growth model at the student level is behaving the way we expected it to behave. For example, a student’s prior year achievement score (2017) is unrelated with the 2018 growth score–that is what we want! It means that students all along the achievement spectrum CAN AND DO MEET OR EXCEED their growth expectation every year. The converse is also true–that regardless of a student’s prior achievement–every student can AND SOME DO lose ground to expectations every year. This has been true since early modeling of the student level growth scores.

    Bottom line: A student’s achievement, whether high, low, average, very high, very low–you get the picture–does not prevent or bias a student from meeting or exceeding growth the next year!

    Even better, a student’s characteristics–economic advantage or disadvantage, race, disability or English learner status– does not have a relationship with a student’s growth score. So even though disadvantaged students are likely to have lower achievement, they ARE NOT likely to have low growth unless learning is not happening at expected or accelerated levels.

    2- The current year growth score and the current year achievement score at the student level have a moderate relationship-which means that when students are meeting or exceeding growth they tend to achieve at higher levels that same year! And when students are not meeting or exceeding growth, then they tend to lose ground that same year.

    Bottom line: If students are learning and growing in their learning then it shows up in their achievement.

    At the student level, the growth model levels the playing field for learners. So, what does that mean for teachers and schools?

    Focus on the learning of each and every student–that is what matters most. When students have access to and support to be successful in increasingly rigorous learning opportunities they grow in achievement. When this happens for more and more students within a school there is a snowball effect at the school level. It shows up in the Growth score, the School Quality and Student Success score, and the Achievement score.

    The ESSA School Index scores and its indicators were designed to serve as signals for support and improvement, not verdicts. OEP captures that well in this blog. Look at each indicator as a signal to help you dig deeper.

    Schools, use these signals to dig deeper into your local data and use a cycle of inquiry that informs learning for your students and classrooms. Enlist parents and district level staff to help support your school-level efforts.

    Districts, use these signals to think about how you allocate resources such as support and expertise to schools whose needs are greatest for improvement.

    It will take time to see if the theory of action behind the the ESSA accountability plan plays out as expected, and so far the early signals show we are moving in the right direction!

    • Thanks for the great comment! We agree that the growth model is working well and levels the playing field for learners. We also recommend that parents, educators, and policy makers look carefully at each indicator and reflect on what they could do to increase growth and learning for students. Change doesn’t happen overnight, and we need consistent measures over time to determine the result of the efforts that Arkansas is making to improve education in the state.

Comments are closed.

%d bloggers like this: