University of Arkansas Office for Education Policy

ACT Aspire: The Summer Sequel with a Bummer Ending

In The View from the OEP on July 12, 2018 at 4:47 pm

This summer it seems like all the movies are sequels: and the preliminary scores from the spring administration of the ACT Aspire for the 2017-18 school year seems like a sequel too!

Unfortunately, even though student performance hasn’t changed, ESSA Weighted Achievement scores will decline for most schools!! This bums us out because we hate it when systems send inconsistent messages about how well schools are doing. 

In today’s blog we share new data visualizations, review ACT Aspire performance patterns by subject overall and by grade, and explain why your school’s ESSA achievement scores will likely be lower this year than last year.


Show me the Data!

We are excited about some new interactive data visualizations of performance patterns across the state. Maps are available for both district and school-level, and you can select specific districts/schools and see how they scored. Unlike other data currently available, we have combined the grade level results to provide an OVERALL score for each district/ school.

We use the OEP GPA as this overall indicator of performance. The OEP’s GPA is a weighted measure of student performance that gives the most credit to students who have exceeded expectations and the least credit to those that are in need of support. In this GPA measure, we treat the ACT Aspire test scores similar to the familiar grade point average for individual students: 1.0 is the lowest score, indicating that all students in a districts were In Need of Support, while 4.0 is the highest score, indicating that all students in a districts were Exceeding Expectations on the ACT Aspire.

ACT Aspire_map

As you can see in the maps above, districts in the upper left hand half of the state are more likely to be blue, indicating higher performance. This is not surprising since we are showing performance on the ACT Aspire, which is highly correlated with the percentage of students in the district participating in the Free/ Reduced Lunch program.

If you want to see more detailed district- and school- level information, both by grade level and OVERALL, you can find it on the OEP website here! Also, check out our interactive data visualizations of district and school performance. You can select specific districts/schools (no limit) and see how they scored, or limit the view to particular regions or levels of FRL participation.


ACT Aspire Performance:

The figure below presents the percentage of Arkansas students who met or exceeded expectations in each content area by year since we began administering the ACT Aspire in 2016. Overall, the percentage of students meeting or exceeding benchmarks stayed exactly the same in mathematics, science, English, and reading. This year, unlike prior years, writing scores were not provided. Note: From here on, we use the Benchmark percent meeting or exceeding benchmarks so we can compare Arkansas performance with national results.

Figure 1. Percentage of Arkansas students meeting or exceeding expectations on the ACT Aspire, by content area, 2016 to 2018.

ACT_Aspire_18_Text

While overall ACT Aspire results did not change at all, there were some changes by grade level within each content area. We examine each content area by grade level and consider the national average performance as well.


Math: Performance in math declined somewhat in the middle grades, with 6th grade evidencing the greatest decline from the prior year (-6 percentage points). Since 6th grade experienced an increase of 7 percentage points in 2016-17, this year’s performance is similar to that of 6th graders in 2015-16. There were increases in the percentage of 8th, 9th, and 10th graders meeting or exceeding expectations. This continues a pattern of improvement in these grades, which is good news, since these grades have lower math performance than earlier grades. Math performance in 7th and 8th grades was above the national average (indicated by the red diamonds).

Figure 2. Percentage of Arkansas students meeting or exceeding expectations on the ACT Aspire Math, by grade, 2016 to 2018.

ACt_Math_Grade_2018


Science: Performance in science was generally consistent with prior performance, but declined somewhat in the middle grades, with 6th grade again evidencing the greatest decline from the prior year (-3 percentage points). Even with the decline, however, 6th grade maintained the highest performance across the grades. Science performance in 4th, 7th, and 8th grades was above the national average (indicated by the red diamonds).

Figure 3. Percentage of Arkansas students meeting or exceeding expectations on the ACT Aspire Science, by grade, 2016 to 2018.

ACT_Science_2018


English: Performance in English was generally consistent with prior performance. Scores for 8th grade students increased by 2 percentage points. English performance in grades 3-8 was above the national average (indicated by the red diamonds).

Figure 4. Percentage of Arkansas students meeting or exceeding expectations on the ACT Aspire English, by grade, 2016 to 2018.

ACT_English_Grade_2018


Reading: Performance was generally consistent with prior performance. The greatest increase was for 5th graders, with a 4 percentage point increase. Reading performance in grades 3-8 was at or near the national average (indicated by the red diamonds).

Figure 5. Percentage of Arkansas students meeting or exceeding expectations on the ACT Aspire Reading, by grade, 2016 to 2018.

ACT_Reading_Grade_2018


Plot Twist:

Every good sequel has a plot twist, and this year’s ACT Aspire results are no different. The one that has everyone scratching their head (okay- maybe just us here at OEP) is the updated cut-points for ELA and STEM readiness benchmarks. As you can see in the figure below, these new cut points (which are, according to ACT Aspire, more aligned with the performance expectations of the ACT) decreased the percentage of students meeting readiness benchmarks. In ELA, the change was a decrease of 8 percentage points from last year, but in STEM the new cut points resulted in a 25 percentage point decrease.

Figure 6. Percentage of Arkansas students Meeting Readiness Benchmarks on the ACT Aspire.

ACT_readiness_2018

The decrease in STEM readiness is startling, especially considering that we saw NO CHANGE in statewide math or science performance, but won’t have an impact on schools because STEM readiness is not used in accountability.

The decrease in ELA readiness, however, will impact schools because ELA scores are used to calculate the Weighted Achievement score in the ESSA school index. As we’ve discussed before, although stakeholders intended for growth to ‘count more’, Achievement continues to be the primary factor driving school letter grades. On a positive note, however, these changes to the ELA cut points should not impact growth scores in any way!

Decreases in ELA readiness are apparent in grades 4-10, with declines in the double digits for 6th, 7th, and 8th grades. This means that schools serving students in these grades are going to receive LOWER achievement scores on the ESSA index even though performance in English and reading was unchanged. It is unfortunate that this change was made, as Arkansas schools were just gaining familiarity with the ACT Aspire and the new ESSA metrics.

ACT_ELA_2018_arrow


So- what are the big takeaways from the preliminary ACT Aspire results? 

  • Performance on the ACT Aspire is related to school/ district poverty rates
  • Performance is generally the same as last year
  • Arkansas is scoring at or near the national average in most subjects/ grades
  • ELA cut points for meeting readiness benchmarks were changed this year by ACT Aspire, so schools should expect to have lower Weighted Achievement scores than last year on the 2018 ESSA reports
  • Growth scores should be unrelated to changes in the ELA and STEM cut points.
  • You can use the data resources from OEP to see overall district and school values
  • Data visualizations can help us see statewide patterns in performance and compare performance to other schools/ districts of interest.

We are happy to be able to share these resources with you and looking forward to seeing the Growth Scores on the ESSA reports in September. Stay tuned to OEP for more info.

Update 7/23: The cut scores for 2018 A-F school letter grade assignment have been lowered to compensate for the new ACT Aspire ELA cut points. You can find the new cut points here. 

 

 

Advertisements
  1. I read that! This test isn’t appropriate for accountability. However, it is what it is!

    • Hi Sandra- Thanks for reading and thanks for your comment! The ACT Aspire provides information regarding student achievement and growth as indicated in the state’s ESSA plan. Each state developed their own plan for how to measure student achievement, and were able to select what type of assessment to use for accountability. Louisiana and New Hampshire may soon be named as the first two states to pilot innovative assessments. Hopefully, the pilot will provide information about measures of student understanding that could be more sensitive and informative than our current assessments.

Comments are closed.

%d bloggers like this: