University of Arkansas Office for Education Policy

Math Proficiency to Increase this Spring!

In The View from the OEP on March 2, 2016 at 12:02 pm

This spring we should see an increase in the percentage of Arkansas students found to be meeting performance expectations (‘on grade level’) in math. Arkansas students will be taking a new assessment, the ACT Aspire, and according to a new study from AIR, the achievement standards are significantly lower in mathematics than they were on the assessment students completed last year.

How many Arkansas students are performing ‘on grade level’ seems like a straightforward question, but there are actually many different answers. States set their own criteria for what it means to be ‘on grade level’ and which assessment will be used to measure student achievement of the criteria.  In the past, Arkansas measured student performance through the Arkansas Benchmark exams. Using these performance levels in Spring 2014, 78% of students in grades 3-8 were Proficient or Advanced in Literacy, and 72% were Proficient or Advanced in Math.  In Spring 2015, however, Arkansas administered a new assessment called the PARCC test.  Only 34% or Arkansas students Met or Exceeded Expectations in Literacy, and only 24% did so in Math.  The difference, as we have discussed before, lies in the change in assessments and the criteria to be considered ‘on grade level’.   This spring, Arkansas students will be measured against yet a different standard that seems to have a significantly lower criteria for ‘grade level’ performance, especially in mathematics.

How Do We Know?

 

There is an assessment given in every state that can be used as a common metric to compare state assessment results.  The National Assessment of Educational Progress (NAEP) in reading and math is administered to a representative sample of 4th and 8th grade students in every state every two years.  The authors of this study used equipercentile linking to benchmark state achievement standards against NAEP achievement levels. Think of it as determining the price of an item in dollars, euros and yen.

Because NAEP is only administered in grades 4 and 8, the comparisons are limited to those grade levels, but it sure is interesting!  The chart below shows the NAEP scaled score equivalent to being ‘on grade level’.  For NAEP, this is the Proficient category, for PARCC it is termed “Meeting Expectations” and for ACT Aspire the term is on track for “College Ready”.  Overblown terminology aside, the information shows why Arkansas students are more likely to be ‘on grade level’ this spring.

Figure 1. NAEP Equivalent Scores of Achievement Standards PARCC and ACT Aspire.

Screen%20Shot%202016-03-02%20at%2010.10.43%20AM

What Does This Mean?

 

4th grade reading: ↔ Not significantly different.

The NAEP equivalent score to meet performance expectations is the same for PARCC and ACT Aspire; 232. This is slightly below the score needed to be identified as Proficient on the NAEP.

8th grade reading: ↓ Significantly lower.

The NAEP equivalent score to meet performance expectations is significantly lower for ACT Aspire than for PARCC. A NAEP score of 273 was equivalent to Met Expectations on the PARCC assessment, but students would be identified as college ready on the ACT Aspire with NAEP score of 264. There is an effect size difference of .28.  Neither score was commensurate to the score needed to be identified as Proficient on the NAEP.

4th grade math:  ↓ Significantly lower.

The NAEP equivalent score to meet performance expectations is significantly lower for ACT Aspire than for PARCC. A NAEP score of 252 was equivalent to Met Expectations on the PARCC assessment, but students would be identified as college ready on the ACT Aspire with NAEP score of 235. There is an effect size difference of .55. While the PARCC score is slightly above the score needed to be identified as Proficient on the NAEP, the ACT Aspire equivalent score is well below.

8th grade math: ↓ Significantly lower.

The NAEP equivalent score to meet performance expectations is significantly lower for ACT Aspire than for PARCC. A NAEP score of 307 was equivalent to Met Expectations on the PARCC assessment, but students would be identified as college ready on the ACT Aspire with NAEP score of 290. There is an effect size difference of .48. While the PARCC score is slightly above the score needed to be identified as Proficient on the NAEP, the ACT Aspire equivalent score is below.

So What?

 

This can not be interpreted as a move from a ‘bad test’ to a ‘good test’ or vice versa. The comparison to the NAEP standards for proficiency simply helps us give some context to the changing assessment results for Arkansas students.  In addition, the NAEP standards for “Proficient” should not be interpreted as the criteria needed to be ‘on grade level’ or as predicting college success, but can help us understand that higher proficiency rates on the ACT Aspire may not reflect actual increases in student achievement.  As shown in this study, the achievement required to meet ACT Aspire’s performance standard is significantly lower than the PARCC standards in 8th grade English language arts and 4th and 8th grade math.  Given this information, if math proficiency rates increase in 4th grade this spring, we might be cautious in assuming it is because students actually know more math.

The big issue is that we don’t actually know what being ‘on grade level ‘ or ‘proficient’ or ‘college ready’ means in terms of performance later in life. A slightly lower standard on the ACT Aspire could be problematic if the standards are too low to provide meaningful information about student ability, falsely indicating to teachers that the student is doing fine when in fact he or she does not have the skills needed to be successful in college or careers.  Given that the ACT Aspire standards are still much more challenging than Arkansas achievement standards used to be, however, here at the OEP we aren’t too concerned about that.

After Spring 2017, when we have our second year of ACT Aspire assessment results, we will be able to more confidently identify positive or negative changes in Arkansas student achievement.

 

 

Advertisements
%d bloggers like this: