University of Arkansas Office for Education Policy

What’s the Deal with NAEP Results?

In The View from the OEP on November 18, 2015 at 11:49 am

kidmath2

Today we dig more deeply into the recently released National Assessment of Educational Progress results- examining how Arkansas students scored compared to neighbor states and other states that adopted, and did not adopt, Common Core State Standards.  For more detailed information read the full policy brief.

Two weeks ago we discussed that Arkansas’ scores had declined somewhat across the board,  and that 4th grade Math scores had declined significantly since the last NAEP assessment in 2013.

That got us to wondering- why? So we asked a series of questions:


Was it just us? No, but in our neighborhood- yes.

 

A significant decline in scores feels bad, but we feel better is we aren’t declining alone.  Nationally, there was a decline in 4th grade Math scores as well, but not for our neighbor states.  We examined the 4th grade Math performance of Louisiana, Mississippi, Missouri, Oklahoma,  Tennessee and Texas, and found that while our scores decreased, on average their scores increased!  For the first time since 2000, Arkansas’ border states surpassed our performance in 4th grade Math.


Was it because of Common Core? It’s hard to tell, but probably not. 

 

Common Core State Standards (CCSS) have been a much debated topic over the past year, and are currently being revised to ensure they meet Arkansas’ needs.

Pursuing the question of whether or not the full implementation of CCSS in 2014-15 could have impacted Arkansas’ performance on the NAEP, we compared Arkansas with other states that have implemented the standards as well as those that have not (a full list is in the policy brief).

As seen in Figure 1, non-CCSS states did not experience a decline in 4th grade Math scores in 2015.  Other states that implemented CCSS experienced a slight decline, but it was not as significant as Arkansas’.

Figure 1: NAEP Mean Scaled Score for 4th Grade Math: Arkansas, CCSS States and Non-CCSS States, 2003-2015CCSSvNon2

The fidelity and quality of CCSS implementation varied across states, and perhaps elementary math standards were particularly difficult for Arkansas teachers to implement effectively. We must conclude, however, that Common Core State Standards are not the sole contributor to this decline in Arkansas’ performance.


Were these 4th grade students lower math performers on other tests too? Not really. 

 

Here at the OEP we always use multiple pieces of data.  Perhaps the students who took the 4th Grade NAEP in 2015 were just academically lower achieving than students who had completed the NAEP in previous years.  To examine this question, we considered the 3rd grade Math Benchmark performance for the years prior to the 4th grade NAEP administrations.  Although not exactly the same students, because only a representative sample complete the NAEP, we can assume that the students are essentially the same in regard to characteristics that would impact academic performance.

Figure 2: Arkansas 3rd Grade Benchmark Math Percent Proficient and Subsequent 4th Grade NAEP Math Percent Proficient,2005-06 through 2013-14

NAEP BM

The percentage of 3rd grade students scoring Proficient/ Advanced on the Math Benchmark steadily increased between 2005-06 through 2011-12, but shows a 3% decline between 2011-12 and 2013-14. The subsequent 4th grade Math NAEP exams reflected slight increases in proficiency over time, but  a steeper decline of eight percentage points from 2013 to 2015. The trend in the Benchmark patterns of the years are similar to that of the NAEP exams but does not present us with a cause for Arkansas’ significant decline in 4th grade Math performance on the 2015 NAEP.  The scores of 3rd grade students on the Arkansas Benchmark exam in 2013-14 did not indicate there would be such a significant decline on the 4th grade NAEP.


So What Should We Do?  Get More and Better Information! 

 

Digging into assessment results often leads us to more questions.  Unfortunately, it isn’t clear why the NAEP 4th grade Math scores declined so significantly between 2013 and 2015.  Recently released PARCC results, however,  indicate that fewer than one in four 4th graders scored proficient in math– that’s even lower than the NAEP results (see Table 1).

Table 1: PARCC and NAEP Percent Proficient, by Grade and Subject, 2014-15

PARCCNAEP

This spring, Arkansas students will take a new assessment, the ACT Aspire, making comparisons between states and from one year to the next difficult.  NAEP for 4th and 8th grade will not be administered again until 2017, which is a long time to wait to see if this year’s results were a minor blip or the beginning of a larger decline.  Our students can’t wait.

Many districts throughout the state are not waiting.  They are using high quality interim assessments that compare students to peers nationally.  Not waiting until 3rd grade to see how students are performing, many use short interactive assessments as early as kindergarten to ensure that students have a strong foundation. Not waiting for state assessment data to identify students in need of support, because these assessments can help schools understand where students are- not just those students that are behind, but also students that are ahead of their peers and can benefit from enrichment.  If we all don’t wait another day, and use data to make the most of every learning moment, and make sure that students are getting the instruction they need to grow, hopefully Arkansas NAEP scores will rebound in 2017, and our students will be more prepared for success.

 

 

 

Advertisements
%d bloggers like this: