University of Arkansas Office for Education Policy

Are We Closing the Achievement Gap?

In The View from the OEP on April 10, 2019 at 1:23 pm

ednext_XIX_3_hanushek_img01Yesterday, EducationNext discussed new research demonstrating a persistent 50 year achievement gap between “Haves and Have Nots”, and we wanted to chime in about what the gap looks like for students in Arkansas.

While the national and Arkansas-specific research use different data sources and methodologies, the conclusions are the same: the gap in academic performance between students with fewer and greater economic resources is large and isn’t getting smaller.

The national research used an index of student socioeconomic status (SES) developed from student-reported information about the education level of their parents and possessions that they have in their home. The achievement data come from four testing programs: National Assessment of Educational Progress (NAEP) Long-Term Trend;  Main NAEP; the Trends in International Mathematics and Science Study (TIMSS); and the Program for International Student Assessment (PISA). The researchers found that the gap in achievement between the highest 10% SES and the lowest 10% SES students has remained consistent over the last 50 years.  The conclusion holds when the ranges are expanded to the highest and lowest SES quartiles as well.

What about Arkansas’ Gap?

Here at OEP, we looked into the achievement gap for Arkansas students and found similarly disheartening results.

For the Arkansas analysis, we use the student-level indicator of Free-Reduced Lunch (FRL) program participation as the indicator of student socioeconomic status because we don’t have information on parental education or home possessions. FRL eligibility is based on family income, and is often used as an imperfect proxy for a student’s socioeconomic status. We group students into two groups: Not participating in FRL, or participating in FRL. In 2008-09, the first year of our analysis, 56% of Arkansas students participated in FRL.  The percentage increased to 59% in 2009-10, and has remained fairly consistent since then, ranging as high as 61% before declining slightly to 60% in 2017-18 .

For the achievement data, we use ten years of the annual Arkansas state assessments, which have changed over time. For the first six years of our analysis we use data from the Benchmark and End-of-Course exams. In the 2014-15 school year, students in Arkansas completed the PARCC exam, before switching to the ACT Aspire exam for 2016-17 and beyond.

All of these test changes have made it essentially impossible for Arkansas schools to examine achievement gaps over time, because each test scores and reports performance in different ways. Although the assessments have varied over time, each provides a measure of how students statewide perform in literacy and mathematics. We think it is very important for education stakeholders to be able to examine performance over time, so we used a common standardizing procedure to track the relative performance of different groups.  We transform these scores into percentiles to aid in interpretation. The statewide average percentile in literacy or mathematics at each grade level is the 50th percentile each year. Note that the percentiles are standardized within year and state, meaning that they are not indicators of how ‘true’ achievement has changed over time or how performance compares to students in other states. Percentiles are used to compare the relative annual performance of Arkansas student groups over time and examine if the gap in achievement has changed (details about the methodology can be found at the end of this blog).

We found that in Arkansas, the difference in achievement between “Haves and Have Nots” has remained essentially unchanged over the past decade.

As seen in Figure 1, Arkansas’ literacy achievement gap closed slightly from 25 percentage points to 23 percentage points over the past 10 years. In 2008-09, students participating in the FRL program scored at the 39th percentile in literacy, while students who were not participating scored at the 65th percentile on average.  Over time, the performance on non-participating students remained consistent, while FRL-participating student performance increased slowly over time to the 42nd percentile by 2017-18. The literacy achievement gap closed slightly from 25 percentage points to 23 percentage points over the past 10 years.

FRL LIT

As seen in Figure 2, Arkansas’ mathematics achievement gap increased slightly from 20 percentage points to 23 percentage points over the past 10 years. In 2008-09, students participating in the FRL program scored at the 42nd percentile in mathematics, while students who were not participating scored, on average, at the 62nd percentile.  Student performance fluctuated over time for students in both groups, but the performance of non-FRL participating student increased slightly over time, while the performance of their FRL participating peers remained steady, leading to the slight widening of the achievement gap in mathematics.

FRL Math

It is interesting to note that the achievement gaps in both literacy and mathematics were smallest in the 2014-15 school year, the first (and only) year that Arkansas students completed the PARCC assessment. The gaps closed due to an increase in the average performance of FRL-participating students AND to a decline in the average performance of non-FRL participating students. It is tempting to hypothesize causes behind this decreased gap, but futile because we have only a single data point, and the gap then returns to established levels.

Well Shoot!

As with the national research, we feel these are important things to consider as education/ the level of education attained is key to upward mobility. Both the national and Arkansas results show that certain subgroups are limited in their ability to achieve success academically in comparison to their peers, and that the achievement gaps have persisted over time.

Therefore, it is seen that schools in Arkansas and across the country are not truly meeting the challenge of reducing those disparities among these subgroups. How can that be addressed?

We need to keep digging in order to get to the root of this achievement gap issue. OEP is digging into these gaps more in depth, and will share more information about trends in grade-level achievement gaps, as well as gaps between racial/ethnic and gender groups. We look forward to highlighting the Arkansas schools and districts that HAVE closed the achievement gaps for some student populations.

 


About the methodology:

Students in grades 3-8 were consistently assessed in both content areas, but there was variation in when high school students were assessed. For our analysis we used all grades assessed in literacy and/ or math in a given year, and limited our analyses to students completing the general assessment.

Scale scores were standardized for each year within grade level and content area, creating a Z-score (mean of 0, standard deviation of 1).  These z-scores were then averaged across student groups and transformed into percentiles to ease interpretation. The statewide average percentile in literacy or mathematics at each grade level is the 50th percentile each year. Percentiles are standardized within year and state, meaning that they are not indicators of how actual achievement has changed over time or how performance compares to students in other states. Percentiles are used to compare the relative annual performance of Arkansas student groups over time and examine if the gap in achievement has changed. 

  1. I appreciate this blog post and found it to be informative and insightful. I do, however, reach very different conclusions. I think part of this issue is how we discuss it. We are talking about 50 years of “achievement gap,” a term I wholeheartedly reject. This is an opportunity gap. To fix it, we need to put way (scientific term…) more resources in our lower income communities and their schools. The richer schools already do this through property taxes in the Arkansas funding formula. We must shift the paradigm from blaming these schools and communities for their low achievement to taking action to help them. The concept of the achievement gap, more or less, is the equivariant of blaming the poor for being poor. There’s no magic school or elixir that will fix this. It’s going to take social context reform, plain and simple. We can start by redistributing the funds from Little Rock differently and stop supporting and publicizing these heinous program that promote false competition like school grades and bonuses. I implore the Office for Education Policy to stop using the term “achievement gap” when in all actuality, an opportunity gap is what you mean. Again, I appreciate the post and opportunity to respond. Chris

    • Hi Chris!

      Thanks for reading and sharing your thoughts! We made the choice to use the term ‘achievement gap’ to be consistent with the terminology used in the Hanushek et al. study and because our data are showing just that- a persistent gap in achievement. When we look at the district financial information for Arkansas school districts, we don’t see a relationship between ‘richer’ schools and increased per student funding. In fact, we see a slight positive relationship with school poverty as the correlation between district percent of students participating in free/reduced lunch and per pupil unrestricted state funding is 0.08. This progressive funding aspect is also shown in Matt Chingos’ work.

      As you know, equal funding, however, isn’t always equitable, as it may be more expensive to provide the same quality of education to children who face greater achievement challenges. This is a complex issue for sure. Arkansas provides additional resources for students with particular characteristics/needs through categorical funding (ALE, ELL, NSL, etc). Although some research has found that schools serving disadvantaged students find it harder (or more expensive) to recruit and retain high-quality teachers, our research on teacher supply in Arkansas indicated that teacher supply was not significantly related to a district’s percentage of students participating in the free/reduced lunch program nor to the percentage of students that were identified as traditionally underserved races/ethnicities. A

      We agree that there could be some productive adjustments to the funding model. One we would like to see is providing special education funding on the individual student level rather than, as is the case currently, embedded in the funding matrix (you can read more about this here) Our overall concern with adding more money to the pot or re-allocating the money that we do have is that there is little evidence the funding is making significant improvement for the students that are struggling academically. We are super interested in the Return On Investment, to make sure that students are getting the best opportunities to learn. We look forward to the school-level expenditure information required under ESSA, so we can see how funding is distributed across schools within a district.

      We also have issues with the school grading process, but we like rewarding schools based on their student GROWTH, and think that you would too! You can check out our thoughts here https://officeforedpolicy.com/2018/10/31/lets-reward-growth/

      We agree that it is an opportunity gap, in part because students who don’t learn the needed skills in school miss out on lots of opportunities later in life.

Comments are closed.

%d bloggers like this: