University of Arkansas Office for Education Policy

Do Students in Arkansas’ Gifted Programs Perform Better?

In The View from the OEP on May 12, 2021 at 12:56 pm

You might have heard lately that gifted programs don’t provide much of an academic benefit. The study, by Christopher Redding and Jason Grissom, was based on a nationally representative sample, and examined student test scores in addition to other student outcomes like attendance and engagement in school. The findings have caused some to question the value of gifted programming.

Here at OEP, we have been digging into gifted education in Arkansas. Our previous research found that 30% of the highest achieving 3rd graders are not identified as gifted, and that the biggest factor in a high-achiever not being identified is an economically disadvantaged background.

In our newest research, we examine how the longer-term achievement of high-achieving Arkansas students who are identified as gifted compared to similarly high-achieving students who are not identified as gifted. We operationalize high-achieving as scoring at or above the 95th percentile statewide on the 3rd grade state assessment. We follow five groups of these high achieving 3rd graders through 8th grade, and examine how their scores change over time.

You can read the policy brief or the full paper for more details, but we find large, statistically significant gains in academic achievement for high-achieving students who were identified as gifted. The relationship was more pronounced in mathematics achievement than in literacy achievement. The findings are consistent across our five independent cohorts.

For the purpose of illustration, check out the graphs below which represent the average statewide achievement percentile for the group of high-achieving students who were in 3rd grade in 2013-14 and 8th grade in 2017-18. The top graph (orange lines) shows mathematics achievement, while the bottom graph (blue lines) shows literacy achievement. In both content areas, although student performance was similar in 3rd grade, students who were identified as G/T consistently demonstrate higher achievement in every year that follows.

Average Percentile on Mathematics Assessment, Cohort 5. N=1,688
Average Percentile on Literacy Assessment, Cohort 5. N=1,615

A couple things are important to note:

  • The average achievement percentile for G/T and Non-G/T students drop in both math and literacy before rising again. When examining performance over time for a sample selected for very high achievement on the third grade test, we expect that the sample’s average score will move somewhat closer to the statewide average (the 50th percentile).
  • Students in our study completed three different exams over the time period examined: Benchmark, PARCC, and ACT Aspire. Although we standardized the scores to z-scores to allow comparison over time, the PARCC results for all of our groups are consistently lower than the preceding or subsequent scores. This group of students took the Benchmark exams in 3rd and 4th grade, the PARCC assessment in 5th grade, and the ACT Aspire in 6th through 8th grades.
  • These graphs are simple illustrations of descriptive trends, and do not control for any student or district characteristics.

In order to account for other factors that we believe would impact student achievement, we conduct multivariate regressions by year and subject for our 5 cohorts of high-achieving 3rd graders. We find G/T identification is associated with math scores that are between 10% and 39% higher (depending on the grade and year) than those of similarly high-achieving students who were not identified as G/T. In literacy the relationship was somewhat less pronounced, as G/T identification is associated with literacy scores that are between 4% and 24% higher (depending on the grade and year) than those of similarly high-achieving students who were not identified as G/T.

Even though this study does not provide causal inferences, it highlights a consistent positive association between gifted services and longer-term student academic achievement for those students that perform in the top 5% on third grade state assessments of literacy and mathematics. This is in contrast to other studies that have found little to no impacts (e.g., Adelson et al., 2012; Redding and Grissom, in press).

The association between academic growth and gifted education may range from curriculum, peer effects, to teachers’ ability to identify the right students who are most likely to benefit from gifted services provided, the motivational or labeling effect of being identified as gifted, in addition to the basic set of individual differences in characteristics or aptitudes that selected students may bring. While we cannot identify what aspects of gifted education in Arkansas casually contribute, individually or in combination, to increased student achievement, our findings are valuable because they provide an academic window into what happens from the 3rd through 8th grade to high achieving students across Arkansas who are and are not identified as G/T.

We note that state assessment scores do not address all the aspects of Arkansas’ G/T model and thus the associations we pick up may not necessarily capture those aspects of identification and programming.

However, it seems like the current G/T process in Arkansas is working for students. School districts at the minimum should keep their G/T practices to help high potential and ability students until any causal mechanism is detected. Though G/T seems to be associated with positive academic outcomes for students, this does not rule out improvements or expansions to the identification or programming processes that might be useful, such as using mathematics and literacy measures as selection tools not just as evaluation tools. Additionally, the success of Arkansas, in a sense, may illuminate useful strategies that may lead to more effective educational opportunities for high achieving students in other states and regions.

Addressing the Teacher Shortage

In The View from the OEP on March 10, 2021 at 11:30 am

On Monday, TNTP released a report examining Arkansas’ teacher shortage and providing some suggestions for how to ensure that every student in Arkansas has a high-quality teacher in their classroom. Here at OEP, we agreed with most of their suggestions, and have some of our own!

According to the report:

  • Statewide, 4% of public school teachers are uncertified, with another 3% teaching out of field.
  • Shortages are concentrated in districts in the southern part of the state and in the Delta region.
  • Students of color are more likely to have uncertified teachers.

Recommended solutions are to:

  • Create a supportive pathway to certification for paraprofessionals, long term substitutes, and classroom aides.
  • Raise average teacher salaries.
  • Improve communication about pathways to licensure and related financial incentives.

One recommendation that we felt was missing was removing barriers in the hiring pipeline. In order to apply for a job, teachers have to go to individual district websites and apply. That’s why we created ARteachers.org, a free resource designed to make it easier for teachers to find great jobs, and for school districts to find great teachers. ARteachers.org uses a common application format that is customized for teachers. Teachers can also indicate that they are interesting in long-term substitute opportunities and that they would like to be contacted by districts looking for teachers. Districts can recruit the teachers they are looking for, instead of waiting for them to find their website and apply. In addition, the site will provide the state better information about how many teaching positions are open each year, and how many teachers are looking for jobs. Having this information is vitally important to developing policies that will be effective in ensuring every student in the state has a great teacher. If you know teachers looking for jobs, or districts looking for teachers, please let them know about ARteachers.org.

We think the report’s recommendation to create more opportunities for unlicensed members of education communities without a degree to obtain certification is great. Here at OEP, we suggest that the programs need to be designed with the understanding that these future teachers continue to work in the school while pursuing a degree and licensure. In addition, although the report suggests eligibility for loan forgiveness after the teacher has taught for five years, the up-front costs would likely be a significant barrier. To reduce the financial barrier, we think Arkansas’s colleges and universities should offer scholarships to support these local educators on their path to licensure.

The report’s recommendation to raise the average teacher salary, however, will be expensive and likely ineffective. As we have said before, all districts receive the same per-pupil funding from the state, and local priorities determine how it is spent. Each district sets their own teacher salary schedule. In our research, we find that teacher salaries are mostly driven by student-teacher ratios; teachers with fewer students receive lower salaries. Arkansas has very low student: teacher ratio of 14:1. In our research into teacher supply, we found that a districts’ average salary was not related to the number of applicants. The largest drivers were district size and location, and raising the average salary statewide wouldn’t change that.

We agree with the report’s recommendation that there needs to be improved communication about pathways to licensure and financial incentives. We feel like DESE has been working on communicating pathways to licensure through the Teach Arkansas campaign. We have been talking about the issues with incentives for a while. It is important to research if these dollars are making a difference for Arkansas students, and continue to learn more about how to effectively recruit and retain teachers in our schools.

Effects of School District Consolidation in Arkansas

In The View from the OEP on March 3, 2021 at 12:34 pm

School district consolidation has been one of the most prevalent education reforms over the last century. As a result of consolidation efforts, the number of public school districts in the U.S. declined from 117,108 to 13,551 between 1940 and 2018. Despite the scale of this reform effort, relatively little rigorous research ‎explores the effect of district and school consolidation on student achievement. In this blog post, we summarize a new Arkansas Education Report that investigates the impact of a recent district consolidation law in Arkansans.

The latest round of school consolidation in Arkansas arose in response to school finance litigation that occurred throughout the late 1990s and early 2000s. The decade-long litigation culminated in 2003 with the Arkansas Supreme Court ruling that the state’s school funding system was unconstitutional in Lake View School District vs. Huckabee.

Governor Mike Huckabee responded to the court’s decision in part by proposing large-scale school district consolidation to reduce district administrative costs and provide greater educational opportunity for students. In early 2004, the legislature passed Act 60, which required the consolidation of any district with average daily attendance of fewer than 350 students for two consecutive school years.

The new law resulted in a substantial number of district consolidations in the years that followed. Table 1 presents the number of district consolidations occurring each year beginning with the 2004-05 school year. In the first year the law was in effect, 59 school districts were required to consolidate. Although Act 60 continues to have an impact as enrollments decline in rural districts, only a few districts have been required to consolidate since the initial wave in 2005.

Figure 1 shows the geographic location and district borders of the 99 districts that were either a consolidated or a receiving district in the 2004-05 school year. The map depicts district borders in the year prior to consolidation and districts are color-coded to indicate which districts combined due to Act 60. The initial round of consolidations was relatively widespread across the state, affecting districts in every region. Districts subject to Act 60 enjoy some autonomy in determining which other district to merge with; however, the overwhelming majority have merged with adjoining district.

Studying consolidation is challenging because, in most cases, districts voluntarily choose to consolidate for any number of reasons such as perceived cost benefits or to take advantage of state financial incentives. Districts that choose to consolidate are likely different from those that don’t in many observable and unobservable ways. These differences can lead results from simple comparisons of consolidated districts to unaffected districts to be biased.

Fortunately, Act 60 provides an opportunity to overcome this common challenge. In our analysis, we compare districts whose enrollment was just above or below the enrollment cut-off designated by Act 60. Students in districts with enrollment of less than 350 in the two years immediately prior to the passage of Act 60 are assigned to the treatment group and students in the remaining districts represent the control group. Given that the cutoff was not known in advance, comparing districts right around the consolidation threshold approximates random assignment, yielding two groups of students which should be quite similar except for the treatment group gets consolidated.

We estimate the effect of district consolidation on student performance on the state’s standardized test in math and English Language Arts (ELA). Our results are based on individual student-level data that allow us to follow students across years and school districts. We find small positive effects in both subjects. In math, the average effect is 4 percent of a standard deviation and is only marginally statistically significant. The average effect in ELA is slightly larger, 6 percent of a standard deviation, and is statistically significant. We also investigated how these effects varied over time, finding inconsistent results for mathematics but consistent positive and significant results in ELA. Overall, it appears that consolidation had a positive, albeit small impact on student performance in Arkansas.

While student achievement is important, the primary motivation that policymakers articulated for consolidating smaller school districts in Arkansas was to achieve cost savings through economies of scale. Even if consolidation only had small positive effects on achievement, Act 60 would still be considered a success if consolidation reduced administrative and other spending outside of the classroom, freeing up resources for additional classroom spending or to be redirected toward other important public purposes.

To investigate whether districts affected by consolidation experienced positive economies of scale we compare district-level spending trends before and after consolidation occurred. Table 2 presents a summary of financial information for districts affected by consolidation and Arkansas averages for even numbered school years between 2004 and 2008.

In 2004, prior to consolidation, districts that would be forced to consolidate by Act 60 spent $1,098 more per student, on average, than the state as a whole.  In addition, these districts spent  a lesser share on classroom teachers, and a greater share on other certified staff like administrators than did other school districts in Arkansas (see Columns 1-3 of Table 2). On the surface, these discrepancies support the argument that consolidation had the potential to deliver improvements through greater economies of scale. 

However, when we compare expenditure trends for districts affected by consolidation to unaffected districts, we find little evidence that affected districts meaningfully deviated from broader state trends after consolidation. Columns 4-7 of Table 2 shows that Act 60 affected districts exhibit consistent resource allocation over time to both classroom staff and other certified staff (see last two rows of Table 2). 

While affected districts experienced increased spending per pupil, that trend did not deviate significantly from the overall state trend. State average spending per pupil increased by $1,781 between 2004 and 2008, while spending in consolidation affected districts increased by $1,694.

School district consolidation has been an important and sometime contentious reform in Arkansas. Overall, it appears that the first wave of consolidations under Act 60 may have had small positive effects on achievement but did little to improve the efficiency of the state’s smaller school districts. These findings are relevant today because Act 60 continues to impact Arkansas’s school districts and district leaders and policymakers continue to discuss the value of consolidation/annexation. 

In 2015 the legislature passed Act 377 which allows the State Board of Education (SBE) to grant waivers to the consolidation requirement under Act 60 (see page 11 here). As recently as the fall of 2020, the SBE granted waivers to four districts that were subject to Act 60. In addition, the SBE has used consolidation/annexation as a tool to address consistently poor student performance and/or financial distress. In December, the SBE approved the annexation of the Dollarway School District, which was previously taken over by the state, into the Pine Bluff School District. 

Our research indicates that district consolidation is not likely to result in dramatic improvements in student performance or district efficiency, but small improvements are possible. However, these improvements must be weighed against the legitimate concerns of the communities whose schools are facing consolidation/annexation. If Arkansas is going to continue to use consolidation/annexation as a tool to improve district performance, it is important that we continue to conduct research to determine if these policies are making a positive difference for students and if they make sound fiscal sense.