University of Arkansas Office for Education Policy

Do AP Classes Help Students to be College Ready?

In The View from the OEP on November 18, 2020 at 12:30 pm

The Advanced Placement (AP) program is a nationwide curriculum offering that provides high school students the opportunity to access rigorous, college-level content. Students enrolled in AP may earn college credits for their performance on standardized end-of-course exams.

In recent years AP has come to be seen as a tool to help close achievement gaps in both access to higher education and student outcomes of traditionally underserved students. Since 2008, every Arkansas district has been required to offer AP coursework in the four core disciplines: math, English, science and social studies (“four core”).

Here at OEP we love that Arkansas is supporting AP course availability for all students, but we wondered if increased access to AP was helping our students be ready for college, and if all student groups were benefiting equally.

We use ACT to measure college readiness, as it is used in Arkansas to determine eligibility for a number of scholarship programs and whether students are required to participate in college remediation courses. Our sample includes over 75,000 11th graders from 2016-2018 since 2016 was the first year that Arkansas offered the ACT at no cost to 11th graders during the school day.

You can read the full paper here, and a shorter policy brief here, but we cover the key findings below.

Who Participates in Advanced Placement?

About thirty-four percent of 11th grade students in the study enrolled in at least one core content AP course between 9th and 11th grade, but students who take AP courses are noticeably different relative to their peers who never take AP (Table 1). For example, AP-Takers are more likely to be White and/or qualify for gifted and talented services and have high prior achievement as early as seventh grade. AP-Takers in Arkansas are also substantially more likely to be female. Students who do not enroll in AP are more likely to be Black or Hispanic, English Language Learners, or qualify for free- or reduced-price lunch.

AP-TakersAP Non-Takers
% Female6046
% White7064
% Hispanic911
% Black1722
% Other Races54
% Gifted & Talented215
% English Language Learner27
% Free/Reduced Lunch4159
7th Grade Math Z-score0.68-0.15
7th Grade RLA Z-score0.64-0.15
N24,90151,656
Table 1. Characteristics of AP-Takers and Non-Takers, 11th Graders, 2016-2018

AP participation, overall and by demographic subgroup, varies significantly across different regions of Arkansas. Table 2 shows that Southeast Arkansas has the highest AP participation rate at 39%, while in Northeast Arkansas only 28% of students enroll in AP. On average, there is a 4.6 point difference on the ACT between AP-Takers and Non-Takers, but this too varies by region.

StateNWNECNSWSE
% Taking AP343328353639
Mean ACT (AP-Takers)22.022.722.222.220.619.7
Mean ACT (Non-Takers)17.418.517.116.916.215.7
Table 2. 11th Grade AP Course Taking and Composite Scores, By Region, 2016-2018

AP students do not appear to be representative of the overall population within each region. This suggests access to AP coursework may be subject to selection mechanisms such as peer, parent, and teacher influence or school requirements that provide entry barriers. These findings have implications for policymakers wishing to leverage Advanced Placement coursework to improve student outcomes for students from historically underserved backgrounds.

Do AP Students Score Better on the ACT?

Yes. Students who choose to take at least one AP course between 9th and 11th grade earn higher ACT composite scores relative to their peers. After controlling for demographic characteristics and prior achievement, we find that on average, the difference in ACT score for a student who takes AP is two points higher than a Non-AP Taker (Figure 1). Interestingly, this result is clustered around the 19-point cut score required for students to opt out of remedial math and English courses during their freshman year of college in Arkansas.

There is also evidence that race and socioeconomic status moderates the size of the association between AP course-taking and ACT outcomes (Figure 1). Subgroups from racially diverse or economically disadvantaged backgrounds both score lower on the ACT, on average, and experience smaller increases in ACT composite score when taking AP courses, compared to their economically advantaged, White peers.

Figure 1. Mean ACT Composite Score for AP-Takers and Non-Takers, 2016-2018

It is important to recognize that AP coursework may not be the mechanism that causes students to score higher on the ACT exam. For example, students who select into AP classes may also be more motivated to complete ACT preparation courses, influencing their achievement outcomes.

What Are the Implications?

Students who select into AP score above the remediation threshold while those who do not take AP courses score below the 19 point cutoff, although the overall association between AP course-taking and ACT scores differs for demographic subgroups. The exception is Black students, where even AP-Takers fail to cross the remediation threshold with an average composite score of approximately 18.5, despite the fact that Black AP-Takers score 1.25 points higher than their Black Non-AP peers.

The finding that AP-Takers tend to score above the 19 point remediation cut score and Non-AP Takers generally score below could be a function of two mechanisms:

  1. The difference in ACT composite score for the two groups results from taking an AP course. In this scenario, AP coursework improves student outcomes through some unknown mechanism such as peer effect, teacher quality, or the AP curriculum itself, OR
  2. Students self-select into, or out of, AP based on their self-perceived college readiness or college-going aspirations. The difference in scores, therefore, may simply be capturing the effect of motivation or parental or teacher influence, none of which have been studied by existing research.

This story is interesting to consider, given that Arkansas policies are seeking increased participation in Advanced Placement. It is possible that these policies may end up “pushing” students into AP who are not ready, either academically or socio-emotionally.

Regardless of the mechanism or the consequences, the question remains whether students induced to take AP will experience the hypothesized benefits of the coursework. Existing research is unable to determine whether AP courses cause students to score higher on the ACT, but future research on such topics would be beneficial in understanding how to best prepare students for the rigors of college.

As Advanced Placement programs continue to expand it is important to understand whether the courses actually benefit all students in the way AP advocates claim it will. Our research, albeit on a specific group of Arkansas students, shows that there seems to be a difference in college readiness for students who take AP compared to students who opt out of these more rigorous classes. While we cannot claim that AP classes cause students to score higher on the ACT, it seems there is certainly a tangible benefit for students who engage in these rigorous learning experiences.

OEP is Listening to Parents’ Perspectives

In The View from the OEP on November 11, 2020 at 12:52 pm

OEP is excited to share a new survey for parents and guardians of Arkansas students. Through our Research Practice Partnership, OEP and the Division of Elementary and Secondary Education (DESE) developed a Parent/Guardian Survey for soliciting family input on what is working now for their students and what they are considering for the future.

We encourage you to share the survey link with your families and ask them to complete the survey before November 20th.  A Spanish version of the survey is available here. The survey takes an average of ten minutes to complete. It is important to include the perspectives of parents across the state- so please share this information widely with parents and guardians in your community.

The survey seeks to involve families in an organized, ongoing, and timely way in planning, review, and improvement. The survey will provide feedback regarding:

  *   family concerns
  *   how families are making decisions
  *   family awareness of options and resources available
  *   family considerations for the future

The anonymous survey does not connect responses to individual schools or districts, but rather establishes an understanding of the state as a whole. The information will be aggregated by geographic region only, based on respondent identification of the region of the state in which they live.

OEP will present initial results at the December State Board meeting, and look forward to sharing the valuable perspectives of Arkansas’s parents and guardians. Stay tuned!

30% of Highest Achievers not Identified as Gifted and Talented

In The View from the OEP on November 4, 2020 at 1:17 pm

Did you know that 30% of Arkansas’ highest achieving elementary students are not identified as Gifted and Talented? It’s true. And the bad news is that if a high achieving student is economically disadvantaged, they are 11 percentage points less likely than their more affluent peers to be provided G/T services despite similar academic achievement.

New research out today from OEP examines the likelihood that the highest achieving 3rd grade students are identified G/T by 4th grade. Our sample includes of five cohorts of 3rd graders that scored in the top 5% statewide in both reading and math (N=4,330). This rigorous definition of high achieving identifies students that are the most likely to benefit from G/T services.

The figure below shows the relationship for the students in 4th grade in 2019. The yellow circle represents all 4,067 G/T students, while the blue circle identifies the 1,011 that scores in the top 5% on the 3rd grade state assessments. The area where the circles overlap reflects that 70% of these highest achieving students were identified G/T, but there are 30% of top 5% students that were not. The pattern in consistent across the most recent five years of 3rd to 4th grade cohorts.

Venn Diagram for 2019 4th Grade G/T Students and Top 5% Students on 2018 3rd Grade Reading and Mathematics Assessments

How are students identified as G/T?

In Arkansas, students are identified as G/T at the school district level. While the process varies by district, it typically begins with a nomination from a teacher, counselor, parent, or peer. Arkansas law requires G/T identification include two objective and two subjective measures, with at least one being a measure of creativity. Districts select their own assessments and process for identification. Unlike SPED or ELL identification that is sustained when students transfer districts, the G/T label may or may not still apply.

Are there differences by student demographics?

Examining trends in G/T identification by student demographics reveals that students from different populations are more or less likely to score in the top 5% on the 3rd grade state assessment, as well as differences by student demographic characteristics in the percentage of those high achievers being identified as G/T. The summary descriptives for all five cohorts examined is presented in the table below. You can see that 12% of our sample was identified G/T, while only 2.5% of the sample scored in the top 5% in both reading and mathematics on 3rd grade state assessments. Of those highest achieving students, 70% were identified G/T by 4th grade. When we further examine our sample, we see that although 65% of students participate in the federal Free/Reduced Lunch program (FRL), only 8% of them are identified G/T. Just over 1% of FRL students scored in the top 5% on 3rd grade assessments, but only 64% of those highest achieving FRL students were identified G/T. We also see evidence of lower rates of G/T identification for high achieving students that are Hispanic (67%) or receive Special Education (SPED) services (60%).

Summary descriptive statistics by G/T and Top 5% Achievers, 3rd to 4th grade cohorts

These descriptive summaries give us a sense that there may be certain types of students that are less likely to be identified as G/T, even though they are high achieving. Because many of the variables of interest are interrelated, we ran a multivariate regression including the listed student demographic characteristics. We also included district characteristics (district size, %FRL, urbanicity, and geographic region) in our model because identification occurs at the district level and we wondered if specific district types were related to the likelihood of high achieving students being identified as G/T.

You can read the policy brief or full paper if you want more details, but we find that after controlling for student and district characteristics, high achieving FRL students are 11 percentage points less likely to be identified as G/T. We found no significant differences in G/T identification rates of high achievers by student gender or race- which is great! While there was some significant variation in the likelihood of G/T identification by geographic region and district size, the main district findings were that high achieving students from lower poverty districts (<52% FRL) were 8 percentage points less likely to be identified as G/T and that students in larger districts (>2,500 students) were much more likely to be identified for G/T services.

Why are students being missed?

Perhaps these high achieving students were tested for G/T and failed to meet the district criteria, but we don’t have the data to determine that since it is not collected by the state. Another option is that these high achieving students were never nominated for G/T testing. Some students, particularly those from low-income households, may be less likely to have a parent that is comfortable with or informed about the process for nominating a student for G/T consideration. This is one reason that we suggest using state standardized tests as universal screeners could be a move toward greater equity in G/T identification.

There was wide variation at the district level the percentage of students in the top 5% of achievers on 3rd grade assessments that were identified as G/T. When we limit our sample to districts that had at least ten 3rd grade students in the top 5% over the five years examined, the G/T identification rates for these high achievers ranges from 0 to 100%. When we similarly examine high achieving FRL students, the district-level G/T identification rates range from 22 to 78%.

As it might be helpful for districts to examine their G/T identification rates of the highest achieving 3rd graders, district identification rates are available to district Superintendents and G/T coordinators upon request to oep@uark.edu.

Here at OEP, we value the services that G/T programs provide for students, and are not proposing that state standardized test scores should be the sole consideration in G/T identification. Rather, we suggest that examining these universally administrated state assessments could be a time and cost effective way for districts to find students that may not have been considered for G/T, but would likely benefit from receiving services.