University of Arkansas Office for Education Policy

Archive for 2021|Yearly archive page

New Research on Arkansas Challenge Scholarships

In The View from the OEP on February 17, 2021 at 12:30 pm

The Arkansas Academic Challenge Scholarship (ACS) has awarded over 600,000 college scholarships to Arkansas students. ACS is a state-financed merit-aid program with relatively low eligibility requirements, and there is no expectation of repayment. Governor Asa Hutchinson has expressed interest in enhancing ACS funding for students demonstrating a financial need, and results from new research out today highlights the fact that the timing of receiving money may heavily influence student behavior and postsecondary outcomes. Researchers find that receiving ACS funds initially while already in college resulted in small, negative impacts on short-run outcomes such as GPA and credit accumulation, but large statistically significant declines in the likelihood of graduating within four, five, or six years of entering college.

While a version of the ACS dates back to the 1990s, legislation passed in 2008 dramatically expanded the program by tying funding to the Arkansas Scholarship Lottery. Expansion of the Academic Challenge Scholarship allowed some students who were already enrolled in college to become eligible for the scholarship. In the current study, researchers examined how receipt of the scholarship impacted students’ college GPA, credit accumulation and likelihood of graduation. For more details about the research, you can read the full paper, or the shorter policy brief, but we wanted to share the highlights:

  • Overall, students who received the scholarship funds while already enrolled in college earned lower GPAs, accumulated fewer credits, and were over 40 percentage points less likely to graduate in four, five, or six years relative to their peers who did not receive the scholarship.
  • Compared to their peers, students who began receiving funding during their sophomore year of college enrollment:
    • earn lower GPAs and accumulate a staggering 18 fewer credits within the first year of receiving their scholarship
    • earn GPAs, on average, 0.75 points lower, and accumulate 24 fewer credits two years after receiving funding,
    • were 53-62 percentage points less likely to graduate in four, five, or six years.
  • Compared to their peers, students who began receiving funding during their junior year of college enrollment:
    • appear to have few significant changes in their GPA, or credit accumulation after one or two years,
    • experienced no statistically significant change in their likelihood of graduating within four, five, or six years.
  • Compared to their peers, students who began receiving funding during their senior year of college enrollment:
    • experienced small declines in their credit accumulation and GPA,
    • were 54 percentage points more likely to graduate within six years than students who did not receive funding.

It may seem counter-intuitive that receiving scholarship money would have a negative effect, but the research on college merit aid have found mixed effects of such programs on student outcomes. In the only study of randomly assigned aid offers, Angrist and colleagues (2016), find that being assigned to receive merit-aid increases both the probability of enrolling and persisting in college and demonstrates that students with relatively low academic achievement and those who enrolled in less-selective four-year institutions generated the largest gains in both outcomes. However, this same study also indicates that students appear to delay graduation to a fifth year in order to maximize scholarship funding if the program is renewable beyond four years.

We need to continue to research how the ACS funds effect outcomes for Arkansas students, including longer-term workforce outcomes. If the timing of the money matters, awarding the scholarship funds in the most effective way will lead to better outcomes for our students and our state.

ABC Pre-K Shows Benefits for ELLs

In The View from the OEP on January 27, 2021 at 11:50 am

Last week, OEP reported on the positive student outcomes associated with Arkansas Better Chance (ABC) pre-K attendance. Today, we add further insight to that report’s findings by offering evidence that the ABC program may be especially beneficial for students who enter school with limited English proficiency. These students, often called English Language Learners (ELL), face a unique set of challenges as they navigate K-12 education with the added task of mastering a second language. As we serve an increasing number of ELLs in Arkansas public schools, we must work to determine how to help these students achieve English proficiency as early as possible to mitigate the challenges associated with the language barrier. 

Arkansas’s state-funded ABC program provides tuition-free pre-K opportunities for qualifying families, and it is considered a high-quality program, as ABC teachers must have bachelor’s degrees and AR teacher licenses. Research that examines public pre-K programs in other states demonstrates that ELL students experience especially pronounced benefits from such programs (Gormley, 2008; Lipsey, 2018). We examine whether that pattern holds true for ELL students in Arkansas. In this analysis, we follow three cohorts of Arkansas students (those who began Kindergarten in the fall of 2011, 2012, and 2013) through 5th grade to see the long-term outcomes of ABC participants who enter school with an ELL designation. Our outcomes of interest are math and reading achievement, as demonstrated by statewide assessments, as well as the likelihood of students exiting the ELL program by 5th grade. Our research design does not allow us to make causal claims of the program’s impact, but we hope that these findings will inform us of the potential for ABC programs to serve the ELL student population. You can read the policy brief or the full paper for more details, but here are the takeaways:

•          Approximately 10% of ABC pre-K enrollment is comprised of ELL students, making this student group over-enrolled relative to its 8% of K-12 enrollment. This is a positive and unsurprising pattern, given that ABC targets its enrollment toward high-risk student populations. 

•          We find positive, statistically significant relationships between ABC attendance and math and reading achievement across 3rd to 5th grades in all three cohorts of analysis. Gains are large and meaningful, although they are too limited in most cases to close achievement gaps between ELL and non-ELL students. 

•          Evidence from our subgroup analysis shows that ELL students who attend ABC pre-K experience a differentially positive benefit in two of three cohorts analyzed. This means that ABC attendance is associated with greater achievement benefits for ELL students over and above the program benefit for English-proficient students. 

•          ELL students who attend ABC pre-K are less likely to remain identified as ELL in 3rd to 5th grades. The process for exiting the ELL program is complex and involves demonstrating English proficiency on a variety of measures. ELL students who attended ABC were 9 percentage points less likely to be designated ELL by 5th grade, compared to those who did not attend ABC prior to starting Kindergarten, in the second cohort of analysis. 

While ELL students are currently over-enrolled in ABC programs relative to their K-12 enrollment, it is clear that they experience especially positive benefits from ABC, over and above the benefits for English-proficient students. Based on the findings from this analysis, policymakers and community leaders should consider efforts to further expand ELL enrollment in ABC pre-K programs across the state. The current strategy for program advertisement is through communication with families in schools, churches, and other community spaces. Additional efforts could include outreach in organizations that include many linguistic-minority community members. Also, program organizers in school districts across the state should ensure that applications and program materials are available in all non-English languages represented in their communities, including Spanish and Marshallese. Education leaders should also brainstorm ways to provide sustained support to this student subgroup, as ABC pre-K participation alone does not seem to be sufficient for closing the gap between ELL/non-ELL achievement. While we would love to employ a causal research design in the future, the patterns presented here are encouraging and suggest that ABC pre-K is a promising early intervention for this vulnerable student group. 

ABC Pre-K Students Outperform Peers

In The View from the OEP on January 20, 2021 at 11:30 am

New research out of OEP examines the relationship between students who attend Arkansas’ ABC pre-K programs and later academic outcomes. We find positive, statistically significant relationships between ABC participation and 3rd grade math and reading achievement in three of our four cohorts, but also that the relationship diminishes by the time students are in 5th grade.

The Arkansas Better Chance (ABC) program was launched in 1991, and legislation passed in 2003 outlined specific guidelines and requirements for pre-K programs serving students with ABC funding. ABC educators are required to have bachelor’s degrees and current AR teacher licenses. These are considered rigorous standards compared to many public and private pre-K centers. The ABC program meets eight of ten quality standards set by the National Institute for Early Education Research.

ABC programs primarily serve students at risk for low academic performance. Families with combined household incomes less than or equal to 200% of the federal poverty level are eligible for free tuition and priority enrollment. Students can also qualify for eligibility if they have other risk factors, including disabilities, developmental delays, or limited English proficiency. When there are additional spots in ABC classrooms unfilled by qualifying students, other children in the community can enroll and pay tuition on an income-based sliding scale. 

The new study from OEP describes the 3rd and 5th grade outcomes of students who enroll in ABC pre-K programs in Arkansas public schools. Our research isn’t causal, because students are not randomly assigned to ABC participation. In an attempt to understand how well these programs are serving students, we follow four cohorts of program participants through elementary school, and we compare their math and reading achievement test scores to those of similar peers who did not attend ABC programs.

You can read the full paper or the policy brief for more details, but here are the highlights:

  • Approximately 25% of each Kindergarten class for each of the four analytic samples attended an ABC pre-K program
  • ABC pre-K participants are more likely to fall into demographic groups that are considered at-risk for low academic performance: approximately 75% of ABC participants qualify for Free or Reduced Price lunch when they start Kindergarten
  • ABC students outperform similar peers on math and reading achievement tests in 3rd grade in three of four cohorts. After controlling for student-level demographic characteristics, there are positive, statistically significant relationships between ABC participation and 3rd grade math and reading achievement for three out of four cohorts of students
  • Relationships between ABC participation and 5th grade achievement are smaller in magnitude than those of 3rd grade and largely not statistically significant. The lack of significant findings in 5th grade is largely the result of declines in average z-scores for ABC participants since 3rd grade.

As Arkansas strives to improve academic outcomes for students, particularly the percentage of students reading on grade level by 3rd grade, positive results associated with ABC pre-K participation should be of particular interest to policymakers and advocates. The lack of significant findings in 5th grade math is largely the result of declines in average z-scores for ABC participants since 3rd grade. Our findings suggest that high-quality pre-K education may be a critical tool for ensuring all students have the opportunity to succeed. While ABC participants score demonstrably higher than similar peers on standardized tests in early elementary school, there are likely even more program benefits for these students. Prior pre-K research demonstrates that these programs can lead to positive social and behavioral outcomes in addition to academic and cognitive benefits (Gorey, 2001). Getting an additional year or two to learn classroom rules and procedures, socialize with peers, and adapt to the school routine should equip students with behavioral and social skills that will help them succeed in Kindergarten and beyond. Arkansas educators and policymakers have an opportunity to ensure that the benefits of high-quality pre-K endure into higher grade levels, and that all students are consistently provided the opportunity to learn and achieve their goals.

Parents/Guardians Share Opinions

In The View from the OEP on January 13, 2021 at 12:45 pm

In November, OEP partnered with DESE’s Office for Family Engagement to administer a survey to parents and guardians of K-12 public school students across the state. Our goal was to learn from the families of Arkansas students about what is working currently as well as future considerations. The report goes to the State Board this week, and we wanted to share the results with you too! Our key takeaways are that most students were attending schools in person (69%), most parents feel like their student is learning about the same or more than normal (62%), and most parents rated their child’s school as doing an “excellent” or “good” job on on the quality of teaching and instruction (72%), and on handling health and safety measures to prevent the spread of COVID-19 (71%).

Parents were informed about the survey through social media channels, and superintendents and principals were asked to share the link with the parents and guardians of students in their schools and 17,836 parents/guardians representing 30,381 individual students responded to the survey. We estimate this to be about 6% of the parents/guardians of K-12 students in the state. This is a sample of convenience and the results may not be generalizable to all parents in the state. Parents/guardians of white students were overrepresented in the sample by 15 percentage points compared to statewide student demographics, and the 42% of parents/guardians that reported their student was eligible for the Federal Free/Reduced Lunch Program (FRL) were underrepresented by 24 percentage points compared to statewide student demographics.

In the first part of the survey, parents/guardians responded about what schools should be focused on and their worries regarding COVID-19.

Survey respondents were asked to select which statement they agreed with more:

  • Schools should be focused on rethinking how we educate students, coming up with new ways to teach children moving forward as a result of the COVID-19 crisis
  • Schools should be focused on trying to get back to the way things were before the COVID-19 crisis as soon as it is safe to do so

We found it interesting that the parents/guardians were nearly evenly split in their opinions. 51% of parents statewide selected “rethinking how we educate students… as a result of the COVID-19 crisis” and 47% of parents selected “get back to the way things were before COVID-19.” Two percent of survey respondents did not answer the question. These results are more evenly split than the national sample, in which 66% of parents supported re-thinking education. When we compare the results of parents that indicated that their student was eligible for the Federal Free/Reduced Lunch Program (FRL) with those who indicated that their student was not eligible to participate in the program, we see differences along these socio-economic lines. Parents of FRL-Eligible students are 10 percentage points more likely to believe that schools should be focused on rethinking how we educate students, coming up with new ways to teach children moving forward as a result of the COVID-19 crisis.

Figure 1. Percentage of responses to the question regarding what schools should be focused on, by reported FRL-Eligibility

Overall, parents were most concerned about their child or children staying on track in school, that their child might miss important social interactions, or that someone in their family would get the virus. These were also the top 3 concerns in the national sample. Respondents who indicated that their student was eligible for the Federal Free/Reduced Lunch Program (FRL) reported higher levels of worry than the full sample in all areas except missing social interactions. Figure 1 displays the comparison between the full sample and FRL-eligible parents/guardians who responded that they “worry a lot” or “worry some” to each area.

Figure 2. Percentage of full sample and FRL-eligible parent/guardian responses of “worry a lot” and “worry some” to the question “With regard to the current coronavirus situation, how much do you worry about each of the following as a parent or guardian?”.

In the second part of the survey, parents/guardians responded individually about each child in public school regarding current instructional setting, how much the student was learning compared to the prior year, and perceptions of the student’s school.

Current instructional setting:

The vast majority of parents indicated their student was attending school in person only on their school campus (69%), which is much higher than the national sample, in which only 19% of parents reported that their student was attending school in person. Twenty percent of Arkansas students were reported to be attending online or remotely only, and 10% were reported to be attending part-time in person and part-time remotely. One percent of students were doing “something different” or didn’t respond. These percentages were similar to what DESE reported at the time of the survey: 64% in person, 22% virtual, and 14% part-time in person and part-time remotely.

Reasons for instructional setting selection:

Among parents that selected in-person learning on the school campus, the most popular reasons were worries that their child would not learn as much any other way (74%) and that their child would miss social interactions (52%).  Among parents that selected remote instruction, the main reasons were to reduce the risk of the child getting Covid-19 (84%), as well as health and medical concerns for students, their families, or the community.

Learning amount:

Parents were also asked to compare how much their student was learning this year compared to normal. The majority of parents (62%) felt that their student was learning more (8%) or about the same (54%) as normal, however, there was variation by instructional setting. Parents of remote only or part-time remote students more likely to report that their student was learning less than normal. Among in-person parents, 25% reported that their student was learning less than normal, compared to 38% of remote only parents and 47% of parents whose students were attending school part-time in person and part-time online.

Opinions on daily schedule:

Overall, parents reported that their student was getting about the right amount of time receiving instruction from their teacher(s) (72%), time to communicate directly with their teacher(s), and to ask questions, or get help with assignments (71%). A smaller percentage of parents reported that their student was getting about the right amount of time to interact and communicate with other students (64%). Again there was variation by instructional setting, with parents part-time remote students less likely than parents of in-person students to report that their student was getting “about the right” amount of time receiving instruction from their teacher(s), time to communicate directly with their teacher(s), and to ask questions, or get help with assignments, and to interact and communicate with other students. Parents of students who were only remote were the least likely to report students were getting “about the right” amount of time in all three areas, particularly relative to interacting with other students.

Opinions of school performance:

Survey respondents were asked to rate how well their child(ren)’s school was doing on a variety of measures. Over 70% of parents rated their child’s school as “excellent” or “good” on the quality of teaching and instruction (72%), and on handling health and safety measures to prevent the spread of COVID-19 (71%). Over 60% of parents rated their child’s school as doing and “excellent” or “good” job on assessing their child’s progress and level of learning (68%), communicating with parents (66%) and providing additional resources and support to help their child continue learning (62%). Schools received lower ratings in two areas: providing additional resources to support learning and to support students’ mental health and emotional wellbeing. Some areas received lower levels of agreement, Only 53% of parents felt that their child’s school was doing an “excellent” or “good” job managing online learning programs, and only 43% reported that their child’s school was doing an “excellent” or “good” job support student’s mental health and emotional wellbeing. In large part these lower ratings, however, were due to an increased percentage of respondents who indicated that they did not know how the school was performing in these areas.

Figure 3. Percentage of parent/guardian responses to the question: “How would you currently rate how your child’s school is doing on each of the following?” Full Sample

A challenge in interpreting the survey results is that we do not have the same information from parents before COVID, so we can’t determine if parents’ feelings about their schools have improved or declined as a result of COVID- related changes. Although we examine the survey results by geographic region in the full report, parents and guardians did not identify their school or district, so it is impossible to determine if these results reflect certain school systems or a more general perspective across multiple districts. In addition, certain populations are over- or underrepresented in the responses and parent perspectives may have shifted with the rising rates of COVID. Given these limitations, however, this is valuable information and the large sample indicates that parents want to share their opinions. Here at OEP, we recommend the state consider implementing an annual parent survey so that the opinions of these important stakeholders can continue to be heard and included in the discussion about the future of Arkansas’ public schools. You can find more details about the sample and the results in the full report.