University of Arkansas Office for Education Policy

Author Archive

Early Research on Learning During COVID-19

In The View from the OEP on December 2, 2020 at 12:00 pm

We’ve all been wondering- how did COVID-19 disruptions impact students’ academic learning?

Yesterday, NWEA’s Collaborative for Student Growth released new research that found some good news! You can read more in the full brief here, but the takeaway is that there were not consistent declines in student achievement over the spring and summer.

Using data from nearly 4.4 million students in grades 3-8 who took MAP® Growth™ assessments in fall 2020, the researchers examined three primary research questions:

  • Are students performing at lower levels this fall compared to last fall?
    • Reading held steady: In fall of 2020, students in grades 3-8 performed similarly in reading to same-grade students in fall 2019.
    • Math falls behind: In fall of 2020, students in grades 3-8 performed about 5 to 10 percentile points lower in math compared to same-grade students in fall 2019.
  • Did students demonstrate lower academic growth than typical since schools closed in March?
    • Students still demonstrated academic growth during COVID: In almost all grades, most students made some learning gains in both reading and math since the COVID-19 pandemic started.
    • Growth in reading scores was consistent with typical learning projections.
    • Gains in math were lower on average than in prior years, resulting in more students falling behind relative to their prior standing.
  • Were the early predictions of a COVID slide accurate?
    • NWEA’s earlier projections of a COVID slide were lower than actual performance for reading, but pretty spot on for math.


Although the news about student achievement and growth is better than we had feared, the researchers caution that many students that would typically take the MAP assessment in the fall are not appearing in the data. Student groups especially vulnerable to the impacts of the pandemic were more likely to be missing. Without the information from these students, the understanding of how achievement this fall may differ across student groups is incomplete and the research may be underestimating the impacts of COVID-19. In addition, the research only includes students in grades 3-8, so the relative achievement of students in K-2 and high school is still unknown.

The research highlights a critical need for clear data to understand where students have fallen behind and to guide where additional resources and supports should be deployed to get them back on track. Here in Arkansas, many students complete an assessment in the fall, be it MAP or something else. We should collectively examine data from this fall to determine if the trends observed nationally are reflective for Arkansas’ students.

The reported declines in math achievement are particularly concerning for Arkansas students, as only one in three of our 4th graders scored at grade level on the most recent national assessment in mathematics. We should take this opportunity to think creatively about how students are organized for instruction in mathematics, and providing differentiated support for each student.

Arkansas leaders are continuing to support our schools, proposing in the FY22 budget the largest increase in education in more than a decade. It is our responsibility to ensure that the resources are being used effectively to support Arkansas students.

If you want to be a part of a collective (anonymous) analysis of Arkansas’ data, or if you want support interpreting your data- just reach out to us at oep@uark.edu.

Do AP Classes Help Students to be College Ready?

In The View from the OEP on November 18, 2020 at 12:30 pm

The Advanced Placement (AP) program is a nationwide curriculum offering that provides high school students the opportunity to access rigorous, college-level content. Students enrolled in AP may earn college credits for their performance on standardized end-of-course exams.

In recent years AP has come to be seen as a tool to help close achievement gaps in both access to higher education and student outcomes of traditionally underserved students. Since 2008, every Arkansas district has been required to offer AP coursework in the four core disciplines: math, English, science and social studies (“four core”).

Here at OEP we love that Arkansas is supporting AP course availability for all students, but we wondered if increased access to AP was helping our students be ready for college, and if all student groups were benefiting equally.

We use ACT to measure college readiness, as it is used in Arkansas to determine eligibility for a number of scholarship programs and whether students are required to participate in college remediation courses. Our sample includes over 75,000 11th graders from 2016-2018 since 2016 was the first year that Arkansas offered the ACT at no cost to 11th graders during the school day.

You can read the full paper here, and a shorter policy brief here, but we cover the key findings below.

Who Participates in Advanced Placement?

About thirty-four percent of 11th grade students in the study enrolled in at least one core content AP course between 9th and 11th grade, but students who take AP courses are noticeably different relative to their peers who never take AP (Table 1). For example, AP-Takers are more likely to be White and/or qualify for gifted and talented services and have high prior achievement as early as seventh grade. AP-Takers in Arkansas are also substantially more likely to be female. Students who do not enroll in AP are more likely to be Black or Hispanic, English Language Learners, or qualify for free- or reduced-price lunch.

AP-TakersAP Non-Takers
% Female6046
% White7064
% Hispanic911
% Black1722
% Other Races54
% Gifted & Talented215
% English Language Learner27
% Free/Reduced Lunch4159
7th Grade Math Z-score0.68-0.15
7th Grade RLA Z-score0.64-0.15
N24,90151,656
Table 1. Characteristics of AP-Takers and Non-Takers, 11th Graders, 2016-2018

AP participation, overall and by demographic subgroup, varies significantly across different regions of Arkansas. Table 2 shows that Southeast Arkansas has the highest AP participation rate at 39%, while in Northeast Arkansas only 28% of students enroll in AP. On average, there is a 4.6 point difference on the ACT between AP-Takers and Non-Takers, but this too varies by region.

StateNWNECNSWSE
% Taking AP343328353639
Mean ACT (AP-Takers)22.022.722.222.220.619.7
Mean ACT (Non-Takers)17.418.517.116.916.215.7
Table 2. 11th Grade AP Course Taking and Composite Scores, By Region, 2016-2018

AP students do not appear to be representative of the overall population within each region. This suggests access to AP coursework may be subject to selection mechanisms such as peer, parent, and teacher influence or school requirements that provide entry barriers. These findings have implications for policymakers wishing to leverage Advanced Placement coursework to improve student outcomes for students from historically underserved backgrounds.

Do AP Students Score Better on the ACT?

Yes. Students who choose to take at least one AP course between 9th and 11th grade earn higher ACT composite scores relative to their peers. After controlling for demographic characteristics and prior achievement, we find that on average, the difference in ACT score for a student who takes AP is two points higher than a Non-AP Taker (Figure 1). Interestingly, this result is clustered around the 19-point cut score required for students to opt out of remedial math and English courses during their freshman year of college in Arkansas.

There is also evidence that race and socioeconomic status moderates the size of the association between AP course-taking and ACT outcomes (Figure 1). Subgroups from racially diverse or economically disadvantaged backgrounds both score lower on the ACT, on average, and experience smaller increases in ACT composite score when taking AP courses, compared to their economically advantaged, White peers.

Figure 1. Mean ACT Composite Score for AP-Takers and Non-Takers, 2016-2018

It is important to recognize that AP coursework may not be the mechanism that causes students to score higher on the ACT exam. For example, students who select into AP classes may also be more motivated to complete ACT preparation courses, influencing their achievement outcomes.

What Are the Implications?

Students who select into AP score above the remediation threshold while those who do not take AP courses score below the 19 point cutoff, although the overall association between AP course-taking and ACT scores differs for demographic subgroups. The exception is Black students, where even AP-Takers fail to cross the remediation threshold with an average composite score of approximately 18.5, despite the fact that Black AP-Takers score 1.25 points higher than their Black Non-AP peers.

The finding that AP-Takers tend to score above the 19 point remediation cut score and Non-AP Takers generally score below could be a function of two mechanisms:

  1. The difference in ACT composite score for the two groups results from taking an AP course. In this scenario, AP coursework improves student outcomes through some unknown mechanism such as peer effect, teacher quality, or the AP curriculum itself, OR
  2. Students self-select into, or out of, AP based on their self-perceived college readiness or college-going aspirations. The difference in scores, therefore, may simply be capturing the effect of motivation or parental or teacher influence, none of which have been studied by existing research.

This story is interesting to consider, given that Arkansas policies are seeking increased participation in Advanced Placement. It is possible that these policies may end up “pushing” students into AP who are not ready, either academically or socio-emotionally.

Regardless of the mechanism or the consequences, the question remains whether students induced to take AP will experience the hypothesized benefits of the coursework. Existing research is unable to determine whether AP courses cause students to score higher on the ACT, but future research on such topics would be beneficial in understanding how to best prepare students for the rigors of college.

As Advanced Placement programs continue to expand it is important to understand whether the courses actually benefit all students in the way AP advocates claim it will. Our research, albeit on a specific group of Arkansas students, shows that there seems to be a difference in college readiness for students who take AP compared to students who opt out of these more rigorous classes. While we cannot claim that AP classes cause students to score higher on the ACT, it seems there is certainly a tangible benefit for students who engage in these rigorous learning experiences.

OEP is Listening to Parents’ Perspectives

In The View from the OEP on November 11, 2020 at 12:52 pm

OEP is excited to share a new survey for parents and guardians of Arkansas students. Through our Research Practice Partnership, OEP and the Division of Elementary and Secondary Education (DESE) developed a Parent/Guardian Survey for soliciting family input on what is working now for their students and what they are considering for the future.

We encourage you to share the survey link with your families and ask them to complete the survey before November 20th.  A Spanish version of the survey is available here. The survey takes an average of ten minutes to complete. It is important to include the perspectives of parents across the state- so please share this information widely with parents and guardians in your community.

The survey seeks to involve families in an organized, ongoing, and timely way in planning, review, and improvement. The survey will provide feedback regarding:

  *   family concerns
  *   how families are making decisions
  *   family awareness of options and resources available
  *   family considerations for the future

The anonymous survey does not connect responses to individual schools or districts, but rather establishes an understanding of the state as a whole. The information will be aggregated by geographic region only, based on respondent identification of the region of the state in which they live.

OEP will present initial results at the December State Board meeting, and look forward to sharing the valuable perspectives of Arkansas’s parents and guardians. Stay tuned!

30% of Highest Achievers not Identified as Gifted and Talented

In The View from the OEP on November 4, 2020 at 1:17 pm

Did you know that 30% of Arkansas’ highest achieving elementary students are not identified as Gifted and Talented? It’s true. And the bad news is that if a high achieving student is economically disadvantaged, they are 11 percentage points less likely than their more affluent peers to be provided G/T services despite similar academic achievement.

New research out today from OEP examines the likelihood that the highest achieving 3rd grade students are identified G/T by 4th grade. Our sample includes of five cohorts of 3rd graders that scored in the top 5% statewide in both reading and math (N=4,330). This rigorous definition of high achieving identifies students that are the most likely to benefit from G/T services.

The figure below shows the relationship for the students in 4th grade in 2019. The yellow circle represents all 4,067 G/T students, while the blue circle identifies the 1,011 that scores in the top 5% on the 3rd grade state assessments. The area where the circles overlap reflects that 70% of these highest achieving students were identified G/T, but there are 30% of top 5% students that were not. The pattern in consistent across the most recent five years of 3rd to 4th grade cohorts.

Venn Diagram for 2019 4th Grade G/T Students and Top 5% Students on 2018 3rd Grade Reading and Mathematics Assessments

How are students identified as G/T?

In Arkansas, students are identified as G/T at the school district level. While the process varies by district, it typically begins with a nomination from a teacher, counselor, parent, or peer. Arkansas law requires G/T identification include two objective and two subjective measures, with at least one being a measure of creativity. Districts select their own assessments and process for identification. Unlike SPED or ELL identification that is sustained when students transfer districts, the G/T label may or may not still apply.

Are there differences by student demographics?

Examining trends in G/T identification by student demographics reveals that students from different populations are more or less likely to score in the top 5% on the 3rd grade state assessment, as well as differences by student demographic characteristics in the percentage of those high achievers being identified as G/T. The summary descriptives for all five cohorts examined is presented in the table below. You can see that 12% of our sample was identified G/T, while only 2.5% of the sample scored in the top 5% in both reading and mathematics on 3rd grade state assessments. Of those highest achieving students, 70% were identified G/T by 4th grade. When we further examine our sample, we see that although 65% of students participate in the federal Free/Reduced Lunch program (FRL), only 8% of them are identified G/T. Just over 1% of FRL students scored in the top 5% on 3rd grade assessments, but only 64% of those highest achieving FRL students were identified G/T. We also see evidence of lower rates of G/T identification for high achieving students that are Hispanic (67%) or receive Special Education (SPED) services (60%).

Summary descriptive statistics by G/T and Top 5% Achievers, 3rd to 4th grade cohorts

These descriptive summaries give us a sense that there may be certain types of students that are less likely to be identified as G/T, even though they are high achieving. Because many of the variables of interest are interrelated, we ran a multivariate regression including the listed student demographic characteristics. We also included district characteristics (district size, %FRL, urbanicity, and geographic region) in our model because identification occurs at the district level and we wondered if specific district types were related to the likelihood of high achieving students being identified as G/T.

You can read the policy brief or full paper if you want more details, but we find that after controlling for student and district characteristics, high achieving FRL students are 11 percentage points less likely to be identified as G/T. We found no significant differences in G/T identification rates of high achievers by student gender or race- which is great! While there was some significant variation in the likelihood of G/T identification by geographic region and district size, the main district findings were that high achieving students from lower poverty districts (<52% FRL) were 8 percentage points less likely to be identified as G/T and that students in larger districts (>2,500 students) were much more likely to be identified for G/T services.

Why are students being missed?

Perhaps these high achieving students were tested for G/T and failed to meet the district criteria, but we don’t have the data to determine that since it is not collected by the state. Another option is that these high achieving students were never nominated for G/T testing. Some students, particularly those from low-income households, may be less likely to have a parent that is comfortable with or informed about the process for nominating a student for G/T consideration. This is one reason that we suggest using state standardized tests as universal screeners could be a move toward greater equity in G/T identification.

There was wide variation at the district level the percentage of students in the top 5% of achievers on 3rd grade assessments that were identified as G/T. When we limit our sample to districts that had at least ten 3rd grade students in the top 5% over the five years examined, the G/T identification rates for these high achievers ranges from 0 to 100%. When we similarly examine high achieving FRL students, the district-level G/T identification rates range from 22 to 78%.

As it might be helpful for districts to examine their G/T identification rates of the highest achieving 3rd graders, district identification rates are available to district Superintendents and G/T coordinators upon request to oep@uark.edu.

Here at OEP, we value the services that G/T programs provide for students, and are not proposing that state standardized test scores should be the sole consideration in G/T identification. Rather, we suggest that examining these universally administrated state assessments could be a time and cost effective way for districts to find students that may not have been considered for G/T, but would likely benefit from receiving services.

Got Teachers?

In The View from the OEP on April 29, 2020 at 12:12 pm

AR Teachers Logo_horiz PNG

We don’t know yet what school will look like in the fall, but we do know that Arkansas students will need great teachers.  We may see more teachers choose not to return to the classroom due to concerns about exposure to COVID-19 or the challenges of online instruction, and schools will need to be prepared to support students’ social emotional and academic needs more than ever.

NOW is the time to start recruiting great teachers and our new system ARteachers.org can help!  Research from TNTP shows that early, springtime hiring is critical, and that urban districts can lose up to 60 percent of their applicants by not extending job offers until mid- to late summer. Traditional recruitment methods aren’t available as job fairs and other events are cancelled, in-person interviews aren’t possible, and staffing teams are adjusting to remote working. Districts need to quickly shift to virtual recruitment to avoid a shortage of teachers at a time when students can least afford one.

ARteachers.org is a free site we developed where any public school district in Arkansas can post a job.  The centralized job posting/ teacher application site is enhanced as teachers complete a ‘common application’ and can apply for jobs with just one click.  Teachers are matched to jobs that meet their licensure, and they can select to make their ‘common application’  with all districts.  This provides districts looking for teachers a pool of applicants to recruit from instead of waiting for teachers to apply. Districts can search this teacher pool by licensure type or specific keywords to find the teachers that they need.

We know you have a process for hiring, but adding ARteachers.org is easy and free.  It gives you access to teachers who might not know about your district or who might not know you are looking for teachers like them!


Tips for posting jobs on ARteachers.org

Posting is easy.  All you need is to get someone approved to post for the district. That person goes to ARteachers.org and signs up.  Then, the superintendent of record for the school district gets an email to approve that person to post the jobs for the district. Once approved, the poster can login, click on the button labeled “Start new job posting”, and fill out the required information. Once completed, click “Submit new job” and the post is ready for teachers to apply!

Make your job inviting and interesting! Teachers are going to be more interesting in a posting that shares some information about what makes your school district unique and a great place to work.  Compare the job descriptions below to see how context can make a job posting reflect the passion your district has for teachers.

Posting 1: We are looking for an outstanding biology teacher who is passionate about developing students’ love for learning. We are a small district in a tight knit community and we love our teachers! Benefits include: Discounts from local businesses (including massages and yoga) and free gym membership!

Posting 2: Opening for a biology teacher for the 2020-21 school year. Candidate must hold a valid Arkansas Teacher License.

Check out what the teacher experience is. This short video explains how teachers sign up.  Connecting Arkansas teachers and districts is our goal!

Don’t just link to your HR application.  Teachers want to get the benefit of the ‘common app’.  When you require them go to your website to apply, they have to enter all the same information in again!  Use ARteachers.org as an initial screener, and request the candidates that you are interested in to apply through your site if it is important to your processes.

Find out more in the District FAQs or email arteachermatch@gmail.com.

 

Arkansas’ College Degree Reality Gap

In The View from the OEP on March 4, 2020 at 1:00 pm

Lately, we have been thinking about Arkansas students that are nearing the end of high school. Many students across Arkansas are taking the ACT exam, deciding on which college they will attend, working through financial aid documents, and looking forward to heading out into the rest of their life.  According data reported by ACT, 70% of Arkansas students from the class of 2019 indicated that they wanted to obtain a bachelor’s (BA) degree or higher. This high rate of post-secondary aspiration is actually a decline from the class of 2017, in which 75% of students intended to get a BA or higher degree.

But the reality is, based on current data and prior trends, only 11% of Arkansas high school graduates will obtain a bachelor’s degree within 6 years of their high school graduation.

And, although Hispanic and African American students also report high levels of post-secondary aspirations, the likelihood of getting a degree decreases for these students. Hispanic students from the class of 2017 reported a 72% aspiration rate for a BA or higher, but only 6% are projected to meet that goal.  African American students are the least likely to get a BA, with only 5% of students obtaining one, although 70% of African America students reported that aspiration.

Figure 1 represents the number of students in the class of 2017 progressing through stages from high school to a 4-year college.  All calculations are based on a kindergarten “class” of 20 to ease interpretation.

Figure 1. College Completion Pipeline for the Class of 2017, for All Students, Hispanic Students, and African American Students (4-year institutions)

4year

Step 1. High school graduation: A high percentage of students graduate from Arkansas’ high schools, with 17.6 of a typical kindergarten class of 20 graduating high school in four years (88%).  High school graduation rates are somewhat lower for Hispanic and African American students, with 17.1 and 16.7 of 20 students graduating, respectively (85.7 and 83.4%). Source: Department of Elementary and Secondary Education

Step 2. Post-Secondary Aspirations: Between 15 and 14 of 20 high school students report aspiring to obtain a bachelor’s degree or higher. Source: Department of Elementary and Secondary Education

Step 3. College-Going: Four-year college-going rates differ for the groups examined.  Out of the initial 20 student “class”, 5.6 students attend overall (48.2% of high school graduates).  Hispanic students had the lowest percentage of students attending a 4-year college with only 3.6 of the initial 20 students (39.5% of high school graduates).  African American students attended 4-year schools at a rate of 4.7 (40.3% of high school graduates). Source: Department of Elementary and Secondary Education

Step 4. Persistence: The percentage of college-going students that return for the second year of college is isn’t reported by student population, so we use the overall value reported in the recently released ACT report.  85% of students from the Class of 2017 persisted in year 2 at the in-state 4-year institutions.  Source: ACT High School to College Success Report – class of 2017-18 Freshman

Step 5. Completion: While many of us think of “4-year institutions” as taking 4 years to graduate from, colleges generally report a 150% (6 year) completion rate. The class of 2017 hasn’t had time to complete their degree, so we use the most recent data available to project their completion rates. The overall rate of 6 year completion was 45.8% for most recent cohort (started in 2013). The Hispanic student rate was 39.5% and African American completion rate was 25.6% (both for the cohort that started in 2012). Source: ADHE Comprehensive Reports.

As presented in Figure 1, a high percentage of students graduate from high school, with 17.6 of a typical kindergarten class of 20 graduating high school in four years.  High school graduation rates are somewhat lower for Hispanic and African American students, with 17.1 and 16.7 of 20 students graduating, respectively.  Between 15 and 14 high school students report aspiring to obtain a bachelor’s degree or higher, but 4-year college-going rates differ for the groups.  Out of the initial 20 student “class”, 5.6 students attend overall, Hispanic students had the lowest percentage of students attending a 4-year college with only 3.6 of the initial 20 students.  African American students attended 4-year schools at a rate of 4.7 students.  The majority of students return for the second year of school, but then fail to complete their degree within six years.  Out of the initial 20 students in the “class”, just 2.2 students in the overall population, 1.2 Hispanic students, and 1 African American student are projected to obtain their degree in 6 years.

We know that 4-year college isn’t everyone’s goal, but there are A LOT of students that report wanting to get a degree.  We highlight the gap between the aspirations and the (projected) reality of getting a degree. For the Class of 2017, the difference between students that aspired to obtain a BA degree and those that actually do would be over 22,000 students.

Figure 2.  Four-Year College Completion Pipeline for the Class of 2017, for All Students, Hispanic Students, and African American Students, difference between aspirations and projected degree completion highlighted

4year gap

We also examined the trends in 2-year college enrollment, based on the same 20 student kindergarten “class”.  High school graduation rates remain, but we can see that a much smaller percentage of students report aspiring to an associate’s degree or Voc/Tech training.  In fact, overall and for Hispanic and African American students, more students attend a 2-year school than had indicated that they wanted to, but the rates are very low. Note that Hispanic students are more likely to attend a two-year college than their peers, and are more likely to persist and obtain a 2-year degree in 3 years.

Figure 2.  Two-Year College Completion Pipeline for the Class of 2017, for All Students, Hispanic Students, and African American Students

2year


We have previously raised this issue of low college completion for Arkansas students (in 2016,  2017, and  2019), and increasing the percentage of Arkansas residents with a BA is an important driver for economic development in the state.  Over three years ago we wrote about the Closing the Gap 2020 strategic plan from Arkansas Department of Higher Education.  There have been increases in credentials awarded since the plan was implemented, but the increases are only about 1/10th of the targets identified in the report.  The new funding formula for higher education institutions rewards completion (instead of just enrollment), and early reports indicate that the funding formula may be contributing to increasing the 4-year completion rate. Hooray!

Arkansas Department of Elementary and Secondary Education is providing additional resources to support students’ success while still in high school. The ADE is providing a College and Career Readiness Tool (CCR Tool) for Arkansas students in grades 8-12. Schools can select a provider from the approved list. We recommend that schools use these resources and that the state evaluate their effectiveness.  We need to know if the tools DESE is providing are helping students develop and achieve their goals.

We look forward to greater collaboration and communication between higher education and K-12.  We think that the more information a school has about its students, the better it can serve them. We think more integration between of these systems will lead to better outcomes for Arkansas students.

As students are looking toward to their next steps, we feel like they need to understand the obstacles that they may face in pursuit of a degree.  Parents and school personnel should discuss these challenges, but it would likely be more instructive to have students from the community who went to college (and did/ didn’t complete) share about their experience, challenges, and successes.  As shown by KIPP, Arkansas high schools can do more to support their students through college transition. Education doesn’t stop at graduation. We should do all we can to help students meet their aspirations!

K-2 Assessment? Take your pick…

In The View from the OEP on February 19, 2020 at 3:40 pm

istationnweaRen

Districts are again being given the opportunity to select an assessment to administer to their students in Kindergarten through 2nd grade.  Districts initially selected a K-2 assessment in the spring of 2016, and have been using their selection for three years. This spring, districts are again being given the opportunity to choose a K-2 assessment that they will administer for the next four years.

We know that district leaders and teachers want to make the best choice to support student learning, so we did some digging into the relationship between student outcomes and which assessment was selected by each district.

We needed to use 3rd grade assessments to try to understand any relationship between the selected assessments and student outcomes, because we do not have a consistent assessment in earlier grades. Third grade data include two years of Pre- K-2 assessment and two years of Post-K-2 assessment. We use the terms “Pre” and “Post” terms relative to 3rd graders’ experience. Students who were 3rd graders in 2015-16 and 2016-17 were not exposed to the selected K-2 vendor. In 2015-16 the vendor had not been selected, and in 2016-17, the assessments were implemented in K-2 but the 3rd grade students had not used the assessment in 2nd grade the prior year.  Students who were in 3rd grade in 2017-18, however, had participated in the K-2 vendor assessment when they were in 2nd grade, and 3rd graders in 2018-19 had participated in both first and second grades.

You can read all about it in the policy brief, but here’s a quick summary of what we found:

  • The three K-2 assessments (Istation, NWEA, and Renaissance) were relatively equally selected by districts throughout the state.
  • The geographic and demographic characteristics of the districts that selected each assessment were similar.
  • Academic proficiency in 3rd grade is similar between the districts that selected different K-2 assessments.
  • There is no statistically significant difference in ACT Aspire 3rd grade growth scores between districts that selected different K-2 assessments.
  • Schools using NWEA: MAP evidenced significantly greater growth scores in ELA, although the effect was not present in the district-level analyses.
  • There are very high growth schools and districts using each of the K-2 assessments.

Although this is not a causal analysis, we can detect no relationship between district-level academic growth of 3rd grade students in Math and ELA, and the K-2 assessment selected by the districts. Interestingly, we do find a positive relationship at the school level between ELA growth and districts that selected NWEA: MAP.  This is likely due to the fact that large districts with multiple elementary schools all use the same assessment but some schools have more positive growth than others.  The difference in growth may be capturing the fact that schools which are more effective at ELA instruction are choosing to use NWEA, or that school implementation of NWEA is positively benefitting students in some ELA classes.

Given the variation in growth scores among districts and schools that selected the same assessment, it is important to point out that which assessment that is selected does not seem to be related to student outcomes.  Likely, it is how students and teachers act on the information gathered from the assessments, and what learning opportunities are present in the classroom daily, that results in better learning outcomes for students.

 

Year-Round Schools in Arkansas

In The View from the OEP on February 12, 2020 at 10:07 am

Continuous learning schools in the Fayetteville Public School District have drawn media attention as they consider returning to a traditional school calendar. Asbell Elementary and Owl Creek currently operate on continuous calendars but, pending a school board vote later this month, may switch back to traditional calendars for the next school year. Today we explore the history of these alternative school calendars, and the pros and cons for students, teachers, and families.

Continuous learning schools, also known as year-round schools, incorporate several shorter breaks throughout the school year and a shorter break during the summer. Despite the ‘year-round’ title, students in these schools typically attend the same number of school days as students in schools on a traditional calendar. In Fayetteville, all schools start the same week in August, but continuous calendar schools end two weeks later than other schools in June. The continuous learning schools have two week-long breaks that are different from the traditional calendar – one the first week of October and another during April. As parents and the superintendent push for a return to the traditional calendar it is important to consider why year-round calendars came to exist and what they contribute to the educational landscape.

Brief History of Year-Round Schools

First attempted in the 1980s, year-round schools were created to push back on what was considered an antiquated school calendar based on economic, rather than educational, considerations. Although there is some debate, consensus says the traditional school calendar originated as a result of the need for students in rural areas to return to the fields for work during the summer months. Continuous learning calendars may be implemented to reduce over-crowding in schools or to improve student outcomes. Continuous learning schools can alleviate crowding in large schools where multi-track calendars allow different student groups to attend school at different times, alleviating space constraints. Advocates of continuous schooling suggest the shorter, more frequent breaks in learning could reduce learning loss between school years. They cited evidence from studies, which demonstrated that students experience a “summer slide” in which they lose knowledge and skills during the long break in schooling from June to September. This loss is especially pronounced for children from low-income backgrounds.

The increased frequency of breaks on a continuous learning calendar was also thought to provide non-academic benefits for students, teachers, and parents. Student-learning fatigue and teacher burnout could be reduced though the alternative calendar. Families may enjoy taking vacations when fewer families are traveling, and may avoid some childcare expenses if the school offers a no-cost intersession opportunity for students to participate in learning opportunities at the school over the breaks.

As a result of these hypothesized benefits, states around the country have implemented continuous-learning schools at varying scales. Research, however, demonstrates that switching to a year-round calendar has little effect on student achievement and may even be harmful in certain circumstances (McMullen & Rouse, 2012; Graves, 2010). Despite intersession programming designed to provide remediation and enrichment to students, the hoped-for benefit of continuous schooling to student-learning outcomes has proved insubstantial. The benefit to teachers is questionable as well, as some report enjoying the more frequent breaks while others are nostalgic for a longer respite from the demands of the classroom.

Parents who have multiple children of different ages generate the greatest pushback against the continuous learning calendar. Year-round calendars are frequently implemented at the elementary school level as schools serving older students struggle to accommodate extracurricular practice and game schedules on the alternative calendar. Since continuous calendars aren’t offered comprehensively across districts, parents with students in varying grade levels must juggle multiple breaks and calendars that do not align. This negates potential benefits of shorter breaks and can leave families feeling frustrated.

In addition, year-round schools may increase costs for districts due to increased transportation and operational costs associated with longer calendars and the lack of overlap with other schools. Coupled with the lack of evidence that continuous learning benefits academic achievement, a calendar change can be a hard sell when not implemented district-wide.

Year-Round Schools in Arkansas

Arkansas has had year-round schools since 1993 when Texarkana converted to a continuous calendar (Fritts-Scott, 2005). Data from the Arkansas Department of Education dating back to 2004-05 shows a decline in the number of year-round schools across the state since the early 2000s when as many as ten schools in nine different districts operated on continuous calendars. Between 2005 and 2008, Little Rock and Pulaski Special School Districts led the state in the number of alternative-calendar schools, but they have not operated one since the 2007-08 school year.

The mid-2000s saw an increase in the number of year-round schools that operated in Northwest Arkansas districts, but the number is now declining. Rogers and Bentonville each operated two continuous learning schools but have since returned them to a traditional calendar. Bentonville converted its schools to a traditional calendar in 2016 and Rogers switched the one remaining school in 2019-20. Fayetteville is unique as it is the only district in Northwest Arkansas that increased the number of schools offering year-round calendars in the last five years. Happy Hollow became a continuous learning school in 1996, and was joined by Asbell in 2008-09 and Owl Creek in 2014-15.

Asbell and Owl Creek may return to a traditional calendar next year, pending the vote by the school board next week. Attendance issues and low turnout to intersession activities are cited as impetus for the change. Upon investigation however, the average daily attendance of both Happy Hollow and Owl Creek has increased and shows no variation across quarters, and while Asbell’s attendance declined between 2013 and 2016 it has been increasing again since 2017. These changes in attendance rates, however, are likely due to a variety of factors and may not necessarily be due to the year-round calendar. Surveys from Owl Creek and Asbell reflect that 66% of school staff and 50-55% of parents support returning the schools to the traditional calendar.

Some Arkansas schools, however, are switching to a continuous learning calendar. Arkansas Arts Academy, an open-enrollment charter school in Northwest Arkansas, switched to a continuous calendar in 2017-18. In addition, Magazine School District converted both its high school and elementary school to a continuous calendar in the 2018-19 school year. As opposed to the Fayetteville calendar, which has the continuous learning schools ending later, the Magazine school calendar will start two weeks earlier in August and end at the same time as previous years. Magazine will provide an interesting case study for continual schooling in Arkansas since the change was district- wide. Due to the comprehensive nature of the change, the results in terms of attendance, student achievement, and parent and teacher satisfaction will be easier to gauge and might provide more insights into the value of year-round schooling for Arkansas students.

Since the quantifiable effect of continuous schooling is ambiguous at best, it is up to the stakeholders in each district to make decisions about what calendar meets the needs of their students.

Examining NWA Charter Schools Enrollment Trends

In The View from the OEP on February 5, 2020 at 2:04 pm

This month, open-enrollment charter schools throughout the state will hold public, random lotteries for students hoping to attend the schools in the 2020-21 school year.  Open-enrollment charter schools are public schools that are open to students regardless of their residentially-assigned traditional school district. Charter schools receive their charter from the Department of Elementary and Secondary Education, which holds these schools accountable to certain standards in order to stay open. They are publicly-funded and free of tuition.

Northwest Arkansas is currently home to nine public open-enrollment charter schools, with plans to open a new charter school for the 2020-21 school year. These schools, which serve unique missions, are some of the most highly ranked schools in the State of Arkansas. While critics argue that public charter schools segregate based on race or academic ability, national evidence finds that these claims are highly context specific. In today’s blog (and associated Policy Brief and Arkansas Education Report) we present what conclusions can we draw about Northwest Arkansas charter schools based on enrollment trends in recent years.

Similar to our previous work examining charter school enrollment trends in Little Rock, we begin by examining traditional and charter enrollment trends by student demographics and end with analyzing the academic performance of students that switch between traditional and charter sectors.

Arkansas Arts Academy schools provide an arts-based approach to learning. Northwest Arkansas Classical Academy schools have a classical focus, including the Socratic method and instruction in Latin. Each of the four Haas Hall campuses emphasize preparation for higher education with a semester block schedule. LISA Academy offers a Science, Technology, Engineering, and Mathematics (STEM) curriculum. Finally, Hope Academy, which will open for the 2020-21 school year, will focus on serving children who have experienced trauma.

Charter schools in Northwest Arkansas enrolled 2,581 students in 2017-18, which was just under 3% of the nearly 90,000 public school students in Benton and Washington counties. Figure 1 presents Northwest Arkansas charter school enrollment from 2007-08 through the 2017-18 school year.

charter1

The 2017-18 enrollment data presented in Table 1, shows that charter schools in Northwest Arkansas enroll a larger proportion of White, Asian, and multi-racial students, than the traditional public districts.  The charter schools enroll a smaller proportion of other ethnic groups, students eligible for free– and reduced-priced lunch, English learners, and students eligible for special education services.

NWA Charter 2

When we examine ten years of enrollment data, we see that all Northwest Arkansas schools are becoming increasingly racially/ ethnically diverse and that charter schools are growing more similar to district public schools in their race/ ethnicity demographic composition. In 2007-08, less than 35% of students enrolled in NWA traditional public schools and around 10% enrolled in NWA charter schools identified with a minority group. In 2017-18, over 40% of the traditional public school population and 30% of the charter school population identified as a minority race or ethnicity. The district-charter minority enrollment gap was nearly 25 percentage points in 2007-08, but had shrunk to just over 10 percentage points ten years later.

Between 2009-10 and 2016-17, approximately 50% of students enrolled in traditional public schools were FRL-eligible. In contrast, approximately 20% of students enrolled in public charter schools were FRL-eligible. Similar disparities persisted for EL students (around 20% of traditional public school students and 3% of charter students) and SPED students (around 6% traditional public school and 3% charter). These trends raise the question of why NWA charter schools have become more integrated based on race, but not for FRL, EL, and SPED students?

Public charter schools are often accused of “cream skimming” (enrolling higher proportions of high-performing students) and “cropping” (encouraging low-performing students to enroll elsewhere). Do we see evidence of this with Northwest Arkansas public charter schools? In an effort to answer this question, we examine the academic performance of students who switched between the traditional and charter school sectors.

Students who exit NWA traditional public schools to enroll in a NWA charter school are, on average, academically high performing.  They scored two-thirds of a standard deviation above the state average on state assessments, and one-third of a standard deviation above the school average of the school that they moved from. The traditional public schools that students are leaving to go to a charter are high performing schools as well. Almost 58% of students who exit NWA traditional public schools to enroll in a NWA charter school left a school with a Z score in the top third of all NWA public schools.

Students leaving NWA charter schools to enroll in a NWA traditional school are also academically high performing on average. They scored on third of a standard deviation above the state average on state assessments. They were average performers, however, for the charter school that they exited.  About 40% of the students exiting charters left a school in the top third of all NWA public schools in terms of student achievement.

Taken together, this evidence suggests that higher performing students are leaving traditional schools to attend charter schools.  We have no evidence WHY higher performing students are leaving traditional schools, but possible reasons might be that they are attached to curricular options, changes in peer groups, or smaller classes. On the other hand, we do not see evidence that the students exiting charter schools are being ‘pushed out’ for low academic performance as they are average academic performers compared to their peers at the charter school that they are exiting.

The charter sector in NWA has grown rapidly over the past ten years, but continues to serve a small proportion (3%) of all public school students in the area. The region has grown more racially and ethnically diverse in that time. Public charter schools have also grown more diverse, though they continue to enroll a smaller proportion of certain student populations. Here’s what we think are important steps moving forward:

  1. Continue to monitor differences in demographic enrollment trends by sector.  Charter schools should be reaching out to all communities to communicate the opportunity to enroll, and if particular groups are not expressing interest we should try to learn more about why.  Do they feel that they are not ‘the right kind of applicant’ or do they prefer the opportunities that they are given in the traditional public sector?
  2. Gain a better understanding of why FRL, EL, and SPED students enroll in charter schools at such low rates. These enrollment trends may be related to problems with practical solutions, such as transportation. It may be that the families of these students are satisfied with services provided at their residentially-assigned district public school. It may be that students are interested in attending but are not being selected in the random lotteries that charters must hold if oversubscribed. Understanding the reasons for these enrollment trends is essential to crafting policy-relevant solutions.
  3. Respond to market demands. NWA charter schools enroll only 3% of all public school students in Benton and Washington counties. However, many of the charter schools are oversubscribed with waiting lists of students not selected through random lotteries.  The interest in charters suggests many more students may be interested in enrolling in these schools. Traditional public schools should communicate with students and parents to determine if their needs are being met, and, if not, how they can better support their educational experience.

 

 

 

 

District Funding Equity

In The View from the OEP on January 29, 2020 at 2:04 pm

Over the past few months we’ve written several posts about school spending. Most recently, we showed that there appears to be essentially no relationship between spending and test score growth. Of course the aim of education is broader than just test score growth, but increasing knowledge and skills in core subjects, as demonstrated on tests, is certainly an important outcome. However, given the broader purpose of education and people’s general desire for fairness, funding equity has also been a longstanding education policy issue.

For Arkansas, funding equity has had particular importance since the Lake View school district filed its original court case in 1992 alleging that the state’s funding system was inequitable and inadequate. The Lake View case eventually made its way to the Arkansas Supreme Court and has since had a profound impact on the state’s approach to school funding. This post looks at school funding equity post-Lake View to see where things stand today.

We used the Arkansas Department of Education’s Annual Statistical Reports to analyze property wealth and school district revenue between 2004 and 2018. All of the dollar amounts presented below are per pupil, meaning we divide property wealth and district revenue figures by the number of students the district serves. Using per pupil figures provides an apples-to-apples comparison for districts of varying sizes.

A key concern in the Lake View case was that affluent districts had significantly more property wealth than poor districts, and could, therefore, generate significantly more funding with the same tax effort. Lake View argued that state and federal dollars were not sufficient to make up for local funding deficits in poorer communities around the state. Below we look at (1) how property wealth and revenue have changed over time, (2) the relationship between revenue per pupil and property wealth, and (3) the relationship between local funding share and property wealth.

Both property values and school district revenue have increased significantly. Between 2004 and 2018, the median assessed property value per pupil for Arkansas school districts increased from $49,803 to $93,301, an 87 percent increase over 14 years. The median is the value where half of districts have higher values and half have lower values. Over the same period, median revenue per pupil increased from $7,191 to $12,112 (a 68 percent increase), and the locally generated share of total district revenue increased from 24 percent to 32 percent. Property values and funding per pupil have both grown at healthy rates, and on average, a greater share of school districts revenue is coming from local property taxes. Next we investigate the relationship between property wealth and revenue.

School district revenue has a weak, positive relationship with property wealth, but overall appears to be relatively equitable. Figure 1 below shows the relationship between property wealth and school district revenue for 2005 (blue dots), 2008 (orange dots), and 2018 (grey dots). Each dot in the figure represents a school district. The data show a weak, positive relationship between revenue and property wealth that has not changed much over time – the dots get slightly higher as property wealth increases from left to right and the different colored dots are similarly distributed. It does not appear from this graph that there are large, systematic inequities built into Arkansas’ school district funding system – the dots are somewhat evenly scattered with only a slight upward tilt from left to right.

Figure 1: The relationship between Arkansas school district revenue per pupil and property wealth between 2005 and 2018

State and Federal funding is being used to equalize school district funding between wealthy and poor communities. Figure 2 shows the relationship between the locally generated share of school district revenue and property wealth. There is a strong positive relationship between locally generated share and property wealth, meaning that wealthier school districts’ taxpayers contribute a greater percentage of school district revenue than do taxpayers in less well-off districts. In other words, state and federal dollars are being heavily allocated toward poorer communities, allowing those school districts to be funded with less reliance on local property wealth.

Figure 2: The relationship between the locally generated share of school district revenue and property wealth between 2005 and 2018

Since Lake View, Arkansas has made many changes to its school funding formula in an effort to improve both adequacy and equity. Our analysis shows that even as property values have grown significantly and a greater share of overall district revenue is coming from local property taxes, Arkansas current funding formula appears to be relatively equitable. There is only a weak, positive relationship between property wealth and revenue, but there is a strong positive relationship between local revenue share and property wealth –  state and federal dollars are being used to mitigate wealth differences across districts. While we still might want to do more to support our poorer communities, it is good news that the equity concerns raised in Lake View seem less apparent today.