University of Arkansas Office for Education Policy

Archive for 2019|Yearly archive page

English Language Learners in Arkansas

In The View from the OEP on June 18, 2019 at 3:16 pm

English language learners (ELs) are a growing student population throughout Arkansas.  With the passage of ESSA, the progress of EL students towards proficiency in English has been added to the metrics by which schools and districts are held accountable. With this academic context in mind, we at the Office for Education Policy thought it would be useful to provide a descriptive analysis of the EL population in Arkansas. These analyses, beginning with an overview of the recent enrollment trends in the state, may help policymakers and school leaders better understand this important group of students.

Recent data from the National Center for Education Statistics (NCES) show that ELs comprise about 9.5% of K-12 students in the United States. Some states have a larger share of EL students than others. For example, in fall 2016, only about 1% of West Virginia’s student population were ELs while nearly 16% of students in Nevada were ELs.

We really like this map from USDOE that shows the percentage of ELs by school district across the country.

EL Map

Additionally, some states are experiencing faster growth in the proportion of EL students than others. For example, the states with the highest growth of EL student population from 2000 to 2014 include Arkansas.

The change in Arkansas’ EL population by district from 2009-10 to 2014-15 is presented below.  Most districts had few or no ELs, as represented by black or gray shading, but many districts are colored dark purple, indicating an increase in EL enrollment of at least 50% over the years examined.

El Change legend

Change in EL

 

With such large changes in the EL population by 2014-15, we wanted to bring in more current data to examine the trends here in Arkansas.  Examining data from the Arkansas Department of Education, we describe overall EL enrollment in the state, the proportion of EL students to general population, and the proportion of EL students by academic region for the past 15 years.

Table 1, below, presents the number of students enrolled in English language learning programs over the last 15 academic years. Overall, the total number of EL students enrolled in Arkansas schools has doubled from just under 20,000 students in the 2005 academic year to just under 40,000 in the 2019 academic year. All regions have seen increases in the number of EL students from 2005 to 2019. In total numbers, the Northwest academic region welcomed over 23,000 EL students in AY 2018-19 – the most by far.

Table 1. EL Enrollment in Arkansas, 2004-05 through 2018-19.

ELinAR

Changes in the proportion of EL students in Arkansas from 2004-05 through 2018-19 are illustrated in Figure 1, below. The figure shows the EL percentage of the general student population in Arkansas, by academic region, and in the United States overall. The available data for the national EL rate, represented by the blue dotted line, reflect that the overall share of the EL K-12 population is consistently between nine and ten percent of the general school population.

Figure 1. Percentage of General Population Identified as EL, 2004-05 through 2018-19.

ELProportion

Several key points should be noted from Figure 1.

  1. Although Arkansas enrolls a smaller percentage of EL students than the national average, the share of Arkansas public school students identified as EL is increasing, from 4% in 2004-05 to 8% in 2018-19.
  2. While Northwest Arkansas consistently enrolls the highest percentage of EL students in the general student population, both the Southwest region and the Central region have seen large relative increases in the percentage of EL students enrolled.
  3. A recent decline in the percentage of EL students is also evident in Figure 1. Starting with academic year 2016-17, there is a downturn in the percentage of EL students in Arkansas overall, and a sharp decline in the Northwest region in particular. This decline coincides with a change in the assessments used to reclassify EL students as proficient in English. Beginning in academic year 2016-17, Arkansas replaced the ELDA with ELPA21, and although not evident in all regions, the decline in identified students overall is likely due to a greater percentage of EL students being identified as English Proficient through the new assessment.

In addition to looking at the proportion of ELs enrolled in the general population, we examined which region has the highest proportion of EL students overall and how that has changed over time. Figure 2 illustrates the shift in the overall share of EL students between the academic regions over the last 15 years. While all regions demonstrate growth, Figure 2 illustrates that while Northwest Arkansas consistently has the highest number of EL students enrolled, the region’s proportion of the EL population in the state is decreasing, due, in part, to an increase in the numbers in the Central region.

Figure 2. Share of EL Enrollment by Academic Region, 2004-05 through 2018-19.

ELALLRegionIn 2004-05, over three-fourths of the state’s EL students were enrolled in Northwest Arkansas schools. This has dropped to 61%. In contrast, both the Central and Northeast regions have experienced growth in the share of the overall proportion of EL students. The Central region’s share, for example, has increased from 11% to 2005 to over 21% in 2018-19.

We finally illustrate this major shift in student composition in the state by looking at the extremes – districts with no EL student enrollment and districts with at least 10% of their student population identified as ELs. We illustrate this in Figure 3. While the number of EL students in Arkansas has increased, so has the number of individual districts that enroll these students. Look at the Northeast region, for example. In 2004/05, 56 of the 75 districts in the Northeast region did not enroll a single EL student. None of the remaining districts in the Northeast region had EL enrollments that comprised 10% of the overall student population. By 2019, however, only 17 districts in the Northeast have no EL enrollment and two districts had EL enrollments that comprised over 10% of their student population. In 2019, every academic region of Arkansas has at least two school districts where at least 10% of their student population are English learners.

Figure 3. Share of EL Enrollment by Academic Region, 2004-05 through 2018-19.

ELExtremes

legend


The English Learner student population in Arkansas has more than doubled over the last decade and a half. While the majority of EL students attend schools in Northwest Arkansas, the Central and Southwest regions have higher rates of growth. Additionally, schools in the Northeast academic region account for whittling some of the Northwest region’s share of EL students. The share of EL students in the Northeast has grown by over 2.5 percentage points from 2018 to 2019.

While the number of EL students has grown, so has the number of districts in which they are enrolled.  In 2004-05, 155 districts throughout the state enrolled no EL students, but in 2018-19 there were only 44 such districts.  In addition, over 30 districts have at least 10% of their students identified as EL, compared to fewer than half that 15 years ago.

The differential growth patterns in the EL population in Arkansas may be problematic for districts that have had few or no EL students enrolled. Districts that have long welcomed EL students may have greater institutional knowledge to identify and support their EL students as they learn and become proficient in English. Districts that have experienced high EL growth, or have recently received their first EL students, may struggle with how best to meet the learning needs of these students. This can be an opportunity for the veteran EL districts to be pedagogical leaders, sharing their expertise with other districts.

It is important to ensure that EL students who attend schools in all regions of Arkansas are afforded the opportunities to succeed in learning English as well as core academic content. In addition to looking descriptively at the educational contexts EL students find themselves in, we will examine how EL students are doing academically in future installments of this OEP blog – watch this space!

 

 

Arkansas School Spending by Level

In The View from the OEP on May 22, 2019 at 10:58 am

Two weeks ago we provided a first look at Arkansas’ brand new school-level spending data. Today we dig a little deeper to look at how spending varies by school level. We show that there is significant variation in spending based on the grades schools serve, and we attempt to explain some of that variation by looking at several important spending categories.

In the coming weeks we also plan investigate how spending varies by school type (i.e., traditional public vs. open enrollment charter) and whether Arkansas’ schools exhibit meaningful economies of scale (i.e., do bigger schools spend less on things like administration). For those interested in playing with the school-level spending data on their own, you can either go to MySchoolInfo.Arkansas.gov or download the data from our website.

For this post we divide schools into three levels: elementary, middle, and high. Because districts have the flexibility to distribute grades differently across schools, there is some variation around the state in school level terminology. However, we chose to use the same categorization that is used for Every Student Succeeds Act (ESSA) reporting. In general, this means that we label schools serving kindergarten through 5th grade as elementary, grades 6 through 8 as middle, grades 9 through 12 as high.

Figure 1 shows total per pupil spending by school level. While there is a lot of variation within each level, a clear pattern is evident across levels – high schools spend the most per pupil, middle schools spend the least, and elementary schools are in between. We have included the school level medians to further illustrate this point. The median represents the school exactly in the middle of the per pupil spending distribution for each level. Half of schools spend more than the median and half spend less. The high school median is the highest at $10,594. The elementary school median is $9,586 or $1,008 per pupil lower, and the middle school median is $9,185 or $1,409 per pupil lower.

Figure 1: 2017-18 Total Spending Per Pupil by School Level

To better understand what’s driving spending differences across school levels, we take a closer look at four important categories that together represent nearly 80 percent of all Arkansas school expenditures: instruction, administration, student support services, and instructional staff support.

Figure 2 shows the per pupil spending devoted to instruction. This category includes teachers’ salaries as well as books and other instructional supplies. Here too Arkansas’ high schools spend more than elementary and middle schools. The median high school spends $6,184 per pupil on instruction, while the median elementary and middle schools spend $5,245 and $5,055 respectively. Higher per pupil spending on instruction explains nearly all of the difference in total per pupil spending between the median high school and elementary school and about half of the difference between the median high school and middle school. High schools spend more on instruction because class sizes generally decline as grade level increases and high school teachers can be more expensive due to factors like the prevalence of advanced degrees and other specialized skills/certifications (i.e., national board certified).

Figure 3 depicts the distribution of school spending on administration (i.e., school and district). This category includes school principals as well as district administration and the superintendent. Spending on administration is quite similar across school levels. The median elementary and middle schools spend $727 and $763 per pupil respectively on administration. The median high school spends $845 or right around $100 dollars more per pupil. Given the nature of supervising older students, it’s not too surprising that administration spending increases with grade level and that high schools spend the most on administration.

Figure 2: 2017-18 Per Pupil Spending for Instruction by School Level

Figure 3: 2017-18 Per Pupil Spending for Administration by School Level

Figures 4 and 5 show per pupil expenditures on support services for students and instructional staff respectively. Student support services includes things like physical and mental health services and guidance counselors, and instructional staff support includes expenses related to professional development. Arkansas’ schools generally spend less on support services for students and instructional staff than they spend on administration, but there is significant overlap in the distribution of all three categories, with instructional staff support exhibiting the most variation (i.e., the distribution is much more spread out).

Elementary schools spend more for support services. Median elementary school spending for both types of support services is around $100 more per pupil than median high school or middle school spending. The median elementary school spends $537 per pupil on student support services, while the median middle school spends $431 and the median high school spends $451. And for instructional support, the median elementary school spends $785 per pupil, while the median middle school spends $653 and the median high school spends $628.

Figure 4: 2017-18 Per Pupil Spending for Student Support Services by School Level

Figure 5: 2017-18 Per Pupil Spending for Instructional Staff Support by School Level

The upshot of our analysis is that Arkansas spends the most on high schools and the least on middle schools, with elementary school spending falling in between. Greater total spending on high schools can largely be explained by higher instructional expenditures, which are likely strongly related to smaller class sizes. High schools also tend to spend more on administration. Elementary schools, on the other hand, tend to spend more on student support services and instructional staff support.

Middle schools spent the least overall as well as on instruction and student support services. While we were not surprised by the greater spending on high schools, prior to performing this analysis, we would have guessed that the high school spending gap would have been smaller and that elementary and middle school spending would have looked more similar. Arkansas’ middle school spending might be an area worth further investigation given that it represents a critical juncture for kids, and as a country, we have made less progress improving academic performance in the middle grades and beyond than we have in elementary school. For those interested in some additional reading on middle school performance check out the Education Next articles here, here, and here.

As we mentioned in the introduction, we have a couple of additional topics that we plan to investigate using the new school-level spending data. However, we also want to hear what you are interested in learning more about. Please feel free to suggest topics or questions in the comments or just reach out to us directly.

The Seven Year Hitch

In The View from the OEP on May 15, 2019 at 12:04 pm

Keep Calm

Today we heard that the ACT Aspire is sticking around at least through 2026. ADE recently signed a contract for 7 additional years of ACT Aspire assessments.  We know that the assessment window just ended and that testing is that LAST thing educators throughout the state want to think about right now, but this consistency is great news!

We think sticking with the ACT Aspire is a great plan and applaud ADE for not changing the assessment for grades 3-10 at this time. Although we can’t compare Arkansas’ student achievement with any other state (because, so far as we can tell we are the only state currently using ACT Aspire as our annual assessment for grades 3-10) we think it is important to continue with the ACT Aspire so that we can look over time to see what is (and isn’t) working for students in our state.

For example, consider some recent analyses that we have been doing to see if differences in achievement for particular student groups are decreasing over time.  We discussed the statewide achievement gaps by race and gender in an earlier blog post, but today we present differences in student achievement by participation in the Free/ Reduced Lunch program. To be FRL eligible a student’s family must have an income below 185 percent of the federal poverty line, and so participation in the program is a rough measure of the student poverty.  Students from lower income backgrounds generally score lower on state assessments than their more-economically advantaged peers.

We examine the difference in average achievement percentile on the annual state assessments between FRL-eligible students and students that, based on higher family incomes, are not eligible to participate for the FRL program. Information is presented by grade, and, we would hope that over time, effective schooling would decrease the differences in achievement between the groups.

Figure 1, below, illustrates the difference in the average literacy percentile between FRL students and students that are not participating in the FRL program. We can see that the achievement gap for FRL-participating third graders was -26 percentage points in 2008-09, and that it had closed to -19 in 2017-18. Similar gap reduction is evidenced for 4th, 5th, and 8th grades. There has been a lot of variability in the intervening years, however, and it is difficult to say with confidence that the gaps will continue to close.

Figure 1. Difference in Average Literacy Achievement by Free/Reduced Lunch Eligibility, by Grade, 2008-09 through 2017-18

FRL LIT grade2

Figure 2, illustrates the difference in average math percentile between FRL-eligible students and Non-FRL students. Information is presented by grade, and we can see that the math achievement gap for FRL-participating third graders was -25 percentage points in 2008-09, and that it remained at -24 in 2017-18. A similar pattern is present from all grade 3-8, but, as with literacy achievement, there has been quite a bit of variation in the gap over the last decade.

Figure 2. Difference in Average Math Achievement by Free/Reduced Lunch Eligibility, by Grade, 2008-09 through 2017-18Math FRL Grade

We can’t fail to notice the major difference in both the literacy and math achievement gaps in 2014-15.  As noted on the figures, this was the year that the PARCC assessment was administered. Because this was the only year that the PARCC was administered, we are unable to determine if the reduction in the gaps was a real reflection of how our students were performing, or just some type of effect of a different test. Variation like this makes it difficult to measure our progress in making a difference for kids, so we are happy to hear that Arkansas is committed to ACT Aspire for another 7 years. Fingers crossed that a consistent target will reflect increased achievement for our students!

 

First Look at Arkansas’ School-Level Spending

In The View from the OEP on May 8, 2019 at 12:27 pm

Each year the Arkansas Department of Education (ADE) releases data on public school revenue and expenditures. However, those financial data have traditionally only been available at the district level. It has been nearly impossible to drill down to see how money is being spend at individual schools. But that is beginning to change. The Every Student Succeeds Act (ESSA), which passed in 2015, included a provision that required states to begin reporting school-level spending by the end of 2018 (see this EdWeek article for more info). Secretary DeVos gave states an extra year to comply with the requirement, but Arkansas is ahead of the curve and has already released school-level spending data this year as part of the school report cards. Here is the link to the data that has been posted onto our website, that sources the data from https://myschoolinfo.arkansas.gov. This post is the first in its series that will provide a first look at the brand-new school-level spending data.

With the new data, we can now analyze how spending is distributed across Arkansas’ schools. This is important because public education dollars are allocated to districts, and those districts have relatively broad authority to apportion dollars across the schools that they oversee. Until now it’s been very hard to compare how districts within the state allocate dollars to schools or to know if they are doing so in an equitable manner.

On average, Arkansas spent $9,914 per pupil during the 2017-18 school year. Average spending per pupil is calculated by dividing total expenditures, including state, local, and federal dollars, by the number of students in average daily membership (ADM). Dividing spending by the number of students allows us to compare expenditures across districts/schools of different sizes.

Figure 1 below shows the distribution of Arkansas school-level spending. These data include all public schools for which spending data are available, including the handful of Alternative Learning Environment (ALE) schools, charter schools, magnet schools, etc. Median school spending, the value where 50 percent of schools spend more and 50 percent spend less, was $9,814, roughly in line with the state average. The figure also displays the 25th($8,783) and 75th($11,218) percentiles. Fifty percent of schools’ per pupil spending falls between these two values, while 25 percent spend more and 25 percent spend less.

Figure 1: Distribution of 2017-18 School-Level Spending Per Pupil

Screen Shot 2019-05-08 at 9.24.44 AM

As noted earlier, we are interested to learn whether districts are allocating dollars equitably across schools serving students of varying poverty levels. To begin to answer that question, Figure 2 provides a scatter plot of school-level per pupil spending versus the percentage of students eligible for the free and reduced lunch program (FRL). To be FRL eligible a student’s family must have an income below 185 percent of the federal poverty line, and so the percentage of students in a school who are FRL eligible is a rough measure of the school’s poverty level.

The data presented in figure 2 show that Arkansas’ schools spend more the higher a school’s poverty level. Spending more in higher poverty schools helps to ensure that those students who are most in need get the support and educational opportunities that they deserve. The progressivity of Arkansas’ school funding has been well documented at the state/district level (for example here and here). However, this is the first time we have been able to show that this progressivity extends down to schools.

Figure 2: Scatter Plot of 2017-18 School-Level Spending Per Pupil vs. Percent FRL

Screen Shot 2019-05-08 at 9.24.27 AM

Digging a bit deeper, we can divide school spending into two broad categories – spending on people and everything else. The ADE’s data system calls these two categories personnel and non-personnel. Figure 3 shows how spending in these two categories is distributed at the school level. The median school spent $7,399 per pupil on people and $2,394 per pupil on everything else.

We can also look at these two categories as a percentage of total school spending. On average, 76 percent of Arkansas’ public school spending paid for public school employees’ salaries and benefits and the remaining 24 percent covered everything else. Figure 4 shows the school-level distribution of the percentage of total spending that was devoted to paying for people. While there is some variation, the vast majority of schools spend between 70 and 80 percent of their budget on people. As is the case across the country, Arkansas’s public schools spend most of their money on people.

Figure 3: School-Level 2017-18 Spending Per Pupil used for Personnel vs Non-Personnel

Screen Shot 2019-05-08 at 9.24.59 AM

Figure 4: Percent of School-Level 2017-18 Spending Used for Personnel

Screen Shot 2019-05-08 at 9.25.12 AM

We can also look at spending by activity or purpose. The ADE divides spending into several activity-based categories, but we thought four of the most interesting categories were instruction (e.g., teachers’ salaries, books, etc.), student support services (e.g., physical and mental health services, guidance counselors, etc.), instructional staff support (e.g, professional development, etc., and administration (principles, superintendents, other central office staff, etc.). The median Arkansas school spent 56 percent of its budget on instruction, 5 percent on student support services, 7 percent on instructional staff support, and 8 percent on administration. Arkansas’ schools spend most of their budgets on instruction; however the other three categories together make up approximately 25 percent of the remainder. Figure 5 shows the school-level distribution for instruction, and figure 6 shows the other three spending categories.

Figure 5: Percent of School-Level 2017-18 Spending Used for Instruction

Screen Shot 2019-05-08 at 9.25.24 AM

Figure 6: Percent of School-Level 2017-18 Spending Used for Various Activities

Screen Shot 2019-05-08 at 9.25.37 AM

It has long been extremely difficult, if not outright impossible, to analyze education spending at the school level. Most districts simply did not have accounting systems that captured the information and most states, including Arkansas, did not require districts to report school-level spending. That changed with the passage of ESSA, and now Arkansas’ parents, taxpayers, and policymakers are able to see and compare school spending across the state. This presents an excellent opportunity to use this unprecedented level of transparency to improve the equity, efficiency, and efficacy of Arkansas’ public education spending. In sum, it’s an exciting time for the school finance nerds!

In the coming weeks, we plan to publish another post or two looking at spending by school type (e.g., traditional vs charter) and by level (e.g., elementary, middle, high) as well as an investigation into whether Arkansas’ schools exhibit economies of scale (i.e., larger schools spend less on certain categories like administration). In the mean time, don’t forget to visit our website to take a look at the new School Level Financial Data that we’ve posted. Stay tuned for more info in the upcoming weeks!

What Does it Mean to be One of the “Best” High Schools?

In The View from the OEP on May 1, 2019 at 1:38 pm

US Badge

Yesterday, U.S. News & World Report released their annual “Best High Schools” rankings.  These rankings always make the news but here at the OEP, we want to make sure that you understand what the “best” title is based on. U.S. News changed their methodology this year, so we want to share what we like (and don’t) about the new methodology, and examine what we think is a missing indicator of high school success- whether or not students are enrolling in college after graduation.

First, congratulations to those Arkansas high schools that made the list for 2019! Below are the US News Top 10 for Arkansas (for context, we noted the 2016-17 free/reduced lunch rate provided in MySchoolInfo) :

#1: Haas Hall Academy (did not participate in lunch program)
#2: Haas Hall Bentonville (did not participate in lunch program)
#3: LISA Academy North Charter School (34% FRL)
#4: eStem High School (27% FRL)
#5: Arkansas Arts Academy High (25% FRL)
#6: Maumelle Charter High (20% FRL)
#7: Bentonville High School (20% FRL)
#8: Greenbrier High (32% FRL)
#9: Rogers New Tech (50% FRL)
#10: Concord High (64% FRL)

While many of these high schools have previously been recognized as “Best”, we can’t compare this year’s ranking with any prior year due to the change in methodology.


What makes these schools the “Best”?

U.S. News uses a new methodology this year, and includes six indicators to rank the nation’s high schools.  The indicators are combined to give each high school’s overall score between zero and 100 that represents what percentile position a school is in out of the 17,245 ranked schools across the nation. Listed below, along with how much weight each is given in the overall score calculation, the indicators represent advanced course taking/exam passing, performance on state assessments, and graduation rate.

  • Indicator 1: College Readiness (30% of overall score)  The proportion of a school’s 12th graders that took and passed Advanced Placement/International Baccalaureate exams.
  • Indicator 2: Math and Reading Proficiency (20% of overall score)  A simple measure of schools’ student performance on state assessments. This measure is a weighted measure of performance, where schools are awarded 1 to 4 points per student depending on their performance on the ACT Aspire.
  • Indicator 3: Math and Reading Performance (20% of overall score)  A measure of how students in a school perform on state assessments compared to how U.S. News predicted a school would score given the demographic characteristics of it’s students.
  • Indicator 4:  College Curriculum Breadth (10% of overall score) A measure of how many 12th grade students passed multiple AP/IB exams.
  • Indicator 5:  Underserved Student Performance (10% of overall score) An evaluation of the difference between how underserved students (black, Hispanic, and low-income) scored on state assessments compared with the average for non-underserved students.
  • Indicator 6:  Graduation Rate (10% of overall score) The proportion of students who entered ninth grade in the 2012-2013 academic year who graduated four years later.

College Readiness, College Curriculum Breadth, and Graduation Rate are standardized nationally, while state assessment results were standardized within the state. The standardized scores were weighted, summed, and transformed into a percentile.



thumbs up

 

3 things we like about the U.S. News rankings:

 

1. Performance on state exams factors in the racial/economic background of the students served by the school.

Schools serving a lower percentage of students who are historically underserved (defined as Black/African-American students, Hispanic/Latin students, and students who are eligible for free or reduced-price lunch) typically have higher test scores than schools serving a higher percentage of students from these groups, but the U.S. News ranking takes that into account.

2. Performance of historically underserved populations is considered.

Students who are historically underserved generally score lower on state assessments than white, Asian, and/or economically advantaged students.  Schools where the equity gap between historically underserved and non-underserved groups is smaller than the state average get higher scores.

3. AP passing rates are considered as well as AP participation.

Under Arkansas’ ESSA plan, the number of students taking an AP class is rewarded, but there is no consideration to how well students perform on the AP tests.  This is particularly important because, unlike students in most other states, Arkansas students do not have to pay to take AP tests, so we can consider the passing rates a more reliable measure of how well the AP content is being taught.

thumbs down3 things we don’t like about the rankings:

1. The data are OLD

The data used by U.S. News for the 2019 rankings are from the 2016-17 school year – nearly two years old.  We hope that stakeholders will keep that in mind as they search for their school on the “Best” list.

2. Focus is just on College

Only ‘college ready’ indicators are considered.  We would like to see U.S. News including more indices of career readiness, because not everyone wants to go to college and the ‘best’ high schools should meet the learning goals of all of their students.

3. Focus is on Proficiency, not Growth

Here at OEP, we are strong proponents of student-level growth models.  We understand that it is impossible to compare this type of student growth across states for now, because each state has a different assessment, but we feel it is important to point out that even though they consider the demographics of the students served by the school, Arkansas’ growth model provides better information about how well students are GROWING.



grad cap

Does College-Ready Mean College Going?

With the focus that these rankings place on college-readiness, here at the OEP we felt like a big piece of the success of these schools was not included in the ranking.  We wondered,  “Are students from these ‘Best’ high schools actually going to college?”.

ADE releases the College Going rates for schools through the annual state report card, so we looked up the 2016-17 rates for the top 10 U.S. News schools. (FYI: the 2017-18 rates have been updated since our previous blog). As presented in Table 1, between 25% and 76% of graduates from these high schools went on to enroll in college.

Table 1. College Going Rates for top 10 U.S. News High Schools

ADE_CG_1617

Does this seem weird? Only 60% of students from Haas Hall, the “best” high school in Arkansas, go on to college?  Well, the thing about the college going rates reported by ADE is that it only includes students going to college IN ARKANSAS.  Each year, about 5-6% of Arkansas graduates attend schools out of state.  When we use data (in Table 2) showing where Arkansas students enroll anywhere in the country, we see the college going rates for these high schools range from 53% to 90%.

Table 2. College Going Rates for Top 10 U.S. News High Schools, In-State and National

NSC_CG_1617

The schools identified as the “Best” in Arkansas in 2019 demonstrate high student achievement and lots of opportunities for college- level work. Students in those advanced classes are successful to the AP/ IB exams, and underrepresented populations seem to be performing better than expected. But the methodology used by U.S. News doesn’t tell us if students are going to college after high school, which is, in our opinion, and overlooked measure of a “Best” high school. Other indicators, such as industry certifications earned, completion of a coherent Career and Technical Education sequence, and employment after high school graduation are also indicators of a high school that is doing a great job to prepare students for their future.

Arkansas’ Achievement Gaps: Analysis By Race and Gender

In The View from the OEP on April 24, 2019 at 2:00 pm

This week we continue our examination of long-term achievement for different student groups on the state assessments. Today we are discussing differences in average achievement in math and literacy by gender and race. We think this is important descriptive work for education stakeholders throughout the state, because identifying and acknowledging the differences in achievement is a first step to ameliorating them. 


As we mentioned in our earlier blog about the achievement gap between students participating the Free and Reduced Lunch program (an indicator of low family income), for this analysis we use ten years of the annual Arkansas state assessments, which have changed over time. For the first six years of our analysis we use data from the Benchmark and End-of-Course exams. In the 2014-15 school year, students in Arkansas completed the PARCC exam, before switching to the ACT Aspire exam for 2016-17 and beyond.

All of these test changes have made it essentially impossible for Arkansas schools to examine achievement gaps over time, due to unique test scoring and reporting of each assessment. Although the assessments have varied over time, each provides a measure of how students statewide perform in literacy and mathematics. To enable the examination of performance over time, we have used a common standardizing procedure to track the relative performance of different groups.  We transform these scores into percentiles to aid in interpretation. The statewide average percentile in literacy or mathematics at each grade level is the 50th percentile each year. Note that the percentiles are standardized within year and state, meaning that they are not indicators of how ‘true’ achievement has changed over time or how performance compares to students in other states. Percentiles are used to compare the relative annual performance of Arkansas student groups over time and examine if the gap in achievement has changed (details about the methodology can be found at the end of this blog).

Are there differences in achievement by gender?

Most national studies find that, on average, males outperform females on math tests and females outperform males on reading or English Language Arts (ELA) tests. Recent work by Reardon et al. out of Stanford’s Center for Education Policy Analysis found that the average school district across the county evidenced female-favoring achievement gaps in English Language Arts, but no gender achievement gaps in mathematics. We find similar patterns when we examine the Arkansas assessment data.

Figure 1. Average Literacy Achievement by Gender, 2008-09 through 2017-18

Lit Gender

As can be seen in Figure 1, female students consistently perform above the state average on literacy assessments, and there is a persistent gap of 13-15 percentage points between the average performance of females and males. This difference of 0.33 standard deviations is somewhat larger than the national estimates found by Reardon and represents approximately a year of learning. The difference closed by 50% in 2014-15, the only year that the PARCC assessment was administered. Overall, average literacy achievement by student gender has been consistent over the time examined.

Figure 2. Average Math Achievement by Gender, 2008-09 through 2017-18

Math Gender

In contrast to the differences in literacy achievement, Figure 2 illustrates that Arkansas’ female and male students have performed similarly on the state mathematics assessments since 2009. Female students scored slightly higher than their male peers, and there was no evidence that the gap was different in the year the PARCC assessment was administered.  Overall, average math achievement by student gender has been consistent over the time examined.

Are there differences in achievement by race/ethnicity?

CEPA has also published work on the racial achievement gaps throughout the country.  The researchers find differences in average math and reading achievement by race, and note that one potential explanation for racial achievement gaps is that they are largely due to socioeconomic disparities between white, black, and Hispanic families. The research finds, however, that achievement gaps are still present even in states where the racial socioeconomic disparities are near zero (typically states with small black or Hispanic populations), suggesting that socioeconomic disparities are not the sole cause of racial achievement gaps. Although the CEPA research used data from the National Assessment of Educational Progress (NAEP) as the basis for the research (NAEP), we find similar racial achievement differences using the Arkansas state assessments.

Figure 3. Average Literacy Achievement by Race, 2008-09 through 2017-18

Lit Race

As presented in Figure 3, White students consistently score 7 to 8 percentage points above the state average performance in literacy. The literacy achievement of Hispanic students reflects improvement from an initial percentile of 40, and has been holding steady in the 44-46th %ile range since 2011-12.  Average black student achievement on state literacy assessments is in the low 30th percentiles, beginning at the 31st in 2008-09 and demonstrating a slight increase to 33rd %ile by 2018. The difference between white and Hispanic performance is about a third of a standard deviation, representing approximately a year of learning. The difference between white and black literacy achievement is double that.  Similar to the patterns we have noted for students participating in the FRL program, black student achievement reflects a noticeable relative increase (+7 percentage points) in the 2014-15 school year, which was the year that the PARCC assessment was administered. There is a slight narrowing of the differences in achievement over the past decade, but overall, average literacy achievement by student racial groups has been consistent over the time examined.

Figure 4. Average Math Achievement by Race, 2008-09 through 2017-18

Math Race

Similar to literacy achievement, White students consistently score 7 to 8 percentage points above the state average mathematics performance. Hispanic students’ mathematics achievement reflects improvement from an initial percentile of 43, although there was a slight decline from a high of 48 in the 2013-14 and 2014-15 school years to an average of 46th percentile in 2017-18.  Average black student achievement on state mathematics assessments are in the low 30th percentiles, beginning at the 32nd in 2008-09 and declining to the 31st percentile in 2018. Mathematics differences are similar in magnitude to the literacy gaps. Similar to the pattern we noted for literacy, black students’ math achievement reflects a noticeable relative increase (+8 percentage points)  in the 2014-15 school year, which was the year that the PARCC assessment was administered.


Are there differences in achievement by race/ethnicity and gender combinations?

We were interested to learn if there were differences in average achievement on the state assessment within race by student gender. Based on what we saw in earlier figures, we anticipate that females will perform 15-17 percentage points higher than their male counterparts in literacy, while achievement in mathematics will be essentially the same.   In Figures 5-7, we present literacy achievement by race/ by gender, and in Figures 8-10 we share mathematics results.

Figure 5. Average White Literacy Achievement, by Gender, 2008-09 through 2017-18

White Lit

As expected, Figure 5 illustrates that white females achieve about 15 percentage points above their male peers in literacy, a difference of about a year of learning.  White females score well above the state average in literacy, while their male counterparts achieve just at the 50th percentile, which is average performance for the state average of the 50th percentile.  Interestingly, white female students evidenced the decline in achievement in 2014-15, the year of the PARCC assessment, but white male achievement did not reflect increased achievement that year.

Figure 6. Average Hispanic Literacy Achievement, by Gender, 2008-09 through 2017-18

Hispanic Lit

Similar to white females, Hispanic females demonstrate higher literacy achievement than their male peers in the state by around 13-14 percentage points, representing approximately a year of learning.  Hispanic females performed slightly above about the state average in literacy in 2018, up from 4 percentage points below the average in 2009.  The average literacy achievement of Hispanic males have also increased since 2008-09, leading to a slight closing the difference in achievement with Hispanic females. Unlike their white peers, Hispanic males evidenced increased relative achievement on the PARCC test in 2014-15.

Figure 7. Average Black Literacy Achievement, by Gender, 2008-09 through 2017-18

Black Lit

As is other racial groups, black females demonstrate higher average achievement in literacy than their male peers. Figure 7 illustrates that black females achieve about 15 percentage points above the achievement of black males, which represents approximately a year of learning. With average literacy at the 49th percentile in 2018, however, the average achievement of black females is lower than females in other racial groups. Black males are the lowest relative achievers in literacy – consistently scoring at the 14th-26th percentile since 2008-09 with the exception of a sharp increase of 12 percentage points on the PARCC assessment in 2014-15.  Unlike females in other racial groups, who experience slight decline in 2014-15, black females evidence a score increase of 3 percentage points on the PARCC exam.

Figure 8. Average White Math Achievement, by Gender, 2008-09 through 2017-18

White Math

As anticipated based on the overall similar performance of female and male students in mathematics, Figure 8 represents how white females narrowly out perform their male counterparts on the state mathematics assessments.  Achievement of both white females and white males is above the state average mathematics achievement and has been consistent over time.

Figure 9. Average Hispanic Math Achievement, by Gender, 2008-09 through 2017-18

Hispanic Math

Hispanic math achievement by gender is depicted in Figure 9. Similar to their white peers, Hispanic females narrowly outperform their male counterparts on the state mathematics assessments. Math performance for Hispanic females and males is slightly below the state average, and average mathematics performance has increased slightly  over time for both groups.

Figure 10. Average Black Math Achievement, by Gender, 2008-09 through 2017-18

Black Math

Figure 10 reveals that the difference in average mathematics achievement between males and females is larger for black students than for Hispanic and white students. Black females achieve, on average, 5 percentage points higher than black males. Performance for both females and males, however, is well below the state average. The gap is consistent over time, and both females and males evidenced increased relative performance in 2014-15 with the PARCC assessment.

So, what does this tell us?

Literacy: Arkansas state assessment data reflect differences by gender and race in average literacy achievement that are similar to the national achievement differences. Differences in average literacy achievement between racial groups have narrowed slightly over the past decade.  Differences in literacy achievement between females and males is consistent across racial groups.

Math: Arkansas state assessment data reflect essentially no difference in average mathematics achievement by gender overall, although there are differences in average math achievement by racial groups.  Hispanic students, both females and males, are demonstrating increased math achievement, narrowing the difference between their performance and that of white students. Interestingly, black females and males evidence a greater gap in the average math performance that was present for gender difference among white and Hispanic students.

PARCC: We are really interested in what was different about the PARCC assessment that resulted in higher relative achievement in both math and literacy for Black (and poor) students in Arkansas? Remember that scores from all assessments were standardized, and should be comparable across time.  The changes are interesting, but we have just one year of PARCC results, so we cannot interpret beyond just noting the change.  The performance trends present since 2009 returned when student began being assessed on the ACT Aspire, and the consistency of group achievement (apart from the PARCC year) is a good reminder of how important it is to focus on the long-term trends.

Coming out soon: Achievement trends by grade level!  Find out if the differences in achievement start large and get smaller, start small and grow over time, or stubbornly persist same over the decade.


About the methodology:

Students in grades 3-8 were consistently assessed in both content areas, but there was variation in when high school students were assessed. For our analysis we used all grades assessed in literacy and/ or math in a given year, and limited our analyses to students completing the general assessment.

Scale scores were standardized for each year within grade level and content area, creating a Z-score (mean of 0, standard deviation of 1).  These z-scores were then averaged across student groups and transformed into percentiles to ease interpretation. The statewide average percentile in literacy or mathematics at each grade level is the 50th percentile each year. Percentiles are standardized within year and state, meaning that they are not indicators of how actual achievement has changed over time or how performance compares to students in other states. Percentiles are used to compare the relative annual performance of Arkansas student groups over time and examine if the gap in achievement has changed. 

Are Arkansas’ High School Graduates Prepared to Succeed in College?

In The View from the OEP on April 17, 2019 at 1:50 pm

Earlier this week, the Arkansas Department of Education released 2018 Report Cards for schools, districts, and the state (press release). Included in these data was information on the percentage of Arkansas students who graduate from high school, go to college, and earn college credits.

While college is not the right path for every Arkansas student, college degrees are increasingly valued by employers (Harvard Business School Report). Although there has been some recent pushback on employers’ degree requirements, to compete in tomorrow’s job market, Arkansas schools will need to prepare a large proportion of their students to succeed in college. Unfortunately, this is an area where Arkansas has some ground to make up. Only 22 percent of Arkansans age 25 and older have a bachelor’s degree, which is 9 percentage points below the national average (31%) with only Mississippi (21.3%) and West Virginia (19.9%) having  lower degree attainment (Census Bureau map).

Given our degree deficit and the priority the state has put on college readiness, we were eager to see if more of Arkansas’ students were hitting important milestones on the way to college graduation. We have written about this before (here and here), but at the time only three years of data were available. With yesterday’s release, we now have 5 years of data, but unfortunately, the story is not great.

There were, however, some encouraging signs in the data. For example, Arkansas students are graduating high school in greater numbers (see Figure 1). The state’s 4-year graduation rates inched up slightly, climbing to 89 percent in 2018. Minority students also saw increases over this period, with black students experiencing the biggest gains, increasing by 5 percentage points from 81 to 86 percent.

Figure 1: Arkansas 4-Year High School Graduation Rates 2014-2018

Screen Shot 2019-04-17 at 9.32.18 AM

Even though Arkansas’ students are graduating from high school at higher rates, they are not more likely to attend college (see Figure 2). The college going rate remained relatively flat over most of the 5-year period ending in 2018. However, if the recent data is to be trusted, college going rates declined by 8 percentage points in 2018. Such a large drop makes us question the validity of the 2018 data, but if accurate, this would be a huge deal that demands more discussion and monitoring.

***IMPORTANT NOTE:  ADHE confirmed to OEP that the college-going data are not correct and they are re-calculating the values. The initial re-calculation reflects a statewide college going rate consistent with prior years. *** We will update the analyses below with the new college-going data when released, but the general information still applies.  

Figure 2: Arkansas College Going Rate 2014-2018

Screen Shot 2019-04-17 at 9.35.43 AM

One important caveat about the state’s college going data is that the values only reflect students who attend a college in Arkansas. These rates miss Arkansas high school graduates who leave the state to pursue their post-secondary education, and so understate the state’s college going rate. The National Center for Educational Statistics produced a report in 2017 that can help us estimate by how much the rates are understated (see report here). In the fall of 2016, there were 3,318 Arkansas residents attending a college outside of the state. Assuming this number includes students who graduated at some point over the previous four years and dividing by the roughly 30,000 Arkansas high school graduates per year yields an estimate of approximately 3 percent for the percentage of Arkansas high school graduates in any given year attending college out of state. So our best guess as to how much Arkansas’ college going rate is underestimated is somewhere between 2.5 and 5 percentage points. The upshot is that even accounting for underestimation, Arkansas’ high school graduates enroll in college at far lower rates than the national average of 67 percent (https://nces.ed.gov/programs/coe/indicator_cpa.asp).

As we have done in previous blog posts, we can use the state’s data on graduation and college going rates along with the data on college credit completion to estimate how many of today’s 9th graders we can expect to get to college and earn at least one year of credits (figure 3). The picture is not encouraging. Of a group of 100 hypothetical 9th graders, we would expect 89 to graduate high school, 36 to enroll in college within the next year, and only 19 to complete one year’s worth of credits in the subsequent 2 years. So only 1 in 5 of today’s 9th graders would be expected to get to college and start down the path toward degree completion. And the story is worse for the state’s minority students.

Figure 3: Expected Education Attainment for Hypothetical 9th Grade Cohort

Screen Shot 2019-04-17 at 9.40.58 AM

Part of the challenge is that even once kids get to college, 63 percent of Arkansas’ high school graduates require remediation in one or more subjects. Remedial courses do not count toward degree completion, and can make persistence and degree completion seem out of reach.

College admission exams provide another measure of college readiness. All Arkansas high school students now take the ACT, and only 17 percent of 2018’s graduating class met the ACT’s college readiness benchmarks in all four tested subjects (i.e., English, math, reading, and science; Arkansas ACT report). Missouri, which also requires students to take the ACT and is similar to Arkansas in terms of its rural/urban mix, performed significantly better, with 22 percent of 2018 graduates meeting the college readiness benchmarks in all four subjects (Missouri ACT report). You can find a table summarizing states’ 2018 ACT performance here.

Although college readiness has been a priority for many years and the state’s institutions of higher education have been working to improve remediation and college persistence, it’s clear that we are not making fast enough progress. Arkansas is still far behind in college enrollment and degree completion, and we will need to redouble our efforts and try new strategies if we expect to compete on a global stage.

Note: If you want to dig into Arkansas’ graduation rate and college going rate data, the OEP website provides handy spreadsheets that stratify the data by district and school. You can find graduation rate here and college going rate here (the 2018 rate data table will be released after re-calculated data are released by ADE).

Are We Closing the Achievement Gap?

In The View from the OEP on April 10, 2019 at 1:23 pm

ednext_XIX_3_hanushek_img01Yesterday, EducationNext discussed new research demonstrating a persistent 50 year achievement gap between “Haves and Have Nots”, and we wanted to chime in about what the gap looks like for students in Arkansas.

While the national and Arkansas-specific research use different data sources and methodologies, the conclusions are the same: the gap in academic performance between students with fewer and greater economic resources is large and isn’t getting smaller.

The national research used an index of student socioeconomic status (SES) developed from student-reported information about the education level of their parents and possessions that they have in their home. The achievement data come from four testing programs: National Assessment of Educational Progress (NAEP) Long-Term Trend;  Main NAEP; the Trends in International Mathematics and Science Study (TIMSS); and the Program for International Student Assessment (PISA). The researchers found that the gap in achievement between the highest 10% SES and the lowest 10% SES students has remained consistent over the last 50 years.  The conclusion holds when the ranges are expanded to the highest and lowest SES quartiles as well.

What about Arkansas’ Gap?

Here at OEP, we looked into the achievement gap for Arkansas students and found similarly disheartening results.

For the Arkansas analysis, we use the student-level indicator of Free-Reduced Lunch (FRL) program participation as the indicator of student socioeconomic status because we don’t have information on parental education or home possessions. FRL eligibility is based on family income, and is often used as an imperfect proxy for a student’s socioeconomic status. We group students into two groups: Not participating in FRL, or participating in FRL. In 2008-09, the first year of our analysis, 56% of Arkansas students participated in FRL.  The percentage increased to 59% in 2009-10, and has remained fairly consistent since then, ranging as high as 61% before declining slightly to 60% in 2017-18 .

For the achievement data, we use ten years of the annual Arkansas state assessments, which have changed over time. For the first six years of our analysis we use data from the Benchmark and End-of-Course exams. In the 2014-15 school year, students in Arkansas completed the PARCC exam, before switching to the ACT Aspire exam for 2016-17 and beyond.

All of these test changes have made it essentially impossible for Arkansas schools to examine achievement gaps over time, because each test scores and reports performance in different ways. Although the assessments have varied over time, each provides a measure of how students statewide perform in literacy and mathematics. We think it is very important for education stakeholders to be able to examine performance over time, so we used a common standardizing procedure to track the relative performance of different groups.  We transform these scores into percentiles to aid in interpretation. The statewide average percentile in literacy or mathematics at each grade level is the 50th percentile each year. Note that the percentiles are standardized within year and state, meaning that they are not indicators of how ‘true’ achievement has changed over time or how performance compares to students in other states. Percentiles are used to compare the relative annual performance of Arkansas student groups over time and examine if the gap in achievement has changed (details about the methodology can be found at the end of this blog).

We found that in Arkansas, the difference in achievement between “Haves and Have Nots” has remained essentially unchanged over the past decade.

As seen in Figure 1, Arkansas’ literacy achievement gap closed slightly from 25 percentage points to 23 percentage points over the past 10 years. In 2008-09, students participating in the FRL program scored at the 39th percentile in literacy, while students who were not participating scored at the 65th percentile on average.  Over time, the performance on non-participating students remained consistent, while FRL-participating student performance increased slowly over time to the 42nd percentile by 2017-18. The literacy achievement gap closed slightly from 25 percentage points to 23 percentage points over the past 10 years.

FRL LIT

As seen in Figure 2, Arkansas’ mathematics achievement gap increased slightly from 20 percentage points to 23 percentage points over the past 10 years. In 2008-09, students participating in the FRL program scored at the 42nd percentile in mathematics, while students who were not participating scored, on average, at the 62nd percentile.  Student performance fluctuated over time for students in both groups, but the performance of non-FRL participating student increased slightly over time, while the performance of their FRL participating peers remained steady, leading to the slight widening of the achievement gap in mathematics.

FRL Math

It is interesting to note that the achievement gaps in both literacy and mathematics were smallest in the 2014-15 school year, the first (and only) year that Arkansas students completed the PARCC assessment. The gaps closed due to an increase in the average performance of FRL-participating students AND to a decline in the average performance of non-FRL participating students. It is tempting to hypothesize causes behind this decreased gap, but futile because we have only a single data point, and the gap then returns to established levels.

Well Shoot!

As with the national research, we feel these are important things to consider as education/ the level of education attained is key to upward mobility. Both the national and Arkansas results show that certain subgroups are limited in their ability to achieve success academically in comparison to their peers, and that the achievement gaps have persisted over time.

Therefore, it is seen that schools in Arkansas and across the country are not truly meeting the challenge of reducing those disparities among these subgroups. How can that be addressed?

We need to keep digging in order to get to the root of this achievement gap issue. OEP is digging into these gaps more in depth, and will share more information about trends in grade-level achievement gaps, as well as gaps between racial/ethnic and gender groups. We look forward to highlighting the Arkansas schools and districts that HAVE closed the achievement gaps for some student populations.

 


About the methodology:

Students in grades 3-8 were consistently assessed in both content areas, but there was variation in when high school students were assessed. For our analysis we used all grades assessed in literacy and/ or math in a given year, and limited our analyses to students completing the general assessment.

Scale scores were standardized for each year within grade level and content area, creating a Z-score (mean of 0, standard deviation of 1).  These z-scores were then averaged across student groups and transformed into percentiles to ease interpretation. The statewide average percentile in literacy or mathematics at each grade level is the 50th percentile each year. Percentiles are standardized within year and state, meaning that they are not indicators of how actual achievement has changed over time or how performance compares to students in other states. Percentiles are used to compare the relative annual performance of Arkansas student groups over time and examine if the gap in achievement has changed. 

What’s the deal with vouchers?

In The View from the OEP on April 3, 2019 at 2:06 pm

The Arkansas legislature is considering two bills, SB 539 and SB 620, that would make it easier for low-income students to attend private schools. These bills highlight two different approaches for expanding school choice by offering families money to cover a portion of their children’s private school tuition – often referred to as vouchers. Discussions about private school vouchers tend to elicit strong reactions from supporters and opponents alike, making it difficult to wade through the noise and understand what they are, why they exist, what we know about their impact. In this post we hope to help our readers understand the answer to these three questions.

What are vouchers?

 Vouchers are publicly funded grants or scholarships that are used use pay for private school tuition. Vouchers are not new in education policy. As the National Council of State Legislatures’ (NCSL) notes in their summary of vouchers’ history, “state support for private school education has existed in Maine and Vermont for nearly 140 years.” Vouchers have been used in higher education since 1965, offered by the federal government in the form of Pell Grants. However, there has been renewed interest in vouchers since the early 1990s when the first modern K-12 voucher program was established in Milwaukee.

Today twenty-three states offer some form of vouchers for private K-12 education, and they come in three different flavors: 1) tax-credit scholarships, 2) state-funded scholarships, and 3) education savings accounts. Eighteen states offer a tax-credit scholarship, 15 offer a state-funded scholarships, and 6 offer education savings accounts. While program design differs somewhat across states, most of these state-based voucher programs restrict eligibility to either low-income students or students with disabilities (i.e., who have an individual education plan (IEP)). Arkansas currently offers a voucher program called Succeed Scholarship Program to students with disabilities. For more information on state-based voucher programs check out the NCSL Interactive Guide to School Choice Laws and EdChoice’s summary of state choice programs.

Why do vouchers exist?

Vouchers are a public policy tool that can be used to increase families’ schooling options. This is especially true for low-income families who, in the absence of a school choice program, often lack the resources to choose a school different from the one to which they are currently assigned based on their address. Wealthier families, on the other hand, have many more schooling options because they have the resources not only to choose amongst a wider set of neighborhoods but also to send their children to private schools. School choice programs, like vouchers, can help ameliorate the inequity in schooling options, providing low-income parents with the ability to choose the school that best fits their children’ needs.

Some social scientists, most famously Milton Freedman, have also argued that providing more schooling choices in an environment where families have few if any options will increase overall school quality because schools will need to get better to attract and keep students.

What do we know about the impact of vouchers?

Over the years there have been many studies of K-12 education vouchers. Several of these studies are high-quality randomized controlled trials (RCTs) which provide researchers with the best chance of establishing cause and effect. Researchers have studied vouchers in many different locations/contexts and with varying designs (e.g., targeted to low-income families vs not). Despite the amount of high-quality research around vouchers, the story about their impact is equivocal. You can find good summaries of the research in Chalkbeat, Journal of Economic Literature (JEL), and School Choice at a Crossroads ch.4.

The research has studied vouchers’ impact on three main categories of outcomes: 1) voucher students’ achievement on standardized tests, 2) voucher students’ educational attainment (e.g., high school completion, college enrollment, etc.) and longer-term outcomes (e.g., criminal behavior), and 3) the effect on traditional public schools (e.g., achievement, finances, segregation, etc.).

Student Achievement

The results for the impact of vouchers on student achievement are mixed. While several older studies consistently found either no effect or a slight positive effect on achievement, four recent studies have found negative effects and one has found positive effects (see Figure 1 below, omits North Carolina results). Research design may have played a role in the studies’ findings – many of the older studies are RCTs, giving us significant confidence in their results; however, two of the recent studies are as well. The differences in the context and design of the voucher programs are likely more explanatory. The more recent studies evaluated programs that are less tightly targeted to low-income families and students with disabilities and, in some cases, included significant regulation that limited the supply of private options. Another explanation for the mixed results could be that the local public schools used for comparison in the recent research have gotten pretty good at delivering results on state tests. Regardless, while the overarching impact of vouchers on student achievement is not completely clear, it’s likely that, even when well designed, their impact is small to neutral and that the design and context of the voucher program matters a lot.

Figure 1: Math and Reading Test Score Impacts from 12 Voucher Studies

Reproduced from the Brookings Institution Evidence Speaks Reports, Vol 2, #18.

The x-axis is the estimated impact in standard deviation units. For each study, the dot represents the estimated mean impact and the bars represent the 95 percent confidence interval around the mean.

Screen Shot 2019-04-03 at 9.57.10 AM

Attainment and Longer-Term Outcomes

The research story for educational attainment and longer-term outcomes is more positive, although the evidence is not very strong. Several studies have found that vouchers increase high-school graduation and college enrollment. There is, however, mixed evidence on college completion with one study finding a positive impact and another finding none. A recent working paper produced by EDRE’s own Cory DeAngelis and Pat Wolf found that the Milwaukee voucher program reduced criminal activity and paternity suits among participants in adulthood. These results on longer-term outcomes are promising, but more evidence is needed to verify these initial findings.

Effect on Traditional Public Schools

Opponents of vouchers often claim that they harm traditional public schools by lowering the achievement of those left behind, decreasing resources, and increasing segregation. However, we are not aware of any rigorous evidence documenting these harms occurring in real-world programs. In fact, there is significant evidence that voucher programs improve traditional public school performance. As the JEL research summary linked above put it, “Evidence on both small-scale and large- scale programs suggests that competition induced by vouchers leads public schools to improve.” Likewise, there is significant evidence that voucher programs are either cost neutral or save the state and districts money. The evidence on vouchers segregation is not as robust, but does suggest that programs that target low-income families do not have a racially segregative effect (e.g., Louisiana, Milwaukee). However, just because these harms have not been documented in existing programs does not mean that they can be ignored. Policymakers must be aware of the potential harms and do their best to mitigate these concerns when they design voucher programs.

What does that mean for Arkansas?

As noted in the intro, the Arkansas legislature is considering two voucher bills.

SB 620 would create a pilot, state-funded scholarship program for low-income Pulaski County students. This bill is currently in the Senate education committee.

SB 539 would offer tax credits to individuals and corporations who make donations to a fund that will provide scholarships for low-income students to attend private schools. This bill passed out of the Senate a few days ago and is now set to be considered in the House. Seventeen other states already offer similar tax-credit scholarship programs (visit EdChoice for more info on these programs). Our colleague Julie Trivett has produced an analysis of the potential fiscal impact of SB 539, and found that under conservative assumptions the proposed tax-credit scholarship program will be effectively cost neutral.

The bills strictly limit eligibility to low-income families and cap the resources available so that the programs start small, which are both features of programs that have demonstrated positive impacts in past research. The bills also require participants to take a norm-referenced standardized test and schools to report results. Given the research evidence on private school vouchers, we see no compelling reason for policymakers not to consider piloting the proposed voucher programs. While the proposed programs are unlikely to yield large positive results or cost savings, they will increase low-income families’ schooling options and may lead to small improvements for participants and traditional public schools alike. If either of the bills are passed, the legislature should include a requirement that the programs be rigorously evaluated so that we learn from the experience.

No time for field trips?

In The View from the OEP on March 13, 2019 at 11:38 am
process_th

Photo source:  Crystal Bridges Museum of Art

As Arkansas schools enter the final month before state testing, teachers may be focusing instructional time on test prep, foregoing other ‘non-tested’ subjects and activities, but new research finds that students who attended art-related field trips demonstrated increased engagement in school, higher levels of social-emotional skills, and, unexpectedly, higher scores on standardized tests!

The study is a longitudinal, randomized controlled trial, the gold standard for research.    Conducted by Jay Greene, distinguished Professor and head of the Department of Education Reform in the College of Education and Health Professions,  and members of the University of Arkansas National Endowment for the Arts Research Lab that he directs, the study randomly assigned fourth and fifth grade public school students in Atlanta, Georgia to attend three field trips throughout a school year. Students went to an art museum, a live theater production, and a symphony performance at the Woodruff Arts Center. A control group of students within the same schools did not attend the field trips.

You might not think that attending three field trips would lead to measurable, positive outcomes for students, but it did! You can read the working paper on the social emotional effects by lead author Angela Watson here, and the paper by lead author Heidi Holmes Erickson that addresses students’ engagement and academic outcomes here,  but here’s the highlights:

  • Students who were randomly selected to attend the field trips showed significantly higher levels of social-perspective taking through survey items like, “How often do you attempt to understand your friends better by trying to figure out what they
    are thinking?” and “When you are angry at someone, how often do you try to ‘put yourself in his or her shoes?”.  (Effects reflect the limited sample of students with higher academic performance who were more likely to be able to read and interpret the questions)
  • Students who were randomly selected to attend the field trips showed significantly higher levels of tolerance through the survey item, “I think people can have different opinions about the same thing.”
  • Students who were randomly selected to attend field trips reported more positive school engagement. They were less likely to agree that ‘school is boring’, and they had fewer disciplinary infractions in middle school than their control group peers.
  • Female students who were randomly selected to attend field trips were less careless in their survey answering, a measure of conscientiousness.  Female students in the second year of field trips demonstrate even greater levels of conscientiousness, while female students who are not included in a second year of field trips exhibit the same level of conscientiousness as female students who never attended one of the field trips.

Researchers also examined academic outcomes, hypothesizing that there would be no differences in standardized test scores between the students who attended the field trips and those who did not, as three days away from traditional classroom instruction was unlikely to affect students’ academic performance on math or reading exams one way or the other.

  • BUT- students who were randomly selected to attend field trips performed significantly better on their end of year standardized tests in math and English Language Arts than students in the control group.

Greene and his research team are continuing the research with another group of students, and will learn more about the students’ long-term outcomes as they observe them through middle school, high school, and beyond.

In the meantime, here at OEP we think that schools should consider the importance of field trips and arts experiences in a well-rounded education.