University of Arkansas Office for Education Policy

Author Archive

K-2 Assessment? Take your pick…

In The View from the OEP on February 19, 2020 at 3:40 pm

istationnweaRen

Districts are again being given the opportunity to select an assessment to administer to their students in Kindergarten through 2nd grade.  Districts initially selected a K-2 assessment in the spring of 2016, and have been using their selection for three years. This spring, districts are again being given the opportunity to choose a K-2 assessment that they will administer for the next four years.

We know that district leaders and teachers want to make the best choice to support student learning, so we did some digging into the relationship between student outcomes and which assessment was selected by each district.

We needed to use 3rd grade assessments to try to understand any relationship between the selected assessments and student outcomes, because we do not have a consistent assessment in earlier grades. Third grade data include two years of Pre- K-2 assessment and two years of Post-K-2 assessment. We use the terms “Pre” and “Post” terms relative to 3rd graders’ experience. Students who were 3rd graders in 2015-16 and 2016-17 were not exposed to the selected K-2 vendor. In 2015-16 the vendor had not been selected, and in 2016-17, the assessments were implemented in K-2 but the 3rd grade students had not used the assessment in 2nd grade the prior year.  Students who were in 3rd grade in 2017-18, however, had participated in the K-2 vendor assessment when they were in 2nd grade, and 3rd graders in 2018-19 had participated in both first and second grades.

You can read all about it in the policy brief, but here’s a quick summary of what we found:

  • The three K-2 assessments (Istation, NWEA, and Renaissance) were relatively equally selected by districts throughout the state.
  • The geographic and demographic characteristics of the districts that selected each assessment were similar.
  • Academic proficiency in 3rd grade is similar between the districts that selected different K-2 assessments.
  • There is no statistically significant difference in ACT Aspire 3rd grade growth scores between districts that selected different K-2 assessments.
  • Schools using NWEA: MAP evidenced significantly greater growth scores in ELA, although the effect was not present in the district-level analyses.
  • There are very high growth schools and districts using each of the K-2 assessments.

Although this is not a causal analysis, we can detect no relationship between district-level academic growth of 3rd grade students in Math and ELA, and the K-2 assessment selected by the districts. Interestingly, we do find a positive relationship at the school level between ELA growth and districts that selected NWEA: MAP.  This is likely due to the fact that large districts with multiple elementary schools all use the same assessment but some schools have more positive growth than others.  The difference in growth may be capturing the fact that schools which are more effective at ELA instruction are choosing to use NWEA, or that school implementation of NWEA is positively benefitting students in some ELA classes.

Given the variation in growth scores among districts and schools that selected the same assessment, it is important to point out that which assessment that is selected does not seem to be related to student outcomes.  Likely, it is how students and teachers act on the information gathered from the assessments, and what learning opportunities are present in the classroom daily, that results in better learning outcomes for students.

 

Year-Round Schools in Arkansas

In The View from the OEP on February 12, 2020 at 10:07 am

Continuous learning schools in the Fayetteville Public School District have drawn media attention as they consider returning to a traditional school calendar. Asbell Elementary and Owl Creek currently operate on continuous calendars but, pending a school board vote later this month, may switch back to traditional calendars for the next school year. Today we explore the history of these alternative school calendars, and the pros and cons for students, teachers, and families.

Continuous learning schools, also known as year-round schools, incorporate several shorter breaks throughout the school year and a shorter break during the summer. Despite the ‘year-round’ title, students in these schools typically attend the same number of school days as students in schools on a traditional calendar. In Fayetteville, all schools start the same week in August, but continuous calendar schools end two weeks later than other schools in June. The continuous learning schools have two week-long breaks that are different from the traditional calendar – one the first week of October and another during April. As parents and the superintendent push for a return to the traditional calendar it is important to consider why year-round calendars came to exist and what they contribute to the educational landscape.

Brief History of Year-Round Schools

First attempted in the 1980s, year-round schools were created to push back on what was considered an antiquated school calendar based on economic, rather than educational, considerations. Although there is some debate, consensus says the traditional school calendar originated as a result of the need for students in rural areas to return to the fields for work during the summer months. Continuous learning calendars may be implemented to reduce over-crowding in schools or to improve student outcomes. Continuous learning schools can alleviate crowding in large schools where multi-track calendars allow different student groups to attend school at different times, alleviating space constraints. Advocates of continuous schooling suggest the shorter, more frequent breaks in learning could reduce learning loss between school years. They cited evidence from studies, which demonstrated that students experience a “summer slide” in which they lose knowledge and skills during the long break in schooling from June to September. This loss is especially pronounced for children from low-income backgrounds.

The increased frequency of breaks on a continuous learning calendar was also thought to provide non-academic benefits for students, teachers, and parents. Student-learning fatigue and teacher burnout could be reduced though the alternative calendar. Families may enjoy taking vacations when fewer families are traveling, and may avoid some childcare expenses if the school offers a no-cost intersession opportunity for students to participate in learning opportunities at the school over the breaks.

As a result of these hypothesized benefits, states around the country have implemented continuous-learning schools at varying scales. Research, however, demonstrates that switching to a year-round calendar has little effect on student achievement and may even be harmful in certain circumstances (McMullen & Rouse, 2012; Graves, 2010). Despite intersession programming designed to provide remediation and enrichment to students, the hoped-for benefit of continuous schooling to student-learning outcomes has proved insubstantial. The benefit to teachers is questionable as well, as some report enjoying the more frequent breaks while others are nostalgic for a longer respite from the demands of the classroom.

Parents who have multiple children of different ages generate the greatest pushback against the continuous learning calendar. Year-round calendars are frequently implemented at the elementary school level as schools serving older students struggle to accommodate extracurricular practice and game schedules on the alternative calendar. Since continuous calendars aren’t offered comprehensively across districts, parents with students in varying grade levels must juggle multiple breaks and calendars that do not align. This negates potential benefits of shorter breaks and can leave families feeling frustrated.

In addition, year-round schools may increase costs for districts due to increased transportation and operational costs associated with longer calendars and the lack of overlap with other schools. Coupled with the lack of evidence that continuous learning benefits academic achievement, a calendar change can be a hard sell when not implemented district-wide.

Year-Round Schools in Arkansas

Arkansas has had year-round schools since 1993 when Texarkana converted to a continuous calendar (Fritts-Scott, 2005). Data from the Arkansas Department of Education dating back to 2004-05 shows a decline in the number of year-round schools across the state since the early 2000s when as many as ten schools in nine different districts operated on continuous calendars. Between 2005 and 2008, Little Rock and Pulaski Special School Districts led the state in the number of alternative-calendar schools, but they have not operated one since the 2007-08 school year.

The mid-2000s saw an increase in the number of year-round schools that operated in Northwest Arkansas districts, but the number is now declining. Rogers and Bentonville each operated two continuous learning schools but have since returned them to a traditional calendar. Bentonville converted its schools to a traditional calendar in 2016 and Rogers switched the one remaining school in 2019-20. Fayetteville is unique as it is the only district in Northwest Arkansas that increased the number of schools offering year-round calendars in the last five years. Happy Hollow became a continuous learning school in 1996, and was joined by Asbell in 2008-09 and Owl Creek in 2014-15.

Asbell and Owl Creek may return to a traditional calendar next year, pending the vote by the school board next week. Attendance issues and low turnout to intersession activities are cited as impetus for the change. Upon investigation however, the average daily attendance of both Happy Hollow and Owl Creek has increased and shows no variation across quarters, and while Asbell’s attendance declined between 2013 and 2016 it has been increasing again since 2017. These changes in attendance rates, however, are likely due to a variety of factors and may not necessarily be due to the year-round calendar. Surveys from Owl Creek and Asbell reflect that 66% of school staff and 50-55% of parents support returning the schools to the traditional calendar.

Some Arkansas schools, however, are switching to a continuous learning calendar. Arkansas Arts Academy, an open-enrollment charter school in Northwest Arkansas, switched to a continuous calendar in 2017-18. In addition, Magazine School District converted both its high school and elementary school to a continuous calendar in the 2018-19 school year. As opposed to the Fayetteville calendar, which has the continuous learning schools ending later, the Magazine school calendar will start two weeks earlier in August and end at the same time as previous years. Magazine will provide an interesting case study for continual schooling in Arkansas since the change was district- wide. Due to the comprehensive nature of the change, the results in terms of attendance, student achievement, and parent and teacher satisfaction will be easier to gauge and might provide more insights into the value of year-round schooling for Arkansas students.

Since the quantifiable effect of continuous schooling is ambiguous at best, it is up to the stakeholders in each district to make decisions about what calendar meets the needs of their students.

Examining NWA Charter Schools Enrollment Trends

In The View from the OEP on February 5, 2020 at 2:04 pm

This month, open-enrollment charter schools throughout the state will hold public, random lotteries for students hoping to attend the schools in the 2020-21 school year.  Open-enrollment charter schools are public schools that are open to students regardless of their residentially-assigned traditional school district. Charter schools receive their charter from the Department of Elementary and Secondary Education, which holds these schools accountable to certain standards in order to stay open. They are publicly-funded and free of tuition.

Northwest Arkansas is currently home to nine public open-enrollment charter schools, with plans to open a new charter school for the 2020-21 school year. These schools, which serve unique missions, are some of the most highly ranked schools in the State of Arkansas. While critics argue that public charter schools segregate based on race or academic ability, national evidence finds that these claims are highly context specific. In today’s blog (and associated Policy Brief and Arkansas Education Report) we present what conclusions can we draw about Northwest Arkansas charter schools based on enrollment trends in recent years.

Similar to our previous work examining charter school enrollment trends in Little Rock, we begin by examining traditional and charter enrollment trends by student demographics and end with analyzing the academic performance of students that switch between traditional and charter sectors.

Arkansas Arts Academy schools provide an arts-based approach to learning. Northwest Arkansas Classical Academy schools have a classical focus, including the Socratic method and instruction in Latin. Each of the four Haas Hall campuses emphasize preparation for higher education with a semester block schedule. LISA Academy offers a Science, Technology, Engineering, and Mathematics (STEM) curriculum. Finally, Hope Academy, which will open for the 2020-21 school year, will focus on serving children who have experienced trauma.

Charter schools in Northwest Arkansas enrolled 2,581 students in 2017-18, which was just under 3% of the nearly 90,000 public school students in Benton and Washington counties. Figure 1 presents Northwest Arkansas charter school enrollment from 2007-08 through the 2017-18 school year.

charter1

The 2017-18 enrollment data presented in Table 1, shows that charter schools in Northwest Arkansas enroll a larger proportion of White, Asian, and multi-racial students, than the traditional public districts.  The charter schools enroll a smaller proportion of other ethnic groups, students eligible for free– and reduced-priced lunch, English learners, and students eligible for special education services.

NWA Charter 2

When we examine ten years of enrollment data, we see that all Northwest Arkansas schools are becoming increasingly racially/ ethnically diverse and that charter schools are growing more similar to district public schools in their race/ ethnicity demographic composition. In 2007-08, less than 35% of students enrolled in NWA traditional public schools and around 10% enrolled in NWA charter schools identified with a minority group. In 2017-18, over 40% of the traditional public school population and 30% of the charter school population identified as a minority race or ethnicity. The district-charter minority enrollment gap was nearly 25 percentage points in 2007-08, but had shrunk to just over 10 percentage points ten years later.

Between 2009-10 and 2016-17, approximately 50% of students enrolled in traditional public schools were FRL-eligible. In contrast, approximately 20% of students enrolled in public charter schools were FRL-eligible. Similar disparities persisted for EL students (around 20% of traditional public school students and 3% of charter students) and SPED students (around 6% traditional public school and 3% charter). These trends raise the question of why NWA charter schools have become more integrated based on race, but not for FRL, EL, and SPED students?

Public charter schools are often accused of “cream skimming” (enrolling higher proportions of high-performing students) and “cropping” (encouraging low-performing students to enroll elsewhere). Do we see evidence of this with Northwest Arkansas public charter schools? In an effort to answer this question, we examine the academic performance of students who switched between the traditional and charter school sectors.

Students who exit NWA traditional public schools to enroll in a NWA charter school are, on average, academically high performing.  They scored two-thirds of a standard deviation above the state average on state assessments, and one-third of a standard deviation above the school average of the school that they moved from. The traditional public schools that students are leaving to go to a charter are high performing schools as well. Almost 58% of students who exit NWA traditional public schools to enroll in a NWA charter school left a school with a Z score in the top third of all NWA public schools.

Students leaving NWA charter schools to enroll in a NWA traditional school are also academically high performing on average. They scored on third of a standard deviation above the state average on state assessments. They were average performers, however, for the charter school that they exited.  About 40% of the students exiting charters left a school in the top third of all NWA public schools in terms of student achievement.

Taken together, this evidence suggests that higher performing students are leaving traditional schools to attend charter schools.  We have no evidence WHY higher performing students are leaving traditional schools, but possible reasons might be that they are attached to curricular options, changes in peer groups, or smaller classes. On the other hand, we do not see evidence that the students exiting charter schools are being ‘pushed out’ for low academic performance as they are average academic performers compared to their peers at the charter school that they are exiting.

The charter sector in NWA has grown rapidly over the past ten years, but continues to serve a small proportion (3%) of all public school students in the area. The region has grown more racially and ethnically diverse in that time. Public charter schools have also grown more diverse, though they continue to enroll a smaller proportion of certain student populations. Here’s what we think are important steps moving forward:

  1. Continue to monitor differences in demographic enrollment trends by sector.  Charter schools should be reaching out to all communities to communicate the opportunity to enroll, and if particular groups are not expressing interest we should try to learn more about why.  Do they feel that they are not ‘the right kind of applicant’ or do they prefer the opportunities that they are given in the traditional public sector?
  2. Gain a better understanding of why FRL, EL, and SPED students enroll in charter schools at such low rates. These enrollment trends may be related to problems with practical solutions, such as transportation. It may be that the families of these students are satisfied with services provided at their residentially-assigned district public school. It may be that students are interested in attending but are not being selected in the random lotteries that charters must hold if oversubscribed. Understanding the reasons for these enrollment trends is essential to crafting policy-relevant solutions.
  3. Respond to market demands. NWA charter schools enroll only 3% of all public school students in Benton and Washington counties. However, many of the charter schools are oversubscribed with waiting lists of students not selected through random lotteries.  The interest in charters suggests many more students may be interested in enrolling in these schools. Traditional public schools should communicate with students and parents to determine if their needs are being met, and, if not, how they can better support their educational experience.

 

 

 

 

District Funding Equity

In The View from the OEP on January 29, 2020 at 2:04 pm

Over the past few months we’ve written several posts about school spending. Most recently, we showed that there appears to be essentially no relationship between spending and test score growth. Of course the aim of education is broader than just test score growth, but increasing knowledge and skills in core subjects, as demonstrated on tests, is certainly an important outcome. However, given the broader purpose of education and people’s general desire for fairness, funding equity has also been a longstanding education policy issue.

For Arkansas, funding equity has had particular importance since the Lake View school district filed its original court case in 1992 alleging that the state’s funding system was inequitable and inadequate. The Lake View case eventually made its way to the Arkansas Supreme Court and has since had a profound impact on the state’s approach to school funding. This post looks at school funding equity post-Lake View to see where things stand today.

We used the Arkansas Department of Education’s Annual Statistical Reports to analyze property wealth and school district revenue between 2004 and 2018. All of the dollar amounts presented below are per pupil, meaning we divide property wealth and district revenue figures by the number of students the district serves. Using per pupil figures provides an apples-to-apples comparison for districts of varying sizes.

A key concern in the Lake View case was that affluent districts had significantly more property wealth than poor districts, and could, therefore, generate significantly more funding with the same tax effort. Lake View argued that state and federal dollars were not sufficient to make up for local funding deficits in poorer communities around the state. Below we look at (1) how property wealth and revenue have changed over time, (2) the relationship between revenue per pupil and property wealth, and (3) the relationship between local funding share and property wealth.

Both property values and school district revenue have increased significantly. Between 2004 and 2018, the median assessed property value per pupil for Arkansas school districts increased from $49,803 to $93,301, an 87 percent increase over 14 years. The median is the value where half of districts have higher values and half have lower values. Over the same period, median revenue per pupil increased from $7,191 to $12,112 (a 68 percent increase), and the locally generated share of total district revenue increased from 24 percent to 32 percent. Property values and funding per pupil have both grown at healthy rates, and on average, a greater share of school districts revenue is coming from local property taxes. Next we investigate the relationship between property wealth and revenue.

School district revenue has a weak, positive relationship with property wealth, but overall appears to be relatively equitable. Figure 1 below shows the relationship between property wealth and school district revenue for 2005 (blue dots), 2008 (orange dots), and 2018 (grey dots). Each dot in the figure represents a school district. The data show a weak, positive relationship between revenue and property wealth that has not changed much over time – the dots get slightly higher as property wealth increases from left to right and the different colored dots are similarly distributed. It does not appear from this graph that there are large, systematic inequities built into Arkansas’ school district funding system – the dots are somewhat evenly scattered with only a slight upward tilt from left to right.

Figure 1: The relationship between Arkansas school district revenue per pupil and property wealth between 2005 and 2018

State and Federal funding is being used to equalize school district funding between wealthy and poor communities. Figure 2 shows the relationship between the locally generated share of school district revenue and property wealth. There is a strong positive relationship between locally generated share and property wealth, meaning that wealthier school districts’ taxpayers contribute a greater percentage of school district revenue than do taxpayers in less well-off districts. In other words, state and federal dollars are being heavily allocated toward poorer communities, allowing those school districts to be funded with less reliance on local property wealth.

Figure 2: The relationship between the locally generated share of school district revenue and property wealth between 2005 and 2018

Since Lake View, Arkansas has made many changes to its school funding formula in an effort to improve both adequacy and equity. Our analysis shows that even as property values have grown significantly and a greater share of overall district revenue is coming from local property taxes, Arkansas current funding formula appears to be relatively equitable. There is only a weak, positive relationship between property wealth and revenue, but there is a strong positive relationship between local revenue share and property wealth –  state and federal dollars are being used to mitigate wealth differences across districts. While we still might want to do more to support our poorer communities, it is good news that the equity concerns raised in Lake View seem less apparent today.

Examining Arkansas’ Graduation Rates

In The View from the OEP on January 22, 2020 at 11:06 am

0402_graduation_caps_416x416

Take a minute to think about what drives graduation rates for a high school.

  • Is it student characteristics, such as their academic achievement or their socio-economic status?
  • Is it school characteristics, like how large it is or the community in which it is located?
  • Is it some mixture of both student and school characteristics?
  • Which are related to higher graduation rates and which are related to lower graduation rates?

Here at the OEP, we have taken a deep dive into what student and school characteristics  account for variation in high school graduation rates and are pleased to release our report today! This report examines trends in high school graduation rates for the state of Arkansas across the five-year period of 2013-14 through 2017-18. We consider the relationship between graduation rate and variables of interest including school-level indicators of geographic region, achievement in literacy and math, proportion of racial minority and economically disadvantaged students, graduating class size, and the configuration of the school’s grade levels. Graduation rates are evaluated at the school level for students overall and for students who face economic disadvantages.

Here’s the short summary of what we found:

  • Larger graduating classes are associated with lower graduation rates. Graduating class size is the only consistently significant predictor of high school graduation rates across all five years examined. Larger graduating class size is highly significantly associated with lower graduation rates for students overall as well as for students who face economic disadvantages. See below for why we selected this variable.
  • High schools that begin in 10th or 11th grade are associated with higher overall graduation rates. School grade configuration is a significant predictor of graduation rate in 70% of our multivariate analyses. High schools that begin in 10th or 11th grade are associated with higher overall graduation rates, relative to being in a school with only grades 9-12. Although not consistently statistically significant like class size, positive coefficients across all years for both overall and economically disadvantaged students indicate that high schools that begin in 10th or 11th grade are either positively or neutrally related to graduation rates. See below for why we selected this variable.
  • Demographic characteristics of students enrolled in the school are not consistently significantly related to graduation rates when controlling for all variables in our model. The percentage of students participating in the free/reduced lunch program are significantly negatively related to overall graduation rates in only the most recent three years studied, while the percentage of students of minority status is not significantly related to student graduation rate in any of the years examined.
  • Student achievement indicators are not consistently significantly related to graduation rates when controlling for all variables in our model. The relationship between 8th grade (pre-high school) literacy achievement and graduation rates is significant in the first three years studied, but 8th grade math achievement is significantly related to graduation rates only in 2014-15.

High school graduation rates are important for both students and schools.  Graduating from high school opens a door to increased career opportunities and greater lifetime earnings for students. Arkansas education leaders realize that graduating high school is an important milestone for students, so the state includes graduation rate as 15% of a high school’s ESSA accountability score. Developing a better understanding of the relationships between high school graduation rates and student and/or school characteristics may help us think differently about what really drives high school graduation rates.

Although school leaders cannot easily reduce the size of the graduating class or the number of grades enrolled in a high school building, reflection on how they could re-create the benefit of smaller class size experiences through policies like creating smaller pseudo-cohorts of graduates (through ‘houses’, ‘tribes’, ‘teams’, or ‘families’, for example) might lead to increased graduation rates for their students.

This research can help school leaders, policy makers and education stakeholders to examine ways to further increase school graduation rates. The only variable that is consistently associated with graduation rate change is the number of students in the graduating class, although indications of a positive relationship between high schools serving only grades 10-12 or 11-12 and higher graduation rates appear worthy of further study.


Why just study Arkansas?

State-level is the best way to examine graduation rates. Although all states now calculate high school graduation rates the same way (using the Adjusted Cohort Graduation Rate), there can be differences between states in the requirements for graduation.  In addition, school funding can vary across states as well as the profile of a ‘typical’ high school.

Why consider Graduating Class Size?

Earlier research by the Office for Education Policy (2014) found that larger high schools and schools serving more economically-disadvantaged students had lower graduation rates. There has been quite a bit of research into the benefits of smaller schools, but the definition is not uniform, generally using enrollment between 400 and 1200 students as the indicator of “small”. Benefits of small schools are generally ascribed to the fact that students are more likely to be known by school staff and other students, reducing the chances for them to be overlooked when displaying behaviors that might lead to academic difficulties and dropping out.

During the 2017-18 school year in Arkansas, however, 78% of high schools would be considered “small” as they enrolled fewer than 600 students. Further, half of these “small” high schools could be termed “very small” as they enrolled fewer than 300 students.

When considering the root cause of possible benefits of school enrollment, we felt it was critical to consider the grade configuration of high schools in Arkansas.  About half (129) of the state’s high schools begin at a grade lower than 9th grade, with a 7th -12th grade configuration accounting for 85% of this group. The second largest grade configuration group contains the 109 high schools that enroll a traditional 9th -12th grade population. High schools that begin after 9th grade create the smallest grade configuration group, with only 37 schools.

Due to the variation in grades served by these high schools, we elect to use the size of the graduating class as our indicator of school size. High schools that begin prior to 9th grade have fewer students enrolled in the graduating class when compared to the other grade configurations. On average, there were 47 students in the graduating classes of the schools that begin prior to 9th grade, compared to 163 in the 9-12 schools and 238 in the included schools that begin after 9th grade.

Why consider Grade Configuration?

Arkansas has a variety of grade configurations serving high school students.  We felt that there might be a relationship between school grade configuration and high school graduation, particularly since the grade configurations are generally associated with graduating class size.  We also consider the placement of 9th grade, as it is the first year that a student’s academic performance counts toward graduation requirements. We suspect that placement of the 9th grade might make a difference in a student’s likelihood of graduating as attending 9th grade in a familiar school might reduce the stress load on students. In addition, transitioning to a new school for 9th grade might cause a disruption in the learning experience, while transitioning to a new school after 9th grade might result in organizational issues with credit tracking that could lead to students failing to meet graduation requirements.

More Money, More Growth?

In The View from the OEP on December 4, 2019 at 1:32 pm

We’ve been thinking a lot this week about school-level expenditures, and if the expenditures relate to student academic growth. We think that growth is the best reflection of the effect that a school is having on student learning. Spoiler Alert: there is essentially no relationship between how much a school is spending and how much growth is being made by the students enrolled.

As you may remember, last spring the state released school-level expenditures for the first time. This presents an unprecedented opportunity to examine the equity, efficiency, and efficacy of Arkansas’ public education spending.

We have examined school-level expenditures through a variety of lenses in previous posts, and have found that Arkansas’ schools spend more the higher a school’s poverty level, that overall traditional school and charter school expenditures per pupil are quite similar, and that Arkansas spends the most on high schools and the least on middle schools, with elementary school spending falling in between.

We also have posted a lot about growth!  You probably know by now that growth is much less correlated to poverty at the school level than achievement, students in schools with larger average class sizes demonstrated greater academic growth than their peers in smaller classes, and growth doesn’t have the intended impact on school level grades.

In this post we are digging into a big question- is more spending related to higher student growth?

In this analysis, we are considering 2017-18 school-level per pupil expenditures and the 2017-18 school content growth score. We are using 2017-18 data since the 2018-19 expenditure data haven’t been released yet. Please note that we exclude the high school level Alternative Learning Environments (ALEs) from this analysis due to the small and specialized populations that they serve.

School expenditures are reported in various categories.  We start with the personnel expenditures as the majority of school funds are used to pay for teacher salaries and benefits. The data presented in Figure 1 show that there is not a strong relationship between school instructional spending and the academic growth of students at the school.  Some schools are represented in the upper left quadrant, spending more than average per student on personnel, while students demonstrate lower than expected growth.  Other schools are represented in the lower right quadrant, where students demonstrate higher than expected growth, even though the school is spending less than average per student on personnel.

Figure 1: Scatter Plot of 2017-18 School-Level Personnel Expenditures Per Pupil vs. Content Growth

Personnel CG

We also considered the relationship between student academic growth and instructional spending. As presented in Figure 2, we see the same lack of relationship between expenditures and student growth.

Figure 2: Scatter Plot of 2017-18 School-Level Instructional Expenditures Per Pupil vs. Content Growth

Inst CG

 

Finally, we expanded our analysis to total school expenditure, because we think maybe all the dollars spent could have an impact on student learning. Once again, there is a lack of a relationship, as shown in Figure 3.

Figure 3: Scatter Plot of 2017-18 School-Level Total Spending Per Pupil vs. Content Growth

total CG

 

Does the relationship vary by school level?

We might think that greater expenditures in elementary schools lead to larger academic gains for students than investments in later grade levels. We previously found that expenditure varies between school levels (elementary, middle, and high). Content growth should not vary by school level, as average growth at each grade level is 80. Below, we present the figures for Elementary, Middle, and High Schools, respectively.

Figure 4: Scatter Plot of 2017-18 School-Level Total Spending Per Pupil vs. Content Growth, Elementary School Level

Elem CG

Figure 5: Scatter Plot of 2017-18 School-Level Total Spending Per Pupil vs. Content Growth, Middle School Level

Mid CG

Figure 6: Scatter Plot of 2017-18 School-Level Total Spending Per Pupil vs. Content Growth, High School Level

High CG

At all school levels we see essentially no relationship between the amount spent per pupil and the academic growth of students.

Does this mean that money doesn’t matter to student growth?

Nope! This analysis is descriptive, not causal, meaning that we are just describing the relationship between two variables, as opposed to claiming that changes in one will (or will not) lead to changes in the other. What our descriptive analyses do shed light on, however, is that schools spending the same amount per pupil can realize very different growth outcomes for students.  This lack of a direct relationship between expenditures and growth could indicate that it is more about HOW schools are using their resources (including time, money, and people) than how many resources they have. We look forward to seeing the 2018-19 school expenditure data and checking to see if the (lack of) relationship still exists. What do you think?

 

Looking for Teachers?

In The View from the OEP on November 20, 2019 at 9:52 am

AR Teachers Logo_horiz PNG

Question: How many clicks does it take for an Arkansas teacher to find a great job?

Answer: Way too many.

Currently, for a teacher to apply for a job in an Arkansas public school they have to:

  1. know the name of a school district where they might be interested in working
  2. find the district website
  3. locate where jobs are posted on the website
  4. filter through job postings for bus drivers, food service personnel, and other positions, to see if there is one that matches their credentials
  5. either create an online account and answer a bunch of questions or, perhaps, download a paper application to fill out and send in
  6. repeat for each school district

While researching Arkansas’ teacher shortage, we have learned that many school districts were having difficulty in attracting qualified teachers (no surprise). We also have discovered that job postings could be difficult for applicants to locate, and that the application process could be complicated and time consuming. District job postings were uninspiring, and generally neglected to mention why teachers should want to work in the district or any incentives intended to entice teachers to the district.

So we decided to try to make the process easier for teachers and districts, and have been working on a “common application” for Arkansas teachers.  ARteachers.org is the free resource designed to make it easier for teachers to find great jobs, and for school districts to find great teachers.

  • ARteachers.org makes it easier for teachers to find district job postings. Once a district posts a position, it will automatically appear to qualified teachers that have created a profile on the site.  Teachers can filter the jobs by distance or other characteristics.
  • ARteachers.org allows districts to pro-actively recruit teachers. Districts have access to a pool of teachers looking for jobs including those interested in long-term substitute opportunities.
  • ARteachers.org makes the job-application process easier for teachers by automatically identifying jobs that match their qualifications, and streamlining the application process with a common application format.
  • ARteachers.org helps teacher applicants be better informed by providing an estimated salary for each position based specifically on their education/experience as well as the districts’ student/teacher ratio.

We know that you already have a process for hiring teachers, but hope that you will add ARteachers.org to your recruitment plan.  In addition to connecting teachers and districts, the site will generate information about how many teaching positions need to be filled and the types of applicants that districts are looking for. This information will allow Arkansas’ policy makers and teacher-preparation programs to better understand the need for educators around the state. The more districts that participate, the better the information we can use as feedback.

This is a soft launch of the system before the full hiring surge later in the Spring. We will be working with educator preparation programs to sign up prospective teachers, allowing districts to pro-actively recruit teachers, like how colleges reach out to high school students and invite them to apply.

ARteachers.org was designed specifically for Arkansas to meet the needs of Arkansas teachers and public school districts. There’s not a lot of bells and whistles- and no one is making any money off of the process.  Our goal was to make the system as simple as possible for both teachers and districts.


Have an open position in your district, or looking for a long-term sub for the spring?  To get started, just ask your HR staff to go to ARteachers.org and sign up. The superintendent will get an email asking to confirm that the HR person is authorized to post jobs for the district, then the district will be on its way to finding the best teachers for their students.

Know a teacher that is or will be looking for a position? Have them go to ARteachers.org and sign up. It just takes a few minutes to create a profile and it is free!

We know that getting a great teacher in every classroom is a critical step in ensuring a quality education for Arkansas students, and ARteachers.org makes it easy and free.

AR Teachers Logo_ state and apple only png

 

 

Which Schools are “Beating the Odds?”

In The View from the OEP on November 13, 2019 at 1:06 pm

Today we share our final OEP awards, and discuss how (and why) our awards are different from the rewards given last Friday by the state.

We are so excited to release our “Beating the Odds” Outstanding Educational Performance Awards  for 2019!  These special OEP awards are for schools whose students are demonstrating high academic growth despite serving a population where at least 66% of the students participate in the Free/ Reduced Lunch Program, which is based on low household income.  Schools serving such student populations often struggle to demonstrate high academic achievement, and subsequently receive lower letter grades.

Academic growth, however, is less correlated with school poverty rates and we think it is a better reflection of how the school is impacting students. Growth is calculated at the student level, and essentially reflects how much a student has improved his or her score from the prior year compared to what was predicted based on prior achievement history. While poverty can negatively impact student success, the schools awarded today demonstrate that their students are “Beating the Odds”  The highlights are below, and you can read the full report here.

The OEP Awards highlight schools in Arkansas based on student growth on the ACT Aspire exams in Mathematics and English Language Arts (ELA). We choose to give OEP Awards based on student growth because we think it is the best indicator of how the school is impacting students’ learning.

Although school-level growth scores are much less related to the percentage of students at a school who are participating in Free/Reduced Lunch than achievement scores, a negative correlation does exist (-0.21).  This means that students at schools serving higher poverty populations are more likely than their peers at more affluent schools to demonstrate less academic growth than predicted. As can be seen in the scatter plot below, schools with higher FRL rates are more likely to receive lower growth scores.

Figure 1. Combined Content Growth Score by School % FRL, Arkansas Public Schools, 2019

2019 Scatter Plot

If we limit the plot to only those schools with at least 66% of students participating in FRL, as presented in Figure 2, the relationship between poverty and growth essentially disappears. Although all of these schools are serving high poverty populations, there is wide variation in the amount of academic growth that students at the schools are demonstrating.

Figure 2. Combined Content Growth Score by School % FRL, High-Poverty Arkansas Public Schools, 2019

2019 Poverty Scatter Plot

We celebrate the state using this student-level growth model, and are pleased to be able to highlight how students are growing academically in schools across the state.  We hope that drawing attention to this growth information will spark discussions among stakeholders about the ways to ensure that all schools are growing the knowledge of Arkansas’ students.


“Beating the Odds” Elementary Level Schools

The top “Beating the Odds” elementary school overall is Salem Elementary from Salem School District.  Despite serving a student population that is 67% eligible for Free/Reduced Lunch, Salem Elementary students are among the top 5 schools that have demonstrated the greatest growth in the state on the ACT Aspire. Many of these top 10 Beating the Odds schools were also among the high growth elementary schools in the state, regardless of student demographics. The top 10 elementary schools that are beating the odds are:

  1. Salem Elementary, Salem SD (67% FRL)*
  2. Weiner Elementary, Harrisburg SD (68% FRL)
  3. Oscar Hamilton Elementary, Foreman SD (71% FRL)**
  4. Cross County Elementary Tech Academy, Cross County SD (73% FRL)*
  5. Lamar Elementary, Lamar SD (72% FRL)
  6. Des Arc Elementary, Des Arc SD (75% FRL)
  7. Frank Tillery Elementary, Rogers SD (70% FRL)
  8. Green Forest Elementary, Green Forest SD (86% FRL)**
  9. George Elementary, Springdale SD (86% FRL)
  10. Bismark Elementary, Bismark SD (72% FRL)**

**Schools with two asterisks were included in the top 10 list for the last two years, while *schools with a single asterisk were on the list one of the last two years.

You can find the top BTO elementary schools by subject and region in the full report.


“Beating the Odds” Middle Level Schools

Paragould Junior High from Paragould School District is the top middle school beating the odds overall. Paragould JH serves a 7th-8th grade student population where 71% of students are eligible for Free/Reduced Lunch, and was third among the high growth middle schools in the state, regardless of student demographics.  The top 10 middle schools that are beating the odds are:

  1. Paragould Junior High, Paragould SD (71% FRL)**
  2. Oak Grove Middle, Paragould SD (76% FRL)**
  3. Swifton Middle, Jackson Co. SD (66% FRL)
  4. Helen Tyson Middle, Springdale SD (76% FRL)*
  5. Atkins Middle, Atkins SD (68% FRL)
  6. Cedarville Middle, Cedarville SD (73% FRL)**
  7. Pleasant View Campus, Mulberry/Pleasant View Bi-County Schools (77% FRL)*
  8. Harrisburg Middle, Harrisburg SD (74% FRL)
  9. Yerger Junior High, Hope SD (80% FRL)
  10. Beryl Henry Upper Elementary, Hope SD (89% FRL)**

**Schools with two asterisks were included in the top 10 list for the last two years, while *schools with a single asterisk were on the list one of the last two years.

 

You can find the top BTO middle schools by subject and region in the full report.


“Beating the Odds” High Schools

The top high school beating the odds is Flippin High in Flippin School District.  Despite serving a student population that is 69% eligible for Free/Reduced Lunch, it is also among OEP’s top 10 high growth high schools throughout the state.  Flippin High students are demonstrating that they can achieve at levels similar to students who come from higher income communities. The top 10 high schools that are beating the odds are:

  1. Flippin High, Flippin SD (69% FRL)
  2. Hazen High, Hazen SD (70% FRL)
  3. Maynard High, Maynard SD (69% FRL)*
  4. St. Joe K-12, Ozark Mountain SD (73% FRL)
  5. Barton High, Barton-Lexa SD (85% FRL)
  6. Jasper High, Jasper SD (70% FRL)
  7. Gosnell High, Gosnell (67% FRL)**
  8. Decatur High, Decatur SD (72% FRL)
  9. Cave City High Career & Collegiate Preparatory, Cave City SD (77% FRL)*
  10. Sparkman High, Harmony Grove (Ouachita) SD (73% FRL)

**Schools with two asterisks were included in the top 10 list for the last two years, while *schools with a single asterisk were on the list one of the last two years.

You can find the top BTO high schools by subject and region in the full report.

Congratulations to all the OEP “Beating the Odds” award winners!  Keep up the great work and we look forward to recognizing you again next year!


How are OEP awards different from the state rewards that were announced last week?

1) Part of the state rewards go to high-achieving schools, where a lot of students scored well on the state tests. These schools tend to serve a lower population of students facing academic risk factors poverty or second language acquisition.

OEP only awards schools for growth, because we think that it is a better reflection of how the school is impacting students.

2) The part of the state rewards that are awarded for growth use a different measure than the OEP awards.  The rewards program uses the growth value that also includes the progress being made in English language proficiency, a value called the combined value-added growth score. The difference between the values is inconsistent, with the content growth value higher for some schools and the combined value-added value higher for other schools.

OEP awards are based on improvement in the content areas of Mathematics and English Language Arts (ELA) assessments only. 

3) The state rewards program for growth includes graduation rate for high schools.

OEP awards do not include graduation rate, and are based on improvement in the content areas of Mathematics and English Language Arts (ELA) assessments only. 

4) The state rewards program rewards the top 10% of schools overall.

OEP awards are grouped by ESSA school level (Elementary, Middle, and High). We know that achievement and growth vary by school level and are concerned that middle schools demonstrating relatively high growth are not being rewarded by the state. In fact, only two middle schools were recognized in the top 5% growth/grad awards by the state.  See a further discussion here.

The differences between the state rewards program and OEP awards are due to the fact that the state rewards are legislatively mandated, while here at OEP, we created an awards system that supports our passion for highlighting schools where students demonstrate Oustanding Educational Progress!  Oh, and we don’t send money- just paper certificates!

 

 

Crossing Our Fingers Didn’t Work…

In The View from the OEP on October 31, 2019 at 4:00 pm

NAEP

When the disappointing 2017 NAEP results came out, we said, “We have our fingers crossed that the changes laid out in ESSA will make a big difference to student learning in Arkansas, and look forward to seeing NAEP results again in 2019”.

Well, NAEP results were released yesterday, and Arkansas’ scores look about the same as they did in 2015 and 2017. NAEP is administered nationally to a representative sample of students from all 50 states, so acts as a standard measure of student performance across states and time. We present NAEP performance since 2003, when administration became consistent. We dug into the new results and are pleased to share six NAEP nuggets with you.

NAEP Nugget #1: Not Bouncing Back

Arkansas’ 2019 NAEP scores were essentially unchanged from the past two administrations.  Unfortunately, this is not great news because we were all hoping 2015 and 2017 were just blips that we would bounce back from. In fact, as the figure below highlights, Arkansas scores were the highest in 2011 and 2013, and demonstrate declines since.

Figure 1: Arkansas NAEP Scores, 2003-2019

NAEPF1

NAEP Nugget #2: Not Keeping Up With the Neighbors

4th and 8th grade Math scores are lower than those of Arkansas’ border states (this group includes Louisiana, Mississippi, Missouri, Oklahoma, Tennessee, and Texas).  This is particularly a bummer in 4th grade because we outperformed them from 2005 to 2013, and while the border states’ average scores are rising, Arkansas’ are declining. It’s also a bummer because we seem to be falling farther and farther behind.

Figure 2: 4th grade NAEP Mathematics Scores, Arkansas, Border States, and US, 2003-2019

NAEPF2

Figure 3: 8th grade NAEP Mathematics Scores, Arkansas, Border States, and US, 2003-2019

NAEPF3

NAEP Nugget #3: In the Same Boat in Reading

4th and 8th grade Reading scores are also lower than those of Arkansas’ border states, but we seem to be tracking the national performance pattern. Again, this is particularly a bummer in 4th grade because we outperformed them from 2003 to 2013.

Figure 4: 4th grade NAEP Reading Scores, Arkansas, Border States, and US, 2003-2019

NAEPF4

When it comes to 8th grade reading, scores decreased for Arkansas, our border states, and the nation as a whole. In fact, students in 31 states performed worse in 8th grade reading than they did in 2017.

Figure 5: 8th grade NAEP Reading Scores, Arkansas, Border States, and US, 2003-2019

NAEPF5

NAEP Nugget #4: Widening Gaps for Students in Poverty

Score gaps between Arkansas students eligible for Free/Reduced Lunch and their peers who are not eligible continued to widen in 2019 due to decreased performance of at-risk groups and/ or increased performance of other students.  Gaps in 8th grade mathematics and reading between FRL and Non-FRL students are greater than they have been since 2003.

Figure 6: FRL and Non-FRL Eligible Students’ 8th grade NAEP Mathematics, Arkansas, 2003-2019

NAEPF6

 

NAEP Nugget #5: This is GOOD news! 

Hispanic students are closing the gap to their White peers in both 4th and 8th grade reading! Score gaps between the groups in mathematics remain consistent.

Figure 7: Hispanic and White Students’ 4th grade NAEP Reading, Arkansas, 2003-2019

NAEPF7

Gaps between Black and White student scores remain large and persistent in both content areas at 4th and 8th grade.

NAEP Nugget #6: Mixed Messages

Proficiency rates on NAEP math and reading assessments are lower than on ACT Aspire, Arkansas’ assessment used to measure student achievement. Math proficiency rates on NAEP are about 20 percentage points lower than on ACT Aspire.  Reading proficiency rates were around 30% on NAEP, but 45% of 4th graders and 53% of 8th graders were determined proficient on the ACT Aspire.

Figure 8: Arkansas NAEP Proficiency vs. ACT Aspire Proficiency, Math and Reading, 4th and 8th grade, 2019

NAEPF8

National assessments like NAEP provide a common measuring stick for student performance, which is particularly important as Arkansas is the only state administering ACT Aspire statewide in grades 3-10. We need to pay careful attention to the differences between the NAEP and ACT Aspire proficiency rates. Arkansas will be administering ACT Aspire for at least seven more years, and when we send and receive conflicting messages about how well our students are performing, it can make it difficult to determine how well our students are doing and which sorts of educational interventions are making a difference for our students.

Three measures of Arkansas’ academic achievement (NAEP, ACT Aspire, and ACT) are all essentially flat at the state level over the past 3 to 4 years.

During this time the state has consistently increased per pupil funding, invested in literacy training, re-created school accountability systems, moved to universal ACT testing for juniors, refocused teacher and principal evaluation, and much more. Maybe we haven’t allowed enough time to see the results of the changes we have been making in education.  Or maybe those changes aren’t really having an impact at the school level.  Or maybe the changes ARE working and these measures just aren’t capturing the improvement.

Or maybe we need to step back, and look at school-level growth instead of statewide achievement to see something worth cheering about!

If you haven’t checked out this year’s OEP awards you should!  There is some GOOD news.

 

 

High School Awards: Outstanding Educational Performance

In The View from the OEP on October 30, 2019 at 11:20 am

 

This week, OEP recognizes high schools demonstrating Outstanding Educational Performance. OEP awards are different than other awards because we focus solely on student academic growth.

Earlier this month, the Division of Elementary and Secondary Education released school letter grades for public schools in the state.  We created a statewide data visualization for you to explore the relationships between letter grades, school poverty, and academic growth.

We think school letter grades can be helpful in communicating with stakeholders about school performance, but are concerned that Arkansas’ letter grades send the wrong message because they are driven primarily by student achievement, which is generally reflective of the demographics of the students in the school.  As we discussed previously when awarding elementary and middle schools, the overall score on which the letter grades are based, are strongly correlated with the percentage of students in the school who are participating in the Free/Reduced Lunch program (an indicator of low-income). For High Schools, as illustrated in Figure 1, we find that the negative correlation is even slightly lower than it was for elementary and middle schools (R= -0.66), likely dure to the inclusion of the graduation rate in the ESSA Index.

Figure 1: 2019 ESSA Score Index and % FRL, High Schools

ESSA FRL HS

 

Figure 2 overlays the High School letter grade categories on top of Figure 1 to further illustrate the relationship between letter grades and high school poverty rates.

Figure 2: 2019 ESSA Score Index and % FRL, High Schools with Letter Grade Overlay

ESSA FRL HS Color

Here at OEP, we choose to highlight student academic growth because we believe that it is the best reflection of the impact that a school is having on students’ academic success.  The relationship between the percentage of students in a school that are eligible for FRL and the students’ academic growth is presented in Figure 3.  At the high school level, these values are even less correlated with demographics than they were at the elementary and middle level (R=-0.29). Improvement in growth scores is positively correlated with improved academic achievement at the school level (R=+0.66), so a good indicator of improvement overall.

Figure 3: 2019 Growth Score and % FRL, High Schools

Growth FRL HS

For OEP awards, we use the purest measure of academic growth (referred to as Combined Content Growth Score) which includes growth for Math and English Language Arts only.  We chose this growth value, which excludes English Learner Progress because, on average, including the ELP progress slightly depresses the growth score for schools. We give OEP awards for high growth overall as well as for Math and ELA growth individually.  We recognize the highest growth schools by school level (elementary, middle and high) and by region of the state.

Today’s awards for High Growth High schools are based on the growth of students on the ACT Aspire Math and English Language Arts assessments in high school level schools.

Highest Growth: High School Level

The top High school for overall student growth is LISA Academy North High Charter School in Sherwood with a growth score of 86.0.  Haas Hall at the Lane in Rogers had the highest Math growth with a score of 88.6, while Shirley High obtained the highest growth score in ELA at 83.1.

The 20 High Schools with the highest overall content growth are:

1.   LISA Academy North High Charter, LISA Academy (55% FRL)**
2.   Calico Rock High, Calico Rock SD (63% FRL)
3.   Haas Hall Academy Bentonville, Haas Hall Bentonville (1% FRL)**
4.   Haas Hall Academy At the Lane, Haas Hall Academy (10% FRL)*
5.   Haas Hall Academy, Haas Hall Academy (6% FRL)**
6.   Haas Hall Academy Jones Center, Haas Hall Academy (16% FRL)*
7.   Danville High, Danville SD (64% FRL)**
8.   Greenbrier Junior High, Greenbrier SD (34% FRL)**
9.   Pottsville High, Pottsville SD (32% FRL)
10. Bismarck High, Bismarck SD (55% FRL)*
11. Eureka Springs High, Eureka Springs SD (46% FRL)**
12. Quitman High, Quitman SD (45% FRL)
13. Concord High, Concord SD (60% FRL)*
14. Kingston High, Jasper SD (45% FRL)
15. Conway Junior High High, Conway SD (45% FRL)
16. Rural Special High, Rural Special SD (62% FRL)*
17. Murfreesboro High, Murfreesboro SD(64% FRL)
18. Flippin High, Flippin SD (69% FRL)
19. Southside Charter High, Southside (Independence) SD (53% FRL)**
19. Hazen High,Hazen SD (70% FRL)*
19. Marmaduke High, Marmaduke SD (60% FRL)

Like we saw with the elementary and middle awards, more than half of the high schools on our top 20 list demonstrate that high growth can be achieved year after year.  Asterisks indicate schools that have been in the top 20 for growth in prior years.

**The seven schools with two asterisks were included in the top 20 list for the last two years, while

*the nine schools with a single asterisk were on the list one of the last two years.

We also like how nearly half of the high schools on the list are newcomers- showing that growth scores can change over time. These schools, and others included in the full report, are growing student’s academic performance more than would be expected. Way to go!

Similarly to last year’s list, a variety of high schools have shown high growth when observed through the lens of the percentage of students served Free/Reduced Lunch. The proportion of students eligible for FRL among these high-growth schools ranges from a low of 1% to a high of 70%, reflecting that students can demonstrate high growth in all types of high schools!

You can find the high schools with the greatest student growth by subject and region in the full report.  You can also download the datafile to check out the growth ranking for all middle schools.

—————Stay tuned to learn about more OEP Award Winners!————–

You can check out the elementary and middle level winners here, and soon we will  release the list of high-growth schools serving high poverty populations, those who are “Beating the Odds!”