University of Arkansas Office for Education Policy

Author Archive

More Money, More Growth?

In The View from the OEP on December 4, 2019 at 1:32 pm

We’ve been thinking a lot this week about school-level expenditures, and if the expenditures relate to student academic growth. We think that growth is the best reflection of the effect that a school is having on student learning. Spoiler Alert: there is essentially no relationship between how much a school is spending and how much growth is being made by the students enrolled.

As you may remember, last spring the state released school-level expenditures for the first time. This presents an unprecedented opportunity to examine the equity, efficiency, and efficacy of Arkansas’ public education spending.

We have examined school-level expenditures through a variety of lenses in previous posts, and have found that Arkansas’ schools spend more the higher a school’s poverty level, that overall traditional school and charter school expenditures per pupil are quite similar, and that Arkansas spends the most on high schools and the least on middle schools, with elementary school spending falling in between.

We also have posted a lot about growth!  You probably know by now that growth is much less correlated to poverty at the school level than achievement, students in schools with larger average class sizes demonstrated greater academic growth than their peers in smaller classes, and growth doesn’t have the intended impact on school level grades.

In this post we are digging into a big question- is more spending related to higher student growth?

In this analysis, we are considering 2017-18 school-level per pupil expenditures and the 2017-18 school content growth score. We are using 2017-18 data since the 2018-19 expenditure data haven’t been released yet. Please note that we exclude the high school level Alternative Learning Environments (ALEs) from this analysis due to the small and specialized populations that they serve.

School expenditures are reported in various categories.  We start with the personnel expenditures as the majority of school funds are used to pay for teacher salaries and benefits. The data presented in Figure 1 show that there is not a strong relationship between school instructional spending and the academic growth of students at the school.  Some schools are represented in the upper left quadrant, spending more than average per student on personnel, while students demonstrate lower than expected growth.  Other schools are represented in the lower right quadrant, where students demonstrate higher than expected growth, even though the school is spending less than average per student on personnel.

Figure 1: Scatter Plot of 2017-18 School-Level Personnel Expenditures Per Pupil vs. Content Growth

Personnel CG

We also considered the relationship between student academic growth and instructional spending. As presented in Figure 2, we see the same lack of relationship between expenditures and student growth.

Figure 2: Scatter Plot of 2017-18 School-Level Instructional Expenditures Per Pupil vs. Content Growth

Inst CG

 

Finally, we expanded our analysis to total school expenditure, because we think maybe all the dollars spent could have an impact on student learning. Once again, there is a lack of a relationship, as shown in Figure 3.

Figure 3: Scatter Plot of 2017-18 School-Level Total Spending Per Pupil vs. Content Growth

total CG

 

Does the relationship vary by school level?

We might think that greater expenditures in elementary schools lead to larger academic gains for students than investments in later grade levels. We previously found that expenditure varies between school levels (elementary, middle, and high). Content growth should not vary by school level, as average growth at each grade level is 80. Below, we present the figures for Elementary, Middle, and High Schools, respectively.

Figure 4: Scatter Plot of 2017-18 School-Level Total Spending Per Pupil vs. Content Growth, Elementary School Level

Elem CG

Figure 5: Scatter Plot of 2017-18 School-Level Total Spending Per Pupil vs. Content Growth, Middle School Level

Mid CG

Figure 6: Scatter Plot of 2017-18 School-Level Total Spending Per Pupil vs. Content Growth, High School Level

High CG

At all school levels we see essentially no relationship between the amount spent per pupil and the academic growth of students.

Does this mean that money doesn’t matter to student growth?

Nope! This analysis is descriptive, not causal, meaning that we are just describing the relationship between two variables, as opposed to claiming that changes in one will (or will not) lead to changes in the other. What our descriptive analyses do shed light on, however, is that schools spending the same amount per pupil can realize very different growth outcomes for students.  This lack of a direct relationship between expenditures and growth could indicate that it is more about HOW schools are using their resources (including time, money, and people) than how many resources they have. We look forward to seeing the 2018-19 school expenditure data and checking to see if the (lack of) relationship still exists. What do you think?

 

Looking for Teachers?

In The View from the OEP on November 20, 2019 at 9:52 am

AR Teachers Logo_horiz PNG

Question: How many clicks does it take for an Arkansas teacher to find a great job?

Answer: Way too many.

Currently, for a teacher to apply for a job in an Arkansas public school they have to:

  1. know the name of a school district where they might be interested in working
  2. find the district website
  3. locate where jobs are posted on the website
  4. filter through job postings for bus drivers, food service personnel, and other positions, to see if there is one that matches their credentials
  5. either create an online account and answer a bunch of questions or, perhaps, download a paper application to fill out and send in
  6. repeat for each school district

While researching Arkansas’ teacher shortage, we have learned that many school districts were having difficulty in attracting qualified teachers (no surprise). We also have discovered that job postings could be difficult for applicants to locate, and that the application process could be complicated and time consuming. District job postings were uninspiring, and generally neglected to mention why teachers should want to work in the district or any incentives intended to entice teachers to the district.

So we decided to try to make the process easier for teachers and districts, and have been working on a “common application” for Arkansas teachers.  ARteachers.org is the free resource designed to make it easier for teachers to find great jobs, and for school districts to find great teachers.

  • ARteachers.org makes it easier for teachers to find district job postings. Once a district posts a position, it will automatically appear to qualified teachers that have created a profile on the site.  Teachers can filter the jobs by distance or other characteristics.
  • ARteachers.org allows districts to pro-actively recruit teachers. Districts have access to a pool of teachers looking for jobs including those interested in long-term substitute opportunities.
  • ARteachers.org makes the job-application process easier for teachers by automatically identifying jobs that match their qualifications, and streamlining the application process with a common application format.
  • ARteachers.org helps teacher applicants be better informed by providing an estimated salary for each position based specifically on their education/experience as well as the districts’ student/teacher ratio.

We know that you already have a process for hiring teachers, but hope that you will add ARteachers.org to your recruitment plan.  In addition to connecting teachers and districts, the site will generate information about how many teaching positions need to be filled and the types of applicants that districts are looking for. This information will allow Arkansas’ policy makers and teacher-preparation programs to better understand the need for educators around the state. The more districts that participate, the better the information we can use as feedback.

This is a soft launch of the system before the full hiring surge later in the Spring. We will be working with educator preparation programs to sign up prospective teachers, allowing districts to pro-actively recruit teachers, like how colleges reach out to high school students and invite them to apply.

ARteachers.org was designed specifically for Arkansas to meet the needs of Arkansas teachers and public school districts. There’s not a lot of bells and whistles- and no one is making any money off of the process.  Our goal was to make the system as simple as possible for both teachers and districts.


Have an open position in your district, or looking for a long-term sub for the spring?  To get started, just ask your HR staff to go to ARteachers.org and sign up. The superintendent will get an email asking to confirm that the HR person is authorized to post jobs for the district, then the district will be on its way to finding the best teachers for their students.

Know a teacher that is or will be looking for a position? Have them go to ARteachers.org and sign up. It just takes a few minutes to create a profile and it is free!

We know that getting a great teacher in every classroom is a critical step in ensuring a quality education for Arkansas students, and ARteachers.org makes it easy and free.

AR Teachers Logo_ state and apple only png

 

 

Which Schools are “Beating the Odds?”

In The View from the OEP on November 13, 2019 at 1:06 pm

Today we share our final OEP awards, and discuss how (and why) our awards are different from the rewards given last Friday by the state.

We are so excited to release our “Beating the Odds” Outstanding Educational Performance Awards  for 2019!  These special OEP awards are for schools whose students are demonstrating high academic growth despite serving a population where at least 66% of the students participate in the Free/ Reduced Lunch Program, which is based on low household income.  Schools serving such student populations often struggle to demonstrate high academic achievement, and subsequently receive lower letter grades.

Academic growth, however, is less correlated with school poverty rates and we think it is a better reflection of how the school is impacting students. Growth is calculated at the student level, and essentially reflects how much a student has improved his or her score from the prior year compared to what was predicted based on prior achievement history. While poverty can negatively impact student success, the schools awarded today demonstrate that their students are “Beating the Odds”  The highlights are below, and you can read the full report here.

The OEP Awards highlight schools in Arkansas based on student growth on the ACT Aspire exams in Mathematics and English Language Arts (ELA). We choose to give OEP Awards based on student growth because we think it is the best indicator of how the school is impacting students’ learning.

Although school-level growth scores are much less related to the percentage of students at a school who are participating in Free/Reduced Lunch than achievement scores, a negative correlation does exist (-0.21).  This means that students at schools serving higher poverty populations are more likely than their peers at more affluent schools to demonstrate less academic growth than predicted. As can be seen in the scatter plot below, schools with higher FRL rates are more likely to receive lower growth scores.

Figure 1. Combined Content Growth Score by School % FRL, Arkansas Public Schools, 2019

2019 Scatter Plot

If we limit the plot to only those schools with at least 66% of students participating in FRL, as presented in Figure 2, the relationship between poverty and growth essentially disappears. Although all of these schools are serving high poverty populations, there is wide variation in the amount of academic growth that students at the schools are demonstrating.

Figure 2. Combined Content Growth Score by School % FRL, High-Poverty Arkansas Public Schools, 2019

2019 Poverty Scatter Plot

We celebrate the state using this student-level growth model, and are pleased to be able to highlight how students are growing academically in schools across the state.  We hope that drawing attention to this growth information will spark discussions among stakeholders about the ways to ensure that all schools are growing the knowledge of Arkansas’ students.


“Beating the Odds” Elementary Level Schools

The top “Beating the Odds” elementary school overall is Salem Elementary from Salem School District.  Despite serving a student population that is 67% eligible for Free/Reduced Lunch, Salem Elementary students are among the top 5 schools that have demonstrated the greatest growth in the state on the ACT Aspire. Many of these top 10 Beating the Odds schools were also among the high growth elementary schools in the state, regardless of student demographics. The top 10 elementary schools that are beating the odds are:

  1. Salem Elementary, Salem SD (67% FRL)*
  2. Weiner Elementary, Harrisburg SD (68% FRL)
  3. Oscar Hamilton Elementary, Foreman SD (71% FRL)**
  4. Cross County Elementary Tech Academy, Cross County SD (73% FRL)*
  5. Lamar Elementary, Lamar SD (72% FRL)
  6. Des Arc Elementary, Des Arc SD (75% FRL)
  7. Frank Tillery Elementary, Rogers SD (70% FRL)
  8. Green Forest Elementary, Green Forest SD (86% FRL)**
  9. George Elementary, Springdale SD (86% FRL)
  10. Bismark Elementary, Bismark SD (72% FRL)**

**Schools with two asterisks were included in the top 10 list for the last two years, while *schools with a single asterisk were on the list one of the last two years.

You can find the top BTO elementary schools by subject and region in the full report.


“Beating the Odds” Middle Level Schools

Paragould Junior High from Paragould School District is the top middle school beating the odds overall. Paragould JH serves a 7th-8th grade student population where 71% of students are eligible for Free/Reduced Lunch, and was third among the high growth middle schools in the state, regardless of student demographics.  The top 10 middle schools that are beating the odds are:

  1. Paragould Junior High, Paragould SD (71% FRL)**
  2. Oak Grove Middle, Paragould SD (76% FRL)**
  3. Swifton Middle, Jackson Co. SD (66% FRL)
  4. Helen Tyson Middle, Springdale SD (76% FRL)*
  5. Atkins Middle, Atkins SD (68% FRL)
  6. Cedarville Middle, Cedarville SD (73% FRL)**
  7. Pleasant View Campus, Mulberry/Pleasant View Bi-County Schools (77% FRL)*
  8. Harrisburg Middle, Harrisburg SD (74% FRL)
  9. Yerger Junior High, Hope SD (80% FRL)
  10. Beryl Henry Upper Elementary, Hope SD (89% FRL)**

**Schools with two asterisks were included in the top 10 list for the last two years, while *schools with a single asterisk were on the list one of the last two years.

 

You can find the top BTO middle schools by subject and region in the full report.


“Beating the Odds” High Schools

The top high school beating the odds is Flippin High in Flippin School District.  Despite serving a student population that is 69% eligible for Free/Reduced Lunch, it is also among OEP’s top 10 high growth high schools throughout the state.  Flippin High students are demonstrating that they can achieve at levels similar to students who come from higher income communities. The top 10 high schools that are beating the odds are:

  1. Flippin High, Flippin SD (69% FRL)
  2. Hazen High, Hazen SD (70% FRL)
  3. Maynard High, Maynard SD (69% FRL)*
  4. St. Joe K-12, Ozark Mountain SD (73% FRL)
  5. Barton High, Barton-Lexa SD (85% FRL)
  6. Jasper High, Jasper SD (70% FRL)
  7. Gosnell High, Gosnell (67% FRL)**
  8. Decatur High, Decatur SD (72% FRL)
  9. Cave City High Career & Collegiate Preparatory, Cave City SD (77% FRL)*
  10. Sparkman High, Harmony Grove (Ouachita) SD (73% FRL)

**Schools with two asterisks were included in the top 10 list for the last two years, while *schools with a single asterisk were on the list one of the last two years.

You can find the top BTO high schools by subject and region in the full report.

Congratulations to all the OEP “Beating the Odds” award winners!  Keep up the great work and we look forward to recognizing you again next year!


How are OEP awards different from the state rewards that were announced last week?

1) Part of the state rewards go to high-achieving schools, where a lot of students scored well on the state tests. These schools tend to serve a lower population of students facing academic risk factors poverty or second language acquisition.

OEP only awards schools for growth, because we think that it is a better reflection of how the school is impacting students.

2) The part of the state rewards that are awarded for growth use a different measure than the OEP awards.  The rewards program uses the growth value that also includes the progress being made in English language proficiency, a value called the combined value-added growth score. The difference between the values is inconsistent, with the content growth value higher for some schools and the combined value-added value higher for other schools.

OEP awards are based on improvement in the content areas of Mathematics and English Language Arts (ELA) assessments only. 

3) The state rewards program for growth includes graduation rate for high schools.

OEP awards do not include graduation rate, and are based on improvement in the content areas of Mathematics and English Language Arts (ELA) assessments only. 

4) The state rewards program rewards the top 10% of schools overall.

OEP awards are grouped by ESSA school level (Elementary, Middle, and High). We know that achievement and growth vary by school level and are concerned that middle schools demonstrating relatively high growth are not being rewarded by the state. In fact, only two middle schools were recognized in the top 5% growth/grad awards by the state.  See a further discussion here.

The differences between the state rewards program and OEP awards are due to the fact that the state rewards are legislatively mandated, while here at OEP, we created an awards system that supports our passion for highlighting schools where students demonstrate Oustanding Educational Progress!  Oh, and we don’t send money- just paper certificates!

 

 

Crossing Our Fingers Didn’t Work…

In The View from the OEP on October 31, 2019 at 4:00 pm

NAEP

When the disappointing 2017 NAEP results came out, we said, “We have our fingers crossed that the changes laid out in ESSA will make a big difference to student learning in Arkansas, and look forward to seeing NAEP results again in 2019”.

Well, NAEP results were released yesterday, and Arkansas’ scores look about the same as they did in 2015 and 2017. NAEP is administered nationally to a representative sample of students from all 50 states, so acts as a standard measure of student performance across states and time. We present NAEP performance since 2003, when administration became consistent. We dug into the new results and are pleased to share six NAEP nuggets with you.

NAEP Nugget #1: Not Bouncing Back

Arkansas’ 2019 NAEP scores were essentially unchanged from the past two administrations.  Unfortunately, this is not great news because we were all hoping 2015 and 2017 were just blips that we would bounce back from. In fact, as the figure below highlights, Arkansas scores were the highest in 2011 and 2013, and demonstrate declines since.

Figure 1: Arkansas NAEP Scores, 2003-2019

NAEPF1

NAEP Nugget #2: Not Keeping Up With the Neighbors

4th and 8th grade Math scores are lower than those of Arkansas’ border states (this group includes Louisiana, Mississippi, Missouri, Oklahoma, Tennessee, and Texas).  This is particularly a bummer in 4th grade because we outperformed them from 2005 to 2013, and while the border states’ average scores are rising, Arkansas’ are declining. It’s also a bummer because we seem to be falling farther and farther behind.

Figure 2: 4th grade NAEP Mathematics Scores, Arkansas, Border States, and US, 2003-2019

NAEPF2

Figure 3: 8th grade NAEP Mathematics Scores, Arkansas, Border States, and US, 2003-2019

NAEPF3

NAEP Nugget #3: In the Same Boat in Reading

4th and 8th grade Reading scores are also lower than those of Arkansas’ border states, but we seem to be tracking the national performance pattern. Again, this is particularly a bummer in 4th grade because we outperformed them from 2003 to 2013.

Figure 4: 4th grade NAEP Reading Scores, Arkansas, Border States, and US, 2003-2019

NAEPF4

When it comes to 8th grade reading, scores decreased for Arkansas, our border states, and the nation as a whole. In fact, students in 31 states performed worse in 8th grade reading than they did in 2017.

Figure 5: 8th grade NAEP Reading Scores, Arkansas, Border States, and US, 2003-2019

NAEPF5

NAEP Nugget #4: Widening Gaps for Students in Poverty

Score gaps between Arkansas students eligible for Free/Reduced Lunch and their peers who are not eligible continued to widen in 2019 due to decreased performance of at-risk groups and/ or increased performance of other students.  Gaps in 8th grade mathematics and reading between FRL and Non-FRL students are greater than they have been since 2003.

Figure 6: FRL and Non-FRL Eligible Students’ 8th grade NAEP Mathematics, Arkansas, 2003-2019

NAEPF6

 

NAEP Nugget #5: This is GOOD news! 

Hispanic students are closing the gap to their White peers in both 4th and 8th grade reading! Score gaps between the groups in mathematics remain consistent.

Figure 7: Hispanic and White Students’ 4th grade NAEP Reading, Arkansas, 2003-2019

NAEPF7

Gaps between Black and White student scores remain large and persistent in both content areas at 4th and 8th grade.

NAEP Nugget #6: Mixed Messages

Proficiency rates on NAEP math and reading assessments are lower than on ACT Aspire, Arkansas’ assessment used to measure student achievement. Math proficiency rates on NAEP are about 20 percentage points lower than on ACT Aspire.  Reading proficiency rates were around 30% on NAEP, but 45% of 4th graders and 53% of 8th graders were determined proficient on the ACT Aspire.

Figure 8: Arkansas NAEP Proficiency vs. ACT Aspire Proficiency, Math and Reading, 4th and 8th grade, 2019

NAEPF8

National assessments like NAEP provide a common measuring stick for student performance, which is particularly important as Arkansas is the only state administering ACT Aspire statewide in grades 3-10. We need to pay careful attention to the differences between the NAEP and ACT Aspire proficiency rates. Arkansas will be administering ACT Aspire for at least seven more years, and when we send and receive conflicting messages about how well our students are performing, it can make it difficult to determine how well our students are doing and which sorts of educational interventions are making a difference for our students.

Three measures of Arkansas’ academic achievement (NAEP, ACT Aspire, and ACT) are all essentially flat at the state level over the past 3 to 4 years.

During this time the state has consistently increased per pupil funding, invested in literacy training, re-created school accountability systems, moved to universal ACT testing for juniors, refocused teacher and principal evaluation, and much more. Maybe we haven’t allowed enough time to see the results of the changes we have been making in education.  Or maybe those changes aren’t really having an impact at the school level.  Or maybe the changes ARE working and these measures just aren’t capturing the improvement.

Or maybe we need to step back, and look at school-level growth instead of statewide achievement to see something worth cheering about!

If you haven’t checked out this year’s OEP awards you should!  There is some GOOD news.

 

 

High School Awards: Outstanding Educational Performance

In The View from the OEP on October 30, 2019 at 11:20 am

 

This week, OEP recognizes high schools demonstrating Outstanding Educational Performance. OEP awards are different than other awards because we focus solely on student academic growth.

Earlier this month, the Division of Elementary and Secondary Education released school letter grades for public schools in the state.  We created a statewide data visualization for you to explore the relationships between letter grades, school poverty, and academic growth.

We think school letter grades can be helpful in communicating with stakeholders about school performance, but are concerned that Arkansas’ letter grades send the wrong message because they are driven primarily by student achievement, which is generally reflective of the demographics of the students in the school.  As we discussed previously when awarding elementary and middle schools, the overall score on which the letter grades are based, are strongly correlated with the percentage of students in the school who are participating in the Free/Reduced Lunch program (an indicator of low-income). For High Schools, as illustrated in Figure 1, we find that the negative correlation is even slightly lower than it was for elementary and middle schools (R= -0.66), likely dure to the inclusion of the graduation rate in the ESSA Index.

Figure 1: 2019 ESSA Score Index and % FRL, High Schools

ESSA FRL HS

 

Figure 2 overlays the High School letter grade categories on top of Figure 1 to further illustrate the relationship between letter grades and high school poverty rates.

Figure 2: 2019 ESSA Score Index and % FRL, High Schools with Letter Grade Overlay

ESSA FRL HS Color

Here at OEP, we choose to highlight student academic growth because we believe that it is the best reflection of the impact that a school is having on students’ academic success.  The relationship between the percentage of students in a school that are eligible for FRL and the students’ academic growth is presented in Figure 3.  At the high school level, these values are even less correlated with demographics than they were at the elementary and middle level (R=-0.29). Improvement in growth scores is positively correlated with improved academic achievement at the school level (R=+0.66), so a good indicator of improvement overall.

Figure 3: 2019 Growth Score and % FRL, High Schools

Growth FRL HS

For OEP awards, we use the purest measure of academic growth (referred to as Combined Content Growth Score) which includes growth for Math and English Language Arts only.  We chose this growth value, which excludes English Learner Progress because, on average, including the ELP progress slightly depresses the growth score for schools. We give OEP awards for high growth overall as well as for Math and ELA growth individually.  We recognize the highest growth schools by school level (elementary, middle and high) and by region of the state.

Today’s awards for High Growth High schools are based on the growth of students on the ACT Aspire Math and English Language Arts assessments in high school level schools.

Highest Growth: High School Level

The top High school for overall student growth is LISA Academy North High Charter School in Sherwood with a growth score of 86.0.  Haas Hall at the Lane in Rogers had the highest Math growth with a score of 88.6, while Shirley High obtained the highest growth score in ELA at 83.1.

The 20 High Schools with the highest overall content growth are:

1.   LISA Academy North High Charter, LISA Academy (55% FRL)**
2.   Calico Rock High, Calico Rock SD (63% FRL)
3.   Haas Hall Academy Bentonville, Haas Hall Bentonville (1% FRL)**
4.   Haas Hall Academy At the Lane, Haas Hall Academy (10% FRL)*
5.   Haas Hall Academy, Haas Hall Academy (6% FRL)**
6.   Haas Hall Academy Jones Center, Haas Hall Academy (16% FRL)*
7.   Danville High, Danville SD (64% FRL)**
8.   Greenbrier Junior High, Greenbrier SD (34% FRL)**
9.   Pottsville High, Pottsville SD (32% FRL)
10. Bismarck High, Bismarck SD (55% FRL)*
11. Eureka Springs High, Eureka Springs SD (46% FRL)**
12. Quitman High, Quitman SD (45% FRL)
13. Concord High, Concord SD (60% FRL)*
14. Kingston High, Jasper SD (45% FRL)
15. Conway Junior High High, Conway SD (45% FRL)
16. Rural Special High, Rural Special SD (62% FRL)*
17. Murfreesboro High, Murfreesboro SD(64% FRL)
18. Flippin High, Flippin SD (69% FRL)
19. Southside Charter High, Southside (Independence) SD (53% FRL)**
19. Hazen High,Hazen SD (70% FRL)*
19. Marmaduke High, Marmaduke SD (60% FRL)

Like we saw with the elementary and middle awards, more than half of the high schools on our top 20 list demonstrate that high growth can be achieved year after year.  Asterisks indicate schools that have been in the top 20 for growth in prior years.

**The seven schools with two asterisks were included in the top 20 list for the last two years, while

*the nine schools with a single asterisk were on the list one of the last two years.

We also like how nearly half of the high schools on the list are newcomers- showing that growth scores can change over time. These schools, and others included in the full report, are growing student’s academic performance more than would be expected. Way to go!

Similarly to last year’s list, a variety of high schools have shown high growth when observed through the lens of the percentage of students served Free/Reduced Lunch. The proportion of students eligible for FRL among these high-growth schools ranges from a low of 1% to a high of 70%, reflecting that students can demonstrate high growth in all types of high schools!

You can find the high schools with the greatest student growth by subject and region in the full report.  You can also download the datafile to check out the growth ranking for all middle schools.

—————Stay tuned to learn about more OEP Award Winners!————–

You can check out the elementary and middle level winners here, and soon we will  release the list of high-growth schools serving high poverty populations, those who are “Beating the Odds!”

Middle-Level Schools with Outstanding Educational Performance

In The View from the OEP on October 23, 2019 at 1:53 pm

This week, OEP recognizes middle schools demonstrating Outstanding Educational Performance. OEP awards are different than other awards because we focus solely on student academic growth.

Earlier this month, the Division of Elementary and Secondary Education released school letter grades for public schools in the state.  We created a statewide data visualization for you to explore the relationships between letter grades, school poverty, and academic growth.

We think school letter grades can be helpful in communicating with stakeholders about school performance, but are concerned that Arkansas’ letter grades send the wrong message because they are driven primarily by student achievement, which is generally reflective of the demographics of the students in the school.  As we discussed last week when we were awarding elementary schools, the overall score on which the letter grades are based, are strongly correlated with the percentage of students in the school who are participating in the Free/Reduced Lunch program (an indicator of low-income). For Middle Schools, as illustrated in Figure 1, we find that the negative correlation is even stronger than it was for elementary schools (R= -0.74).

Figure 1: 2019 ESSA Score Index and % FRL, Middle Schools

ESSA FRL MS

 

Figure 2 overlays the Middle Level letter grade categories on top of Figure 1 to further illustrate the relationship between letter grades and middle school poverty rates.  No Middle School that serves more than 80% FRL students received an A, and only one received a B. Conversely, among schools serving a student population where fewer than 60% of students are eligible for FRL, only two received a D grade.

Figure 2: 2019 ESSA Score Index and % FRL, Middle Schools with Letter Grade Overlay

ESSA FRL MS color

Here at OEP, we choose to highlight student academic growth because we believe that it is the best reflection of the impact that a school is having on students’ academic success.  The relationship between the percentage of students in a school that are eligible for FRL and the students’ academic growth is presented in Figure 3.  At the middle level, these values are correlated with demographics about the same as they were at the elementary level (R=-0.36). Improvement in growth scores is positively correlated with improved academic achievement at the school level (R=+0.66), so a good indicator of improvement overall.

Figure 3: 2019 Growth Score and % FRL, Middle Schools

Growth FRL MS

For OEP awards, we use the purest measure of academic growth (referred to as Combined Content Growth Score) which includes growth for Math and English Language Arts only.  We chose this growth value, that excludes English Learner Progress because, on average, including the ELP progress slightly depresses the growth score for schools. We give OEP awards for high growth overall as well as for Math and ELA growth individually.  We recognize the highest growth schools by school level (elementary, middle and high) and by region of the state.

Today’s awards for High Growth Middle schools are based on the growth of middle level students on the ACT Aspire Math and English Language Arts assessments.

Highest Growth: Middle Level

The top Middle school for overall student growth is Flightline Upper Academy, a Jacksonville Lighthouse Charter, with a growth score of 86.6.  Flightline also had the highest Math growth with a score of 90.12, while Beryl Henry Upper Elementary in Hope obtained the highest growth score in ELA at 85.97.

The 20 middle level schools with the highest overall content growth are:

1.   Flightline Upper Academy, Jacksonville Lighthouse Charter (30% FRL)
2.   Heber Springs Middle, Heber Springs SD (58% FRL)*
3.   Paragould Junior High, Paragould SD (71% FRL)*
4.   LISA Academy North Middle Charter, LISA Academy (57% FRL)**
5.   Ray/Phyllis Simon Middle, Conway SD (58% FRL)
6.   Ruth Doyle Middle, Conway SD (41% FRL)
7.   Eureka Springs Middle, Eureka Springs SD (59% FRL)*
8.   Northridge Middle, Van Buren SD (49% FRL)
9.   Oak Grove Middle, Paragould SD (76% FRL)*
10. Cabot Junior High North, Cabot SD (38% FRL)*
11. Gravette Middle, Gravette SD (51% FRL)**
12. Lincoln Junior High, Bentonville SD (27% FRL)*
12. Swifton Middle, Jackson Co. SD (66% FRL)*
14. Greenbrier Middle, Greenbrier SD (39% FRL)*
15. Glen Rose Middle, Glen Rose SD (50% FRL)
15. Beebe Junior High, Beebe SD (51% FRL)
17. Helen Tyson Middle, Springdale SD (77% FRL)*
18. Bergman Middle, Bergman SD (63% FRL)
19. Woodland Junior High, Fayetteville SD (26% FRL)
20. Pinkston Middle, Mountain Home SD (19% FRL)
20. McNair Middle, Fayetteville SD (47% FRL)

Here at OEP we like that more than half of the middle level schools on our top 20 list demonstrate that high growth can be achieved year after year.  Asterisks indicate schools that have been in the top 20 for growth in prior years. The two schools with two asterisks were included in the top 20 list for the last two years, while the nine schools with a single asterisk were on the list one of the last two years.  We also like how nearly half of the schools on the list are newcomers- showing that growth scores can change over time. These schools, and others included in the full report, are growing student’s academic performance more than would be expected. Way to go!

Similarly to last year’s list, a variety of middle schools have shown high growth when observed through the lens of the percentage of students served Free/Reduced Lunch. The proportion of students eligible for FRL among these high-growth schools ranges from a low of 19% to a high of 77%, reflecting that students can demonstrate high growth in all types of schools!

You can find the middle schools with the greatest student growth by subject and region in the full report.  You can also download the datafile to check out the growth ranking for all middle schools.

—————Stay tuned to learn about more OEP Award Winners!————–

You can check out the elementary winners here, and next week we will share “High Growth” High Schools.  Then, we will release the list of high-growth schools serving high poverty populations, those who are “Beating the Odds!”

Is your elementary school getting Outstanding Educational Performance?

In The View from the OEP on October 16, 2019 at 11:32 am

This week, OEP is pleased to recognize elementary schools demonstrating Outstanding Educational Performance. OEP awards are different than other awards because we focus solely on student academic growth.

Last week, the Division of Elementary and Secondary Education released school letter grades for public schools in the state.  We created a statewide data visualization for you to explore the relationships between letter grades, school poverty, and academic growth.

We think school letter grades can be helpful in communicating with stakeholders about school performance, but are concerned that Arkansas’ letter grades send the wrong message because they are driven primarily by student achievement, which is generally reflective of the demographics of the students in the school.  As shown in Figure 1, the overall score on which the letter grades are based, are strongly correlated (R=-0.71) with the percentage of students in the school who are participating in the Free/Reduced Lunch program (an indicator of low-income). Figure 2 overlays the elementary letter grade categories on top of Figure 1 to further illustrate the relationship between letter grades and school poverty rates.

Figure 1: 2019 ESSA Score Index and % FRL, Elementary Schools

ESSA FRL

Figure 2: 2019 ESSA Score Index and % FRL, Elementary Schools with Letter Grade Overlay

ESSA FRL 2

Here at OEP, we choose to highlight student academic growth because we believe that it is the best reflection of the impact that a school is having on students’ academic success.  As shown in Figure 3, academic growth, is not very correlated with school demographics (R=-0.37), and improvement in growth scores is positively correlated with improved academic achievement at the school level (R=+0.66), so a good indicator of improvement overall.

Figure 3: 2019 Growth Score and % FRL, Elementary SchoolsGrowth FRL

For OEP awards, we use the purest measure of academic growth (referred to as Combined Content Growth Score) which includes growth for Math and English Language Arts only.  We chose this growth value, that excludes English Learner Progress because, on average, including the ELP progress slightly depresses the growth score for schools. We give OEP awards for high growth overall as well as for Math and ELA growth individually.  We recognize the highest growth schools by school level (elementary, middle and high) and by region of the state.

Today’s awards for High Growth Elementary schools are based on the growth of elementary students on the ACT Aspire Math and English Language Arts assessments.

Highest Growth: Elementary Level

The top elementary school for overall student growth is City Heights Elementary from Van Buren School District, with a growth score of 90.21. Stagecoach Elementary from Cabot School District had the highest Math growth with a score of 91.59, while City Heights Elementary also obtained the highest growth score in ELA at 89.97.

The 20 elementary schools with the highest overall content growth are:

1.   City Heights Elementary, Van Buren SD (52% FRL)**
2.   Ward Central Elementary, Cabot SD (58% FRL)
3.   Pottsville Elementary, Pottsville SD (48% FRL)
4.   Carolyn Lewis Elementary, Conway SD (53% FRL)
5.   Salem Elementary, Salem SD (67% FRL)**
6.   Stagecoach Elementary, Cabot SD (39% FRL)
6.   Weiner Elementary, Harrisburg SD (68% FRL)
8.   Center Valley Elementary, Russelville SD (51% FRL)*
9.   Oscar Hamilton Elementary, Foreman SD (71% FRL)**
10. Greenbrier Wooster Elementary, Greenbrier SD (42% FRL)**
11. Westside Elementary, Cabot SD (62% FRL)
12. Cross County Elementary Tech Academy, Cross County SD (73% FRL)*
12. Greenbrier Springhill Elementary, Greenbrier SD (43% FRL)**
12. Bernice Young Elementary, Springdale SD (27% FRL)*
15. Walker Elementary, Springdale SD (54% FRL)
16. Lamar Elementary, Lamar SD (72% FRL)
17. Chenal Elementary, Pulaski County Special SD (23% FRL)
18. Hunt Elementary, Springdale SD (46% FRL)**
19. Des Arc Elementary, Des Arc SD (75% FRL)*
20. Frank Tillery Elementary, Rogers SD (70% FRL)
20. Jefferson Elementary, Little Rock SD (28% FRL)

Here at OEP we like that half of the elementary schools on our top 20 list demonstrate that high growth can be achieved year after year.  Asterisks indicate schools that have been in the top 20 for growth in prior years. The six schools with two asterisks were included in the top 20 list for the last two years, while the four schools with a single asterisk were on the list one of the last two years.  We also like how half of the schools on the list are newcomers- showing that growth scores can change over time. These schools, and others included in the full report, are growing student’s academic performance more than would be expected. Way to go!

Similarly to last year’s list, a variety of schools have shown high growth when observed through the lens of the percentage of students served Free/Reduced Lunch. The proportion of students eligible for FRL among these high-growth schools ranges from a low of 27% to a high of 75%, reflecting that students can demonstrate high growth in all types of schools!

You can find the elementary schools with the greatest student growth by subject and region in the full report.  You can check out the growth ranking of all elementary schools in the downloadable datafile.

—————Stay tuned to learn about more OEP Award Winners!————–

Next week we will share “High Growth” Middle level schools, followed by High Schools, and then we will release the list of high growth schools serving high poverty populations, those who are “Beating the Odds!”

Computer Science + AR = Success!

In The View from the OEP on August 28, 2019 at 11:18 am

Students in Arkansas are getting a lot more opportunities to enroll in a Computer Science course in high school, and an increasing number of students are taking advantage of those opportunities!  In today’s blog, we review how this change came about, what types of students are benefiting, and suggest some next steps to keep the momentum going.

The computer science initiative in Arkansas began in earnest in February 2015, when Governor Hutchinson signed Act 187, which required all public high schools to offer computer science education beginning in 2015-16.  Since then, student enrollment in CS classes has taken off, climbing to over 6,300 students in 2017-18 (the most recent year for which data are currently available).

Figure 1. Computer Science Enrollment in Arkansas, 2007-08 through 2017-18

CS_enroll

Students weren’t required to take the CS courses, so what led to this rapid increase in enrollment?   Some schools had already been offering computer science courses prior to the law, but fewer than 900 high school students were enrolled in such courses in 2014-15.  This enrollment was reflective of a recent increase as fewer than 250 were enrolled annually prior to 2013-14. We expect there was some unmet demand prior to 2015-16, but the results are impressive!

We suggest the increased enrollment is the outcome of clear direction and ensuring resources for success. The state has taken two key steps in the process:

1) Incentivize teachers to get certified to teach CS courses.  With each high school required to offer CS, Arkansas needed a lot more CS teachers!  The state provided money for teacher certification (Praxis reimbursement and teacher stipends of up to $2000). According the Anthony Owen, Chief State STEM Officer and state director of Computer Science Education, currently over 200 teachers are fully certified and nearly 600 others are in the process.

2) Incentivize students to enroll in CS courses. Just because a class is offered doesn’t mean students will want to enroll. There are many other courses competing for their time.  The state allowed a CS course credit to work as a substitute for the 4th math or 3rd science course required for graduation, allowing students to take something else off their plate so they could enroll in CS. In addition, the state offered financial incentives for students to take an advanced CS course.  The AP Advanced Computer Science A initiative awards students between $250 and $1000 for passing the AP Computer Science A exam. The exam (like all AP exams) is free for Arkansas students to take, and they are guaranteed to get credit at all state Universities for a passing score.

It sounds simple, but such clear direction and aligned resource allocation is rare. So teachers are teaching CS, and students are taking CS, but we wondered, has the initiative changed the type of students engaging with the content?

In 2007-08, students who were enrolled in computer science courses in Arkansas were generally white (76%), not economically disadvantaged (76% non-Free/Reduced Lunch), and male (70%).  Figure 2 displays enrollment by race statewide (top) and in computer science courses (bottom).  We need to consider demographics of the state as a whole to provide context to any changes we might see in CS enrollment.

Racial Enrollment Changes

As presented in Figure 2, there are differences between the statewide racial demographic percentages and the racial demographic percentages of students enrolled in CS courses.  In addition, the CS demographics had shifted somewhat over time.

Figure 2. Enrollment in Arkansas Statewide and for Computer Science Courses by Race, 2007-08 through 2017-18

race_legend

Statewide Enrollment by Race

School_Race

Computer Science Enrollment by RaceCS_race

  • White students consistently make up the greatest share of CS students, and generally make up a greater share of CS enrollment than of the overall school population statewide. White students accounted for 68% of CS enrollment in 2017-18 compared to 63% of school enrollment overall.
  • Black students make up about 16% of the CS enrollment, which is slightly less than the 21% share of overall student enrollment. While enrollment share in CS declined in 2009-2013, recent years reflect an increased percentage of CS enrollment.
  • Hispanic students account for about 10% of CS enrollment, which is close to the 12% share of overall enrollment. The Hispanic enrollment share in CS courses has declined somewhat since the CS initiative began, likely due to increased enrollment in more rural school communities.
  • Asian students’ share of CS enrollment has fluctuated over time, ranging from 12% to 4% of students.  This is slightly greater than the group’s 2% representation in the statewide school enrollment.  Similar to Hispanic enrollment, Asian enrollment share in CS courses has declined somewhat since the CS initiative began, likely due to increased enrollment in more rural school communities.

Economically Disadvantaged Enrollment Changes

We do see an increase in the enrollment of economically disadvantaged students since the initiative began in 2015-16.  As presented in Figure 3, 49% of the students enrolled in CS in 2017-18 were participating in the Free/Reduced Lunch Program (a proxy indicator of poverty), which is an increase of 19 percentage points over 2007-08 participation rate. In addition, this has halved the gap between statewide FRL rates and CS enrollment rates from 17 percentage points in 2007-08 to 8 percentage points in 2017-18.

Figure 3. Enrollment in Arkansas Statewide and for Computer Science Courses by Participation in Free/ Reduced Lunch Program, 2007-08 through 2017-18

CS_FRL

Gender Enrollment Changes

When we examined CS enrollment by gender, we find that females consistently make up about 27%-30% of CS students, although about half the students in the state are females.   As presented in Figure 4, the ratio of females in the CS courses in the state has remained relatively unchanged since 2007-08.

Figure 4. Female Enrollment in Arkansas Statewide and for Computer Science Courses, 2007-08 through 2017-18

When we examined CS enrollment by gender, we find that females consistently make up about 27%-30% of CS students, although about half the students in the state are females.   As presented in Figure 4, the ratio of females in the CS courses in the state has remained relatively unchanged since 2007-08.  The under-representation of females in CS is not unique to Arkansas, and work is being done across the nation to increase female participation in STEM.  Arkansas educators are working too, for example, incorporating CS into Family and Consumer Science courses by having students code in embroidery machines!

Figure 4. Female Enrollment in Arkansas Statewide and for Computer Science Courses, 2007-08 through 2017-18

CS_Females

When we examine female participation in AP Computer Science A exams, we find that Arkansas had a higher ratio of females taking the exams than the nation as a whole!  As presented in Figure 5, the percentage of AP CSA exams taken by females has been steadily increasing, and in 2017-18, 27% of Arkansas participants were female (compared to 24% of exam takers in the US overall).

Figure 5. Advanced Placement Computer Science A Exams, Percentage Female, 2007-08 through 2017-18

AP_CSA_Females


So, the data shows some positive student outcomes from the Computer Science initiative- increased enrollment overall, increased participation by economically disadvantaged students, and above average AP exam-taking by female students.  What are the next steps to keep the momentum going?

1) Make sure CS stays important to schools: Arkansas wrapped computer science into the states ESSA school accountability plan,  ensuring that CS stays important to schools.  Schools get points for students that have taken CS, and for students taking AP courses (including CSA).

OEP Suggestion: Enhance the benefit by rewarding schools for industry-recognized certifications and passing scores on AP exams.  These indicators of measurable success demonstrate that students aren’t just enrolling in CS, but are actually mastering meaningful content.

2) Make sure CS is actually beneficial to students. While we can see increased enrollment, we don’t yet know what students are actually DOING with the CS knowledge they are getting.  Anthony Owen indicated that the state is working towards an industry certification program, which is key.  These certifications are meaningful outcomes for students that they can take directly into the workforce, and will be an economic driver for the state.

OEP Suggestion: Keep tracking the data and doing research.  The state, and the students, have made significant investments in CS.  We need to learn more about what really helps students.  Are students taking multiple classes?  How many are getting certifications and/or going to college for CS? For example, about 1/5 of students are taking the courses online- we need to determine if the instructional effectiveness is different than face to face instruction.  Keep an eye on gender enrollment- are there schools where more females or underrepresented races are more likely to enroll?  Can we determine what creates that culture?

3) Make sure there are viable pathways for students to continue CS after high school. How are businesses and colleges preparing for these students?  There is lots of knowledge coming their way, so we need to help them capitalize on it!  The state requires that colleges assign credit for a passing score on AP computer science, but how are our post secondary schools going to continue to challenge them?  How can we create partnerships with businesses to ensure we reap the benefit from their CS experience?

OEP Suggestion: Keep up excellent communication and forward planning.  The website for CS in AR has lots of information and does a great job of getting the word out about what is going on.  With continued focus, we will be able to model the success of the CS initiative beyond high school enrollment.

 

 

Comparing Traditional School and Open-Enrollment Charter School Expenditures

In The View from the OEP on August 8, 2019 at 7:43 am

Back in May we dug into the brand-new school-level spending data in two blog posts. In our first post we showed that Arkansas’ schools spend most of their money on people (76%) and instruction (56%). We also showed that schools with higher poverty levels, as measured by Free and Reduced-Price Lunch (FRL) percentage, tend to spend more.

Our second post broke the spending data out by school level (i.e., elementary, middle, high). We found that Arkansas spends far more per pupil on high school than it does on elementary and middle school (i.e., >$1,000 more per pupil). Somewhat surprising to us, we also found that Arkansas spends the least on middle schools.

In this post, we compare school-level spending between traditional schools and open-enrollment charter schools. For the remainder of the post, we will refer to open-enrollment charter schools simply as charters or charter schools. Arkansas also has district conversion charter schools, which are traditional schools that have been provided with additional flexibility, but they are funded in the same way as traditional schools. Conversion charter schools are treated as traditional schools for our analysis.

Both traditional schools and charter schools receive funding through the state funding formula. However, local property taxes cover a portion of traditional schools’ per pupil funding amount as determined by the formula, and traditional districts have the ability to raise additional dollars if voters approve higher property tax rates. State-level taxes fill in the gap between the formula funding amount and the amount traditional districts raise through property taxes, increasing funding equity between wealthy and poor districts.

Charter schools, on the other hand, do not have geographic attendance zones, and thus cannot raise local property tax revenue. Charters’ per pupil formula funding amount is fully covered through state-level taxes.

The inability of charter schools to access local property taxes can create big differences in expenditures between traditional schools and charter schools. Facilities funding is often a big driver of these differences. A large proportion of traditional districts’ funding for facilities comes from local property taxes, and state and philanthropic funding for charter facilities generally falls short of the amount traditional districts spend on building projects. This post will not dig into the particulars of Arkansas school facilities funding, but will instead focus overall differences and the expenditure categories we presented in our earlier posts. In the coming weeks, will likely follow up with a post devoted to charter facilities funding.

Arkansas currently has 25 open-enrollment charter school districts comprised of 49 individual schools. Charters make up only about 5 percent of schools in the state, educating around 3 percent of students. Figure 1 below shows a scatter plot of school-level spending versus the percentage of students eligible for the Federal Free and Reduced-Price Lunch (FRL) program. Charter schools are represented by the orange circles. Charter schools are scattered throughout the plot, and do not deviate significantly from the overall spending vs. FRL trend we showed in our first post.

Figure 1: Scatter Plot of 2017-18 School-Level Spending Per Pupil vs. Percent FRL

Screen Shot 2019-08-07 at 11.43.21 AM

While the scatter plot above shows that charter schools closely resemble traditional schools in terms of expenditures and percent FRL, there are some differences in spending levels. Figure 2 illustrates the distribution of school-level spending for both traditional and charter schools. In the 2017-18 school year, the median charter school spent $521 less per pupil than the median traditional school. The difference in the median school’s per pupil spending is largely driven by the 6 charter schools (12 percent of charters) that spent less than $7,000 per pupil, otherwise the distribution of spending per pupil looks similar between traditional and charter schools.

Figure 2: 2017-18 Total Spending Per Pupil

Screen Shot 2019-08-07 at 11.41.18 AM

While overall expenditures look quite similar, there are interesting differences in how charter vs. traditional schools spend their dollars. As we did in our earlier posts on school-level spending, we can also break spending out by activity/purpose. Figure 3 shows the distribution of per pupil spending for instruction, which includes things like teachers’ salaries, books, etc. In the 2017-18 school year, the median charter school spent $4,364 on instruction while the median traditional school spent $5,520, a difference of $1,156 or about 20 percent compared to the median traditional school. Since the majority of instructional expenditures are for teacher salaries which are linked to experience, it’s likely that charter schools’ lower average teacher experience is a big driver of this expenditure difference.

Figure 3: 2017-18 Per Pupil Spending for Instruction

Screen Shot 2019-08-07 at 11.41.43 AM

Charter schools spent less on instruction, but more on administration. Figure 4 provides the distribution of per pupil spending on administration. The median traditional school spent $760 per pupil on administration, including both school and central administration costs, while the median charter spent $1,241, a difference of $481 or about 60 percent compared to the median traditional school. The difference in administration expenditures is not very surprising given the fact that several charter schools are still starting up and the median charter school is about 40 percent smaller than the median traditional school. New charter schools generally start small and grow over time, but school leadership needs to be in place from the beginning. Because administration costs are spread over a smaller student base early on, administration costs are often higher per pupil in a school’s startup phase.

Figure 4: 2017-18 Per Pupil Spending for Administration

Screen Shot 2019-08-07 at 11.41.57 AM

Charter schools spent more on student support services like physical and mental health services, guidance counselors, etc. and less on instructional staff support like teacher professional development. Figure 5 shows the distribution of expenditures on student support services. The median traditional school spent $483 per pupil and the median charter spent $398, a difference of $85 or 20 percent compared to the median traditional school.

Figure 6 illustrates the distribution of expenditures for instructional staff support. The median traditional school spent $716 per pupil on instructional staff support while the median charter spent $438, a difference of $278 or 40 compared to the median traditional school.

Figure 5: 2017-18 Per Pupil Spending for Student Support Services

Screen Shot 2019-08-07 at 11.45.16 AM

Figure 6: 2017-18 Per Pupil Spending for Instructional Staff Support

Screen Shot 2019-08-07 at 11.42.25 AM

Overall traditional school and charter school expenditures per pupil are quite similar; however, there are interesting difference in how they spend those dollars. In the 2017-18 school year, charters spent somewhat less on instruction, owing to their younger, less experienced workforce, but they spend more on administration. Charters also spent more on student support services but less on instructional staff support.

While the state funding formula largely determines the amount of funding schools receive, districts and schools have a lot of discretion around how those dollars are spent. The distribution charts above show that schools spend their dollars in a variety of ways, and the differences across traditional schools are generally bigger than the differences between traditional schools and charter schools. There is no “right” way to spend education dollars. The context and the student population matter a lot. That’s why we are big fans of local decision making as long as it is strategic and student focused.

Why are ACT Aspire scores flat?

In The View from the OEP on July 31, 2019 at 11:42 am

We have been wondering why Arkansas’ most recent ACT Aspire scores are essentially the same as they have been for the past two years.   As seen in Figure 1, the preliminary scores from the spring 2019 ACT Aspire show that statewide, student performance is flat, with fewer than half of Arkansas’s students meeting grade-level expectations in math, science, and reading.

The lack of improvement in these scores is confusing because funding for Arkansas public schools continues to increase (currently $6,899 per student annually),  class sizes are small (an average of 15 students per class), and teachers are being provided professional development, particularly in how to teach reading.  We don’t have any other states to check in with and see how they are doing on the ACT Aspire since we are the only state administering it right now.

Figure 1. Percentage of Arkansas students meeting or exceeding expectations on the ACT Aspire, by content area, 2016 to 2019.2019 ACT Aspire state level


Is anyone improving?

Although statewide scores have not changed, we wanted to highlight some school districts that have demonstrated consistent growth across subjects over the past years. Table 1 highlights these districts, the percentage of students who met or exceeded grade-level expectations in 2018-19, and the percentage point increase in the percentage meeting expectations since the 2015-16 school year.

Table 1. Districts demonstrating consistent increases in the percentage of students meeting or exceeding expectations on the ACT Aspire, by content area, 2016 to 2019.

ACT Gains

It is interesting to note the percentage of students in the districts participating in the Free/ Reduced Lunch program (FRL- a proxy measure of student poverty).  Generally,  districts serving more economically advantaged students have higher rates of students meeting expectations, but we can see variation even among these districts with consistent improvement in performance.  For example, Booneville, Cross County, and Jessieville all serve student population in which 72% of students are eligible for FRL, but proficiency rates vary between them.


Dig into the Data!

There’s so many different variables to consider as we try to make sense of the most recent ACT Aspire scores.  We know you want to dig into the data yourself and see how your district or school is doing so we have created interactive data visualizations. Maps are available for both district and school-level, and you can select specific districts/schools and see how they scored and whether they have improved performance over last year.  There are subject-specific maps available (tabs above the main map) if you want to see one content area in particular. You can also use filters to find schools/ districts with similar rates of economically disadvantaged students enrolled and serving students in similar grade levels.

2019 ACT Aspire school level

In these maps, we use the OEP GPA as the overall indicator of performance. The OEP’s GPA is a weighted measure of student performance that gives the most credit to students who have exceeded expectations and the least credit to those that are in need of support. In this GPA measure, we treat the ACT Aspire test scores similar to the familiar grade point average for individual students: 1.0 is the lowest score, indicating that all students in a districts were In Need of Support, while 4.0 is the highest score, indicating that all students in a districts were Exceeding Expectations on the ACT Aspire. OEP’s GPA is more sensitive to changes in student performance than the % Met Expectations value, and although calculated slightly differently, OEP’s GPA is highly correlated with the ESSA Weighted Achievement measure (r=0.98 in 2018), and the overall ESSA index score (r=0.96 in 2018).

As you can see in the maps above, schools in the upper left hand corner of the state are more likely to be blue, indicating a higher OEP GPA. This is not surprising since we are showing performance on the ACT Aspire, which is correlated with the percentage of students in the school participating in the Free/ Reduced Lunch program (a proxy measure of student poverty), so more economically advantaged areas of the state generally have higher scores.


Digging Deeper Into ACT Aspire

We now turn to examining each content area by grade level.

While overall ACT Aspire results did not change, there were some changes by grade level within each content area. Although we prefer the OEP GPA,  here we use the percent meeting or exceeding benchmarks so we can compare Arkansas performance with national results. When interpreting National Averages, however, it is important to remember that Arkansas students make up an ever increasing percentage of the “national” three – year rolling average.  This is because Arkansas is the only state currently administering the ACT Aspire in grades 3-10.  Alabama and South Carolina, the other states with significant representation in the norming sample, stopped administering the ACT Aspire in 2017 and 2015, respectively.


Mathematics

Figure 2. Percentage of Arkansas students meeting or exceeding expectations on the ACT Aspire Math, by grade, 2016 to 2019.

2019 ACT Math_Grade

Math: We see consistent increases in the percentage of 3rd, 8th, 9th, and 10th graders meeting or exceeding expectations. The increases in 3rd grade performance are important as they might foreshadow higher proficiency levels in later grades (although that pattern isn’t evidenced so far). The continued improvement in 8th-10th grades is great news, since these grades returned the lowest math proficiency rates in the first year of testing.


Science

Figure 3. Percentage of Arkansas students meeting or exceeding expectations on the ACT Aspire Science, by grade, 2016 to 2019.

2019 ACT Science_Grade

Science: Performance in science was generally consistent with prior performance, although there was a slight increase in 9th grade proficiency.  The proficiency of 6th grade had been declining, but held steady this year.


English

Figure 4. Percentage of Arkansas students meeting or exceeding expectations on the ACT Aspire English, by grade, 2016 to 2019.

2019 ACT English_Grade

English: Performance in English was generally consistent with prior performance, although there were slight declines in some grades.

 


Reading

Figure 5. Percentage of Arkansas students meeting or exceeding expectations on the ACT Aspire Reading, by grade, 2016 to 2019.

2019 ACT Reading_Grade

Reading: We can see consistent increases in the reading proficiency in grades 3-5, although these gains are offset by performance declines in the high school grades.

We got to wondering about the performance of the Outstanding R.I.S.E schools from 2018, and if their training in and successful implementation of the R.I.S.E strategies would be evidence in increased reading proficiency for the third graders assessed in 2019. The 3rd grade students tested last spring would have been 2nd graders in the first year of R.I.S.E Arkansas. Figure 6 displays the 3rd grade reading performance of these R.I.S.E. schools and the state overall since ACT Aspire testing began in 2015-16.

Figure 6. OEP GPA for the ACT Aspire 3rd grade reading, 2016 to 2019.

RISE Reading

In 2015-16, schools that would be identified in 2018 as Outstanding R.I.S.E. schools had the same 3rd grade reading performance as 3rd graders across the state as a whole.  This is great because it shows that these RISE schools were scoring no better or worse that the average school across the state. The reading performance in the schools that would be identified as R.I.S.E schools in 2018, began to decline in 2016-17 and 2017-18, while the statewide 3rd grade reading performance continued to increase slightly. In 2018-19, however, there was improved reading performance among 3rd graders in the R.I.S.E schools.  Although we can’t say for sure that the recent increase in 3rd grade reading performance is due to the implementation of R.I.S.E strategies, it is a positive sign that there is improvement in these schools!


So- what are the big takeaways from the preliminary 2019 ACT Aspire results? 

  • Performance in all subject areas is generally the same as last year (and the year before..), although some districts are demonstrating consistent improvement.
  • Performance on the ACT Aspire is related to school/ district poverty rates.
  • OEP GPA is highly correlated with Weighted Achievement and ESSA Index scores.
  • You can use the data resources from OEP to see overall district and school values.
  • Data visualizations can help us see statewide patterns in performance and compare performance to other schools/ districts of interest.

We are happy to be able to share these resources with you and looking forward to seeing the Growth Scores (our favorite!) on the ESSA reports in October. Stay tuned to OEP for more info, and let us know if there’s something you want us to check out!