University of Arkansas Office for Education Policy

OEP News and Another C- for Arkansas

In The View from the OEP on January 17, 2018 at 11:58 am

 

OEP Logo JPEG-2

 

OEP News!

 

Our big news is that Gary Ritter is leaving us in August to take on the role of Dean of the School of Education at Saint Louis University in St. Louis, MO. Since OEP’s inception in 2003, Gary has been the driving force behind and director of the Office.  We are sorry to see him go, but excited for him to expand his excellent work to our neighbor state! Gary will continue to share his insightful perspective and passion for improving education as a member of the OEP advisory board, and OEP will continue to support for Arkansas education stakeholders under the direction of OEP Executive Director Sarah McKenzie.

Make plans to visit with Gary and Sarah at OEP’s conference this April in Little Rock- tentatively scheduled for Tuesday, April 24th.  We are focusing on Teacher Pipeline issues and have some great information that you won’t want to miss so save the date!

OEP is changing our weekly news roundup/ blog/ report releases from Wednesday to Tuesday. You can get these automatically by emailing oep@uark.edu with “Sign me up” in the subject line.

v37-17-quality-counts-2018-web-cover-image-200

Quality Counts

 

Today Education Week released their annual Quality Counts report, which grades each state on their education performance.  This year Arkansas (again) received an overall grade of C- and is ranked 43rd overall.  Arkansas has received a C- for the last four years as our national ranking slipped from 36th in 2015, to 41st in 2016, to the current 43rd in 2017 and 2018.

As we have discussed in previous blog posts there are several issues with the grading system, and Arkansas’ scores have remained stagnant.This year’s report includes summative grades and rankings for states on education indicators.  The report finds “paints a portrait of middling performance overall with patches of high achievement, along with perennial struggles to improve on the part of states mired at the bottom.”

A state’s overall grade is the average of its scores on three separate indicators: Chance-for-Success, K-12 Achievement, and School Finance. Previous blog posts have discussed the flawed nature of the grading system, and this year’s grade is even less informative due to the use of old academic data (from the 2015 NAEP because of a delay in the release of the 2017 NAEP results), old financial data (2015 is the most recent year data were available), and the difficult-to-impact state data including annual income and percentage of adults with a two- or four-year degree.

Below are the most recent four years of  Arkansas grades in each of the categories considered for 2018.

Quality Counts Categories AR Grade 2015 AR Grade 2016 AR Grade 2017 AR Grade 2018
Chance for Success C- C-  C- C-
School Finance C C-  C- C-
K-12 Achievement D+ D  D*  D*

* Note: K-12 Achievement data are unchanged from the 2016 Quality Counts report

It is time to grade ourselves…

 

Quality Counts is EdWeek’s measure of educational quality, but here at OEP we don’t think it accurately captures all the strengths and areas for improvement.  ESSA allowed states to develop measures of student achievement that are meaningful to them, and Arkansas’ plan was approved yesterday!

Now that Arkansas’ ESSA plan is approved, we look forward to its implementation.  One of the first orders of business is to determine the details for how Arkansas will assign  school grades (A-F) to be used in the school rating system and the school recognition program.

We believe that if policymakers and education leaders focus on meaningful data, like student achievement AND growth, equity and efficiency in the face of disadvantage, and post-secondary transitions, students in Arkansas can continue to improve and reach greater levels of educational and lifelong success.

Advertisements

Where do Arkansas Students Learn the Most?

In The View from the OEP on December 13, 2017 at 2:06 pm

GravetteGravette school district has the highest rate of student growth in the state, according to data released last week from researchers at Stanford, with students learning the equivalent of almost a whole extra year in school between 3rd and 8th grade! You can see how well your school district did here.

The new study examined rates of achievement and growth for nearly every school district in the country using 300 million scores on 3rd through 8th grade English/Language Arts and math assessments from 2008-09 to 2014-15. (If you are interested in learning how the researchers were able to compare scores across states- see notes at the bottom of this blog.)

Achievement represents how well students score on state tests, while growth reflects the rate of change in achievement over time. Examining growth allows us to see how students’ scores change while students are in school, essentially measuring how much students are learning.

To determine where kids ‘started’, the researchers examined the average third grade achievement in reading and math.  The measure represents the average third grade achievement across seven years (2008-09 to 2014-15). In the map below, green represents districts scoring above the national average grade achievement, and purples represents districts scoring below average. Darker colors indicate greater difference from the average.

Figure 1. Average Third Grade Test Scores, 2009-2015NAtional G3

When we zoom in on Arkansas below, we see a mixture of green and purple indicating varied achievement throughout the state. In a few districts, 3rd graders are scoring the equivalent of 1.5 to 2.5 grades above the national average (represented by the darkest green), while 3rd graders in a some districts are scoring the equivalent of 1.5 or more grades below the national average (represented by the darker purple colors).

Achievement_g3_2b

ach legend

The orange circle on the figure indicates Spring Hill school district, where third graders were the highest achieving in the state, on average.  Appearing in dark green, Spring Hill students were more than 2.5 grade-equivalent units above average national achievement.

The red circle on the figure represents Gravette school district.  Colored light purple, Gravette third graders scored about half a grade below average, placing the district at the 43rd percentile for achievement.

Now that we know how well Arkansas’ students are achieving at the earliest tested grade, let’s see what the researchers found about how they grew while in school. The growth measure represents changes in reading and math achievement from 2008-09 to 2014-15 for cohorts moving from grades 3 through 8. In an effective school system, we would expect achievement to increase over time. The map below represents the rate of growth, with green being above national average growth rate, and purple being below average growth rate, with darker colors indicating greater difference from the average.

Figure 2. Average Test Score Growth Rates, 2009-2015

NAtional_growth

When we zoom in on Arkansas below, we see much more purple in the growth figure than we did in the performance figure. Throughout the state, only 19 school districts demonstrated growth above the national average (identified in green).  In the remaining 92% of districts in the state (identified in purple), students learned at a rate of less than one year per year.

growth_AR_2bgrowth legend

The orange circle again indicates Spring Hill school district, which, as we indicated previously, had the highest third grade achievement.  Appearing in dark purple in the growth figure, however, Spring Hill scores in subsequent years reflect very low growth rates over time.

The red circle represents Gravette school district.  Although third grade students were not initially high achieving, they grew at a faster rate than students in any other Arkansas school district. Colored dark green, students in Gravette experienced 5.9 years of growth in five years, ranking them in the 96th percentile in the country for growth in student learning.  Students learned the equivalent of almost a whole extra year in school by the time they were in 8th grade! Way to go Gravette!

If you are thinking that high growth is only possible for districts where initial student performance is low, consider the case of Bentonville and Brookland. Third graders in both of these districts score nearly a grade level above average, and serve relatively advantaged student populations (26% and 36% of students eligible for Free/ Reduced Lunch, respectively).  So, although these districts have similar third grade student achievement, and serve similar populations, there are large differences in student growth by 8th grade: in one district the growth is high (5.3 years in 5 years, which is at 78th percentile nationally) and in the other it is low (2.7 years in 5 years, which is at the 11th percentile nationally). Growth rates for all districts are available here

There is a weak and negative relationship between third grade achievement and growth.  The Stanford researchers report a correlation of -0.13 nationally, but the in-state correlation for Arkansas is somewhat stronger at -0.19. The figure below illustrates the growth and 3rd grade achievement of Arkansas’ school districts.

Figure 2. Achievement Growth Rates by Grade 3 Achievement, Arkansas  2009-2015

AR Growth_Achievement_2a

You can see that no district where third graders are highest-achieving (more than one grade level above average) exceed average national growth.  You can also see, however, that only two of the districts where third graders were the lowest-achieving (less than one grade level below average) exceed average national growth.

You might be wondering about how charter schools performed.  In this research, charter schools are not reported separately as they were collapsed into the geographic district in which they were located.  While there were relatively few charter schools operating in Arkansas from 2008-09 to 2014-15, it is important to consider the possible impact on district achievement and growth outcomes.  Also remember that these data do not include any information about learning experiences happening before 3rd grade or after 8th grade, including high school graduation or college readiness. Test performance is a proxy for opportunity and achievement, and can be affected by many factors including what students have been taught and have learned, and how motivated they are to perform.

The big takeaways from this new research are:

Achievement: Arkansas school districts vary in 3rd grade achievement (we knew that) but the achievement is not consistently behind the nation (shown by all the green in the achievement figure).

Growth: Arkansas school districts vary in growth achievement and the rate of growth is consistently behind the nation (shown by all the purple in the achievement figure).  There are some districts, however, with growth rates that are above the national average.

The researchers recommend using caution when interpreting growth rates as pure measures of school effectiveness:

“It is tempting to think of growth rates in test scores as a rough measure of school district effectiveness. This is neither entirely inappropriate nor entirely accurate. The growth rates better isolate the contribution to learning due to experiences during the schooling years. Grade 3 average scores are likely much more strongly influenced by early childhood experiences than the growth rates. So the growth rates are certainly better as measures of educational opportunities from age 9 to 14 than are average test scores in a school district. But that does not mean they reflect only the contribution of schooling. Other characteristics of communities, including family resources, after school programs, and neighborhood conditions may all affect growth in test scores independent of schools’ effects.” (pg. 26)

Nonetheless, growth rates are closer to a measure of school effectiveness than average test scores. It is not unreasonable to think that the growth measures carry some signal regarding school quality. In particular, learning what conditions make high growth rates possible, and how we can spread those conditions to all students throughout the state.

 


Note on scaling:  Since states administer different tests and set their own standards regarding ‘proficiency’,  comparing student performance nationally required the researchers to create an achievement scale that was comparable across states, years, and grades. Using data from the National Assessment of Education Progress (NAEP), which is taken by a sample of students in every state every other year, researchers placed state achievement on a common scale. You can read more about this here

 

Hey School Board: Is Your District Successful?

In The View from the OEP on December 6, 2017 at 12:27 pm

asba

Here at OEP we think an informed school board is critical to the success of students in the district, so we are looking forward to presenting at the Arkansas School Boards Association annual meeting tomorrow in Little Rock.   Our presentation, “Is Your District Successful? Do You Have Proof?” is built to provide a road map for determining if your district is getting the job done.

Although sometimes folks think that the school board’s attention should be limited to the financial aspects of the district, under Arkansas law A.C.A § 6-13-620 (2012), school boards are tasked with several responsibilities, two of which are:

  • “Understand and oversee school district finances required by law to ensure alignment with the school district’s academic and facility needs and goals” and
  • “Do all other things necessary and lawful for the conduct of efficient free public schools in the school district”

So, school boards need to conduct EFFICIENT free public schools, or schools that achieve maximum productivity with minimum wasted effort or expense. To determine if their schools are efficient, school boards need to examine both the inputs (financial resources supporting learning) and the outcomes (such as academic achievement, student growth, and graduation rate).

But how do school boards know if the community is getting a good return on the investments they are making?  Primarily, the board needs to determine a reasonable comparison group of districts to get a reasonable frame of reference for efficiency.

How do you pick which districts make a good comparison group? In our experience,  there is rarely a perfect match.  So here at the OEP, we choose districts about the same size with a similar percentage of students eligible for free or reduced lunch (FRL).  We focus these two characteristics because the percentage of students eligible for FRL is a proxy for students academically at risk, and in many of our analyses district size impacts academic outcomes.

We suggest you pick 5-8 districts that ‘look like’ yours- You can find this information here and easily sort the results based on:

1) Enrollment: Select a few districts that are slightly larger and a few that are slightly smaller.

2) % Eligible for Free or Reduced Lunch (FRL): Select a few districts that have higher FRL and a few that have lower.  Stay as close as you can- but ideally within a 10 percentage point difference above or below.

3) % Limited English Proficient (LEP): If you are one of the 90% of the districts in Arkansas that enrolls 10% or fewer LEP students, you don’t need to consider this, but if your district more that 10% English Language learners, you may want to consider this in your match since these students may face unique academic risks.

Sometimes districts want to be compared to who they play in football, or who is in their region, or who is down the street.  That’s fine, but START with similar districts.

Compare Inputs:

School boards may want to start by considering the inputs, or investments being made into the school district.

Here at the OEP, we suggest comparing per-pupil revenue, per-pupil expenditure (total and net current) and per-pupil instruction expenditures. You can access the data here, and put them in a chart like the one below.  It helps the interpretation to order the districts from greatest %FRL to least %FRL.

finances

We see that our ‘home’ district (represented by the gray bar) has a lower per-pupil revenue than similar districts, is spending average amounts per pupil overall, but is spending the least of comparison districts on instruction per pupil.

Another key input that the district will want to consider is teacher salary. The most recent teacher salary scales for each district are available here. Last week, the OEP released a report examining several comparisons of teacher salary (read the blog) and you can access the salary measures data here.

teacher salary

We see that our ‘home’ district is paying competitive salaries overall, so should be able to attract and retain quality teachers.

School boards should also consider comparing to similar districts on Student: Teacher Ratio, to determine if staffing is comparable.  We find that increasing student teachers ratio (even by one student) can allow more resources to fund higher teacher salaries.


Compare Outcomes:

School boards should also examine the outcomes of students in their districts. A key academic outcome is the percentage of students meeting grade level expectations on annual assessments.

Use ACT Aspire district-level data to compare student performance in ELA, Math, and Science.  You can access these data here, and put them in a chart like the one below.  It helps the interpretation to order the districts from greatest %FRL to least %FRL.  Interestingly, in this example, the district with the greatest % FRL (represented by the far left hand bar) often scores higher than districts serving fewer at-risk students.

ACT Aspire

Unfortunately, we see that our ‘home’ district (represented by the gray bar) has a lower percentage of students Meeting or Exceeding Expectations on the ACT Aspire than comparison districts of a similar size and serving similar rates of at-risk students.

Other outcome data that school boards should consider comparing to similar districts are:

  • NEW!  Just released yesterday from Stanford- see how your district’s student growth compares.  You can type in the name of your district in the box near the bottom of the article and check it out (more on this in next week’s blog)!
  • Student growth and school letter grades: Here at the OEP, we like to compare how much kids are growing in schools.
  • ACT: Now that all districts are testing all 11th graders, the results are comparable between districts, and we like the “percentage meeting readiness benchmark” as the indicator.
  • High school graduation rates: Since this measure has in been place for a while, we suggest looking at three years of graduation rates, for the overall population and for students at-risk (TAGG)
  • College-going rates: (NOTE: this rate only reflects graduates that go to school in state- you can read our thoughts about that here)

 


After examining a variety of inputs and outcomes for the school district, school boards should ask themselves: Are the schools achieving maximum productivity with minimum wasted effort or expense? 

If the answer is ‘No’, it is the board’s task to instigate change.

Developing a strategic plan will mean getting deeper understanding of where the challenges are, identifying clear, measurable (and reasonable) goals for the district, and making necessary changes to support the achievement of those goals. Turns out, focusing on the finances may not ensure that students in your district are getting the education the community expects.

Let us know how we can help!