The last month has been chock-full of information about how Arkansas schools are performing, and we have gotten a lot of questions about how to interpret the flood of sometimes confusing (and sometimes conflicting) information. Yesterday, U.S. News & World Report released their annual “Best High Schools” rankings, and some folks are wondering what does the ranking mean and how it compares to the A-F letter grades and ESEA school status information released last month?
Our answer: You can only judge the information if you understand what is being measured. This blog post is going to summarize how U.S. News & Wold Report rankings are calculated and compare them to the letter grades and ESEA status information. Arkansas had 102 high schools ranked by U.S News: 1 gold medal, 23 silver medals, and 78 bronze medals. Among the “top 10″ Arkansas high schools, state letter grades ranged from A to C, and nine of the ten were identified as Needs Improvement. Compared to last year, even our “Best” high schools fell in the national rankings behind other high schools across the country.
First, congratulations to those Arkansas high schools that made the Best High School list! Below are the Top 10 in Arkansas:
#1: Haas Hall Academy
#2: Bentonville High School
#3: Rogers High School
#4: Lakeside High School
#5: Rogers Heritage High School
#6: Benton County School of the Arts
#7: Parkers Chapel High School
#8: Centerpoint High School
#9: Prairie Grove High School
#10: KIPP: Delta Collegiate High School
What do the U.S. News rankings mean?
Before we answer, keep in mind that the U.S. News rankings are based on state assessment data from the 2012-13 school year, so the ranking is reflecting student performance from nearly 2 years ago. There are three aspects to the ranking: 1) the performance of students on state assessments in literacy and mathematics; 2) the performance of disadvantaged student subgroups; 3) the degree to which high school prepare students for college by offering a college-level curriculum.
Schools must pass the first step by performing better than expected based on their student population in order to continue in the ranking process.
STEP 1: Identify High Schools Performing Better than Expected
To determine if schools are performing better than expected, U.S. News created a Performance Index for each high school by examining student performance on state assessments, and compared it to the percentage of students participating in Free/Reduced Lunch Programs (which are an indicator of low socioeconomic status). This model reflects the understanding that students who face economic challenges outside of school are typically less likely to achieve at the same levels at their peers who do not face economic hardships. We are going to skip the details, but you can read more about it here.
The figure below represents Arkansas high schools’ school-level Performance Index scores plotted against the school percentage of students participating in Free/Reduced Lunch Programs. Blue dots on the graph represent schools performing higher than expected, red dots are schools performing as expected, and green dots represent high schools performing lower than expected given the percentage of students participating in FRLP (FYI- we would have made the color-coding more representative of performance!)
The blue dot on the far left side is easily identified as Haas Hall because they are the only high school in the state that reports 0% of students participating in FRLP. The Performance Index for Haas Hall is 140, which is 20 points above the expected performance. As you move to the right side of the graph, the percentage of students participating in FRLP increases. At the far right hand side of the graph are dots representing schools with 100% of students participating in FRLP. The highest blue dot on the right hand side shows a school whose enrollment is entirely low-income, but whose Performance Index is nearly 40 points higher than expected!
Only schools whose Performance Index is ABOVE the light blue performance zone are represented by blue dots and passed on to the next step. This is the critical step for Arkansas high schools. The majority of Arkansas schools do no pass this step, and are unranked. This year, 110 (39%) of Arkansas high school were performing above expectations and move on to Step 2 of the ranking.
STEP 2: Identify High Schools Performing Better than State Average for Their Least Advantaged Students
For this step, the performance of African American, Hispanic and economically disadvantaged students on the state assessments are compared to state averages. Schools where these disadvantaged students are performing as well or better than state averages are automatically considered bronze-medal high schools and move on to Step 3 of the ranking to determine silver or gold medal.
STEP 3: Identify High Schools That Performed Best in Providing Students with Access to Challenging College-Level Coursework
For this final step, the participation of 12th grade students in AP or IB examinations is examined.
How do the rankings compare to ESEA status and letter grades?
Rankings, status and letter grades are based on different years and different models, so they aren’t directly comparable but may add to the understanding of a school’s performance over two years and multiple measures of performance.
The Big Difference is that the letter grades and status are based on the most recent data (from 2013-14), while the the U.S. News rankings are based on state assessment data from the 2012-13 school year.
Another difference is that letter grades (A-F) for high schools incorporate graduation rate for high schools! The number of performance targets being met by the school and the gap between at-risk students and their peers who are not at risk are also considered in the letter grade model. If you want to know more details about the letter grades, read our policy brief.
ESEA status measures if schools are meeting individualized performance targets, and assigns Achieving only to schools that are meeting all performance targets.
Even though the data are from different years and use different criteria, we know that you are still interested in the letter grades and status for those top 10 high schools in the state, so they are presented below:
Arkansas Ranking (US News): School Name: A-F Letter Grade: ESEA Status
#1: Haas Hall: A: Achieving
#2: Bentonville High School: B: Needs Improvement
#3: Rogers High School: B: Needs Improvement
#4: Lakeside High School: A: Needs Improvement
#5: Rogers Heritage High School: C: Needs Improvement
#6: Benton County School of the Arts: A: Needs Improvement
#7: Parkers Chapel High School: B: Needs Improvement
#8: Centerpoint High School: C: Needs Improvement
#9: Prairie Grove High School: C: Needs Improvement
#10: KIPP: Delta Collegiate High School: B: Needs Improvement
So…what does it all mean?
It is challenging to measure school performance. There are lots of different models, and to determine if it is important to you and your community you need to understand what is being measured.
It is not surprising that there is variation between the U.S. News rankings, the letter grades, and status given that they use different data and represent different criteria. It is interesting that there are no ‘D’ or ‘F’ schools in the top 10. It is also informative that only 1 of the “top 10″ schools were identified as Needs Improvement under ESEA status, reflective of how few schools across the state were identified as Achieving in 2014. This is one of the reasons why here at the OEP, we appreciate the Letter Grades – it includes more varied criteria and provides more information about student performance than “Needs Improvement.”
In the same thought, we like the U.S. News rankings because it provides information that can be helpful! We want to know which high schools are performing better than expected, serving their most disadvantaged students and preparing kids for college. We also like being able to compare to other high schools across the country. It is a somewhat clumsy comparison, howvever, since each state currently uses a different test to measure performance, and we look forward to the day when cross- state comparisons are facilitated by common assessments. We DON’T like that the data used by U.S. News are nearly two years old and hope that stakeholders will keep that in mind as they search for their school on the “Best” list.