University of Arkansas Office for Education Policy

Is Your High School one of the “Best”?

In The View from the OEP on May 16, 2018 at 2:04 pm

US BadgeLast week, U.S. News & World Report released their annual “Best High Schools” rankings, and we want to clarify what the rankings mean, share some thoughts about what we like (and don’t) about the methodology, and compare the rankings to Arkansas’ ESSA scores for schools.

First, congratulations to those Arkansas high schools that made the list for 2018!  We made it easy for you to find the US News information for all Arkansas high schools here. Below are the US News Top 10 for Arkansas (we added the Free/ Reduced Lunch Rate) :

#1: Haas Hall Academy (does not participate in FRL program)

#2: eStem High School (30% FRL)

#3: LISA Academy North High (35% FRL)

#4: Prairie Grove High School (35% FRL)

#5: Bentonville High School (23% FRL) 

#6: Rogers High School (52% FRL)

#7: LISA Academy High (40% FRL) 

#8: Arkansas Arts Academy High (26% FRL) 

#9: Fayetteville High School (35% FRL) 

#10: Scranton High School (47% FRL)

These rankings always make the news, but here at the OEP, we want to make sure that you understand what the “best” title is based on. There are four steps used to identify high schools that are performing better than expected:

STEP 1 | Students exceeded expectations in their states.

STEP 2 | Underserved students performed better than the state average.

STEP 3 | Student graduation rates met a threshold of 80%.

STEP 4 | Students were prepared for college-level coursework.

Schools must pass the first step by performing better than expected based on their student population in order to continue in the ranking process.


Things we like about the rankings:

1. Performance on state exams factors in the economic background of the students served by the school.

Schools serving a lower percentage of students who are economically disadvantaged typically have higher scores than schools serving a higher percentage of economically disadvantaged students, but the US News ranking takes that into account.

The figure below represents Arkansas high schools’ school-level Performance Index scores plotted against the school percentage of enrolled students participating in the Free/Reduced Lunch Program. You can see the relationship between Performance and the percentage of students who are identified as economically disadvantaged. Schools who do not participate in the Free/Reduced Lunch Program are assigned the average FRL participation rate for the state.

The red line represents the ‘typical performance’ of schools in Arkansas given the percentage of students in the school that participate in the FRL program.

Dark green markers indicate schools where students performed BELOW what is typical for schools with the same percentage of economically disadvantaged students.

Light blue markers represent schools where students performed AS expected.

Yellow markers represent schools where students performed BETTER than expected and are selected for initial consideration for a “best” high school.

US Best

On the upper left hand side of the figure above, you can see a yellow marker indicating NWA Classical Academy where 13% of the students participate in FRL with a performance index of 92.  On the far right hand side of the figure you can see Clinton High School represented by the yellow marker with a performance index of 71 and 100% of students participating in FRL. Clinton High School is one of many schools identified as 100% FRL due to the Community Eligibility Provision (CEP).

Ninety- nine high schools were found to be performing above expectations, an increase of 8 school compared to last year. The schools with the highest performance indices (129 and 121) are Haas Hall Fayetteville and Bentonville. These schools are assigned the average FRL participation rate for the state as they do not participate in the Free/Reduced Lunch Program.

This year 8 more high schools performed above expectations!

2. Performance of historically underserved populations is considered.

Students who are black, Hispanic, and low-income traditionally score lower on state assessments than white, Asian, and/or economically advantaged students.  Only schools where these students outperformed the state average were selected for further consideration.

3. AP passing rates are considered as well as AP participation.

Under Arkansas’ ESSA plan, the number of students taking an AP class is rewarded, but there is no consideration to how well students perform on the AP tests.  This is particularly important because, unlike students in most other states, Arkansas students do not have to pay to take AP tests, so we can consider the passing rates a more reliable measure of how well the AP content is being taught.


Things we don’t like about the rankings:

1. The data are OLD

The data used by U.S. News are from the 2015-16 school year and are nearly two years old.  We hope that stakeholders will keep that in mind as they search for their school on the “Best” list.

2. Focus is just on College

Only ‘college ready’ indicators are considered.  We would like to see US News including more indices of career readiness, because not everyone wants to go to college and the ‘best’ high schools should meet the learning goals of all of their students.

3. Focus is on Proficiency, not Growth

Here at OEP, we are strong proponents of student-level growth models.  We understand that it is impossible to compare this type of student growth across states for now, because each state has a different assessment, but we feel it is important to point out that even though they consider the demographics of the students served by the school, Arkansas’ growth model provides better information about how well students are GROWING.


How do these US News rankings compare with Arkansas’ letter grades?

We wondered how the US News rankings compare to the recently released Arkansas grades for schools? It is important to note three major differences:

  • Arkansas’ scores and letter grades are based on 2016-17 data, while US News is based on data from 2015-16
  • Arkansas uses a student-level growth model in the calculation while US News does not.
  • Arkansas’ scores use a weighted performance measure that is different from US News:  While the weights are used at the lower levels, US News weights the highest performers at 1.5, while Arkansas uses a weight of 1.25

Below we provide the 2016-17 Arkansas ESSA information for the US News Top 10 high schools.  You can find this information for all Arkansas school here.  (Arkansas Achievement and Growth ranking is within 303 Arkansas high schools)

#1: Haas Hall Academy: Grade A, 1st in Achievement, 7th in Growth

#2: eStem High School: Grade A, 54th in Achievement, 19th in Growth

#3: LISA Academy North High: Grade A, 34th in Achievement, 2nd in Growth

#4: Prairie Grove High School: Grade B, 81st in Achievement, 76th in Growth

#5: Bentonville High School: Grade A, 16th in Achievement, 81st in Growth

#6: Rogers High School: Grade C, 127th in Achievement, 193rd in Growth

#7: LISA Academy High: Grade B, 110th in Achievement, 22nd in Growth

#8: Arkansas Arts Academy High: Grade A, 5th in Achievement, 9th in Growth

#9: Fayetteville High School: Grade B, 62nd in Achievement, 186th in Growth

#10: Scranton High School: Grade B, 100th in Achievement, 47th in Growth

Even though the data are from different years and use different criteria, almost every one of the US News high schools received a letter grade of A or B.  They vary in terms of achievement and growth, and you KNOW we love to focus on growth, but overall the indicators seem to be pointing in the same direction.  It will be interesting to see how the Arkansas and US News rankings line up for the 2016-17 school year.

Differences in methodology aside, the top 10 are some of the ‘Best’ high schools in Arkansas!

 

 

Advertisements

Growth, Poverty, and the Recognition Blues

In The View from the OEP on April 17, 2018 at 4:17 pm

school moneLast week, the ADE released a bunch of information about Arkansas schools, including A-F letter grades, state report cards, and ESSA reports.  Here at OEP, we feel that growth scores are the most important piece of information that was released, and today we want to share some more details about why growth is so important, and why some deserving schools may have missed out on the recognition and reward money.

Growth and Poverty

Growth is so important because it gives a different perspective on how well students are learning in a school, and is not as correlated with student demographics as achievement is. In the graph below, we present the weighted achievement scores and the % of FRL students enrolled at the school (a proxy for poverty).  Weighted achievement scores range from 2 to 105, and FRL rates range from 100% of students eligible to fewer than 5% (note: Haas Hall does not report FRL %ages and so are excluded from the graph).

The values are related in the way that we would expect (a lower percentage of FRL students= higher achievement), and are correlated at R=0.52.  There are some schools that have much higher than typical achievement given the % FRL in their student population, which is awesome, but in general, schools serving more FRL-eligibile students have lower achievement scores.

WA.png

By contrast, below we present the content growth scores and the % of FRL students enrolled at the school.  You can see that the values are not as related  (fewer FRL students doesn’t always mean higher growth), and have a lower correlation at R=0.21.  This is a good thing- because we want kids in all schools to be making growth in learning from one year to the next!

CG

You will also notice that the content growth values are all clustered around 80, making it is hard to tell a difference between ‘high growth’ and ‘low growth’ when the axis is scaled from 0 to 125 like the weighted achievement graph.  This is exactly what we mentioned in the OpEd last week– the growth values have a relatively small range (very small standard deviations) compared to the achievement scores.  Below we share a version of the content and a FRL graph with an ‘adjusted axis’ that runs from 70 to 90, so you can see differences in the growth scores.

CG_adj

With the adjusted axis, you can see differences in content growth scores! There are some schools with low-FRL percentages with high content growth in the upper right corner of the graph.  One example is Willowbrook Elementary in Bentonville, with 15% of students eligible for FRL and content growth score of 89.35.  There are also some high-FRL schools with high content growth scores which are in the upper left corner of the graph. One example is Jones Elementary in Springdale where 98% of students are eligible for FRL, 84% are identified as limited English, and a content growth score of 88.96.  Despite the differences in the student populations served by these two schools, students at both schools demonstrated high growth scores. This is something to celebrate!


The Recognition Blues

Arkansas’ School Recognition Program provides funds for “outstanding schools”.  Schools are rewarded for being in the top 5% (or the 6th to 10th %) in achievement and/or growth.

Given what we know about the relationship between achievement and FRL rates, it should not be surprising that Willowbrook Elementary (with 15% FRL) received reward money for being in the top 5% for achievement, and that Jones Elementary (with 97% FRL) did not. However, both Willowbrook and Jones Elementary received a reward and recognition money for being in the top 5% of content growth among Arkansas schools.

When we were examining who else was rewarded, we noticed that most of the money went to elementary schools. In fact, 59% of the performance rewards, and 65% of the growth rewards went to elementary schools. This made us scratch our heads.

We know that elementary schools are different from schools serving middle and high schools in many ways, but school level also matters when it comes to achievement and growth. In the powerpoint that summarizes the ESSA Indicators, descriptive statistics for each indicator is provided by school level: Elementary, Middle, or High.  You can find the rules for how schools were assigned a level here.  There are substantial differences between the school level groups on achievement scores. For example, let’s examine the achievement score received by schools in the top 5% of each school level.

  • Elementary level = 93.79,
  • Middle level = 91.85, and
  • High school level = 76.53.

The top 5% of elementary schools have higher achievement scores than middle schools and much higher achievement scores (+17 points  or greater than 1 standard deviation) than high schools. It makes sense, then, that about 7% of elementary level and middle level schools were rewarded for highest 5% achievement, but only 1% of high schools received reward money for achievement.

The differences for growth scores between the groups are not as glaring as achievement differences, but remember that the standard deviation is only about 3, so the top 5% of elementary schools have growth scores again about 1 standard deviation higher than middle and high schools.

  • Elementary level = 87.09,
  • Middle level = 84.71, and
  • High school level = 83.94.

We expected to find the top 5% rewards again dominated by  elementary schools, but were surprised to find that 7% of elementary level schools were rewarded for growth along with 5% of high schools. Interestingly, NO middle level schools were rewarded for growth.

We saw that the top 5% of middle schools and high schools have similar growth scores, so why are middle schools not getting recognized?  The recognition program for high schools includes graduation rate, (70% for growth and 30% graduation rate), which generally increases the growth score because graduation rates are typically higher than growth rates. With the deck stacked against them, not even J.O. Kelly, the highest growth middle level school in the state ( J.O. Kelly from Springdale) could crack the top 5% for growth.

The legislation for the reward program clearly states that schools will be rewarded for being in the top tier “of all public schools”, but here at OEP, we would love to see schools  awarded recognition and reward money based on their ranking WITHIN their school level.  Making this change would be more equitable for all schools, and would align more closely with the state’s ESSA plan. If we want to incentivize schools to achieve and show growth, we have to make sure schools in all levels have a chance for rewards and recognition.

Hey! If you want to see how your school ranks within schools serving similar grade levels , check our database. We ranked schools within their school levels, making it easy to identify the elementary schools with the highest achievement scores as well as the high schools with the highest growth scores.

NAEP Nuggets!

In The View from the OEP on April 10, 2018 at 3:31 pm

NAEP results were released today, and Arkansas’ results look about the same as they did in 2015. NAEP is administered nationally to a representative sample of students from all 50 states, so acts as a standard measure of student performance across states and time.

This trend of ‘meh’ was widespread across the country (although Florida had some strong gains!).  Here at OEP, we dug into the new results and are pleased to share six NAEP nuggets with you. You can learn more details in today’s policy brief!

 

NAEP Nugget #1:  Arkansas’ 2017 NAEP scores were essentially unchanged from the 2015 results BUT 2015 was a decline from 2013, so this is not great news because we were all hoping 2015 was a one-year-blip that we would bounce back from. In fact, as the figure below highlights, Arkansas scores were the highest in 2011 and 2013, and have trailed off since. Fingers crossed for 2019!!

NAEP 2017

Arkansas NAEP Scale Scores, 2003-2017

 

NAEP Nugget #2: 4th and 8th grade Math scores are lower than those of Arkansas’ border states (this group includes Louisiana, Mississippi, Missouri, Oklahoma, Tennessee, and Texas).  This is particularly a bummer in 4th grade because we outperformed them from 2005 to 2013!

4thMath

4th Grade NAEP Math Scores, 2003-2017

 

NAEP Nugget #3: 4th and 8th grade Reading scores are also lower than those of Arkansas’ border states. Again, this is particularly a bummer in 4th grade because we outperformed them from 2003 to 2013.

g4 reading

4th Grade NAEP Reading Scores 2003-2017

 

NAEP Nugget #4: Math score gaps between student groups widened in 2017 due to decreased performance of at-risk groups and increased performance of other students.

frlg4math

FRL NAEP Math Score Gap: 4th Grade 2003-2017

 

NAEP Nugget #5: 8th grade Reading score gaps between student groups decreased slightly in 2017, due to an increase in the scores for at-risk student groups.  Although the scores for black and FRL-Eligible students increased, they remain below 2013 levels.

bwg8reading

Black/White NAEP Reading Score Gap: 8th Grade 2003-2017

 

NAEP Nugget #6: ACT Aspire ELA performance is similar to NAEP Reading, but Math proficiency rates are higher for ACT Aspire than for NAEP. We need to pay careful attention to the difference between the NAEP and ACT Aspire math scores.  When we send and receive conflicting messages about how well our students are performing in math, it can make it difficult to determine how well our students are doing and which sorts of educational interventions are making a difference for our students.

ACTASpireNAEP

NAEP Proficiency and ACT Aspire Performance, 2017

We have our fingers crossed that the changes laid out in ESSA will make a big difference to student learning in Arkansas, and look forward to seeing NAEP results again in 2019.

Meanwhile- there is a lot more data coming out this week about Arkansas’ schools- follow OEP to get insight about what all the numbers really mean!