University of Arkansas Office for Education Policy

Archive for the ‘The View from the OEP’ Category

Arkansas’ Struggling Readers

In The View from the OEP on October 10, 2018 at 3:12 pm

Today we are excited to release new research about Arkansas’ struggling readers. We thought since school performance reports are being released this Friday, it is a good time to remember that actual kids are behind the test scores used to generate the reports.  We hope you take a moment to reflect on who Arkansas’ struggling readers are, and how their reading skills develop through early high school.

We think this research is particularly important in light of all the effort that Arkansas educators are putting into improving early reading ability. By better understanding the historical improvement patterns of students who demonstrate low reading ability in third grade, we can better evaluate the effectiveness of the efforts to improve outcomes for struggling readers.

We examined the reading achievement of nearly 77,000 Arkansas students who were continuously enrolled in Arkansas public schools from 3rd grade through early high school. We hope you read the policy brief and more in-depth Arkansas Education Report, but we briefly summarize our findings here:


Who isn’t reading ‘on grade level’ in 3rd grade?

  • Students who qualify for free or reduced price lunches were twice as likely to be low-achieving readers in 3rd grade, compared to their more economically advantaged peers.
  • Students who are Black or Hispanic were twice as likely to be low-achieving readers in 3rd grade, compared to their White peers.

We know- you’re like “Duh” any teacher could have told you that, but it is important to have the data, the facts, about out struggling readers.

  • Males are somewhat more likely than females to be identified as low-achieving readers but the difference is not as large as it is between economic and racial groups.
  • ELLs are somewhat more likely than non-ELLs to be identified as low-achieving readers but the difference is not as large as it is between economic and racial groups.

Do the students who demonstrate low reading achievement in 3rd grade ‘catch up’ to their peers over time and what are the characteristics of students who do?  Note- we use standardized scores (z-scores) to examine student achievement over time due to changes in assessment.  You can read more about the methodology in the full report.

  • Of students who were initially low-achieving in 3rd grade, 12% ‘caught up’ to average state reading performance by early high school.
  • Students who were economically advantaged, White, Hispanic and/or female students were more likely to reach average reading achievement by early high school than their Black, male, and economically disadvantaged peers.
  • Among over 6,000 Black students who were identified as low-achieving in 3rd grade, only 6% demonstrated average reading achievement by early high school.
  • All types of low-achieving students demonstrated large improvements between 3rd and 4th grades, although rates of improvement after 4th grade is very different for different types of students.

 

Presented below are the standardized scores of initially low-achieving students from 3rd through 10th grade.  Results are presented by FRL participation and by race.

Figure 1: Average Reading Scores of Initially Low-Achieving Students: Grade 3 through 10 by Economic Disadvantage (FRL) Status g3 reading frl

Figure 2: Average Reading Scores of Initially Low-Achieving Students: Grade 3 through 10 by Race

g3 reading race

None of these initially low-performing student groups, even White or economically advantaged students, caught back up to the state average as a group by early high school.

  • Hispanic and economically advantaged students are achieving almost a half standard deviation increase in achievement as a group, and White students are making approximately 0.4 standard deviation increase, while Black and low-income students are making closer to a quarter of a standard deviation increase in achievement.
  • Even though low-achieving Hispanic students initially have very low average scores, they make advancements comparable to those of White students, the most advantaged group. This is an exciting trend to observe because it indicates potential for a narrowing achievement gap between White and Hispanic students.

You might be thinking that there are differences in reading score improvement between Hispanic students who are identified as ELL and Hispanic students who are not. We were pleased to find that both ELL and Non-ELL students made large gains in reading achievement over time as presented in Figure 3.

Figure 3: Average Reading Scores of Initially Low-Achieving Students: Grade 3 through 10 by English Language Learner Status

g3 reading ELL

 


In summary, Arkansas students face large and persistent racial and socioeconomic disparities in third grade reading scores. Moreover, few of our students who are struggling readers in third grade ever catch up to the state average. And these are for relatively stable students, those who are continuously enrolled in our schools from grades 3-10.

Our hope is that Arkansas’ average reading scores will continue to increase and all students will grow to read proficiently, but it is evident that special attention needs to be given to low income and racial minority students and students who are struggling with basic reading skills in third grade.

Although some schools saw double-digit reading proficiency gains after RISE trainings in 2017, similar improvement was not reflected on 2018 assessments. Programs must be carefully monitored to determine what, if any, impact they are having on changing the long-terms outcomes for students who, as demonstrated in this research, are likely to continue to struggle to read proficiently throughout their educational experience.

Schools and districts should carefully examine the progress of their struggling readers and consider the effectiveness of any interventions or programs that are being implemented.  Although this analysis uses state assessments as the measure of student achievement, schools and districts should examine multiple measures, including high quality formative assessments, to evaluate progress in student’s reading.

We must continue to strive to ensure that all students are leaving elementary school as competent readers, equipped with the literacy foundation necessary for future academic success.

Advertisements

Unpacking School Performance Ratings

In The View from the OEP on October 3, 2018 at 1:52 pm

Arkansas school performance ratings and A-F letter grades will be released to the public on October 12th.

Here’s what we think you can expect:

1) Most schools will get the same Letter Grade as they did last year.

2) Schools serving a smaller percentage of students eligible for free and reduced lunch will be more likely to get an “A” or a “B” than schools serving a population where a greater percentage of students experience economic hardship.

3) Arkansas’ growth measure- a powerful indicator of students’ academic improvement over time- will still be over shadowed by single-year achievement and will still be challenging for educators and parents to understand.

____________________

In advance of the release, we wanted to review the purpose of a school performance report.

This is the true baseline year for Arkansas’ new accountability system.  The state has worked diligently to develop an accountability system that will support student learning, as presented in the theory of action presented below.

Theoryofaction

The idea is that if schools get good information about what is really happening in their schools, then they can make improvements that will improve outcomes for students.

The continuous cycles of inquiry will take time to develop and build, and will require some new feedback systems to support schools and districts as they work to identify needs within their systems.

But does a school performance report really give schools the tools that they need to improve?  It’s complicated, and not just for Arkansas! A report released yesterday provides some insight into how stakeholders feel about school accountability across the nation.

One of the big benefits of school accountability identified in the report has been the increased transparency and quality of information about what is going on in schools. The interpretation of the data and the ability of schools to interpret the data and develop a plan to improve learning and resource allocation is key to Arkansas’ plan, as presented below.

Cycle

Arkansas’ school personnel have been able to preview the school performance report since September 21, and we can tell that they have because ADE has made some updates to the reports and extended the private review deadline.

We appreciate the state letting school personnel review the information, and fully support the pursuit of high quality data in the system.  But we worry that school personnel are spending a lot of time trying to check the scores, as opposed to interpreting it and developing a plan to support student learning.

One thing that we think will help school personnel re-direct their time from checking a bunch of data points to developing a plan to support student learning, is to really understand what is driving the school performance score.

The Biggie- Achievement

The majority of the school performance score is determined by student achievement on the 2017-18 ACT Aspire English Language Arts and Mathematics performance of students in grades 3-8.  Wealthier schools will generally have higher achievement.  We know that achievement on assessments is negatively correlated with student risk factors such as poverty.  You can check out the relationship in our data visualization.

Schools and the public have had information about achievement since scores were released early in the summer, which was much earlier than in previous years. Schools may have difficultly, however, calculating what their weighted achievement score would be because student ELA scores not reported by the full range of categories used in the school performance calculations.  Additional information about cut scores for the full range of performance categories is included in the Final ESSA Decision Rules, and could be applied to student scores but that would take quite a bit of time!

Note: As we mentioned in our earlier blog– achievement on the ACT Aspire pretty much stayed the same as last year, but the weighted achievement scores will be lower than last year for most schools because ACT Aspire modified the criteria for ELA readiness. Thankfully, because of the forward thinking actions taken by the ADE to equate the scores with prior years and adjust the cut points for the letter grade, the lower achievement scores should not result in lower school letter grades across the state.

The Most Important (to us!)- Academic Growth

Here at OEP we feel that this is the most important piece of information in the school performance report because it reflects how much students at the school increased their academic performance over time compared to how much the average student improved.  We feel growth is a much more informative indicator of how schools are educating students than achievement, and are pleased that it is much less correlated with school poverty rates.

Unfortunately, schools can’t verify this growth information because it is calculated at the state level- using the relationship between current and historical test scores of every student in the state to develop a ‘predicted score’.  This score is then compared to each student’s actual score to determine if the student’s academic achievement as measured by the ACT Aspire assessment was more than expected, as expected, or less than expected.  These student-level scores are averaged at the school-level and reported as a reported to the school as a transformed variable with a mean of 80 and a standard deviation of around 3. We are confident that the calculations are correct, and would advise schools to worry less about re-calcaulting the values themselves (which they can’t due to only having access to their school’s data) and more about understanding what this indicator means.

Many schools throughout the state are familiar with NWEA’s MAP Growth, where there is a target score required for students to meet annual growth.  This makes it easy for schools to identify if students met or exceed growth.  NWEA has information with 370 million test event records spanning more than 15 years, so they have a really good idea of how a typical student will increase their score over time.  ACT Aspire is a relatively new assessment however, so we want to make sure that we aren’t ‘guessing’ how much a typical student ‘should’ increase.   Instead, the state uses real data to inform how much a typical student DID increase from one year to the next and then compares that to the performance of students with similar test score histories.

Growth is really the most meaningful at the student level.  If students in program X are not meeting growth expectations, while students in program Y are, then careful consideration should be given to re-allocating resources so more students can benefit from program Y.  In discussions with the ESSA advisory team yesterday, we were thrilled to hear that the state may be able to provide student-level growth information in the future which would be super valuable to school leaders as they develop a plan to enhance student learning and resource allocation.

The Most Distracting- SQSS

The School Quality and Success (SQSS) indicator is a mouthful, but is really the smallest contributor to the overall school performance score. Since parts of SQSS reflects achievement, it is not surprising that SQSS scores are also negatively correlated with school poverty rates (r= -0.48).

As we have said before there are a lot of indicators included in this measure and school personnel may be spending a lot of time focusing on each indicator and wondering if the data are accurate. For some indicators, schools could verify the data through their own systems by applying the business rules, but for other indicators they cannot.  The first time schools saw the SQSS indicators was last spring, and the data included in the current school performance reports was pulled soon after.  Because SQSS indicators represent systems in place at schools, such as attendance reporting practices and course enrollment, and because these systems may require some time to adjust, we don’t expect to see large (or meaningful) changes in these scores yet.

We like how SQSS indicators can help schools get more accurate information about what is happening at their school, but are looking forward to when they are presented in a way that schools can really use them in their strategic planning to support student outcomes.

The continuous cycles of inquiry will take time to develop and build, and will require some new feedback systems to support schools and districts as they work to identify needs within their systems.



What’s in a Grade? 

The school performance report also supports Arkansas’ legislation that every school must receive an A-F letter grade. The letter grade was designed to create a method for parents to easily understand the quality of a school, but does an A-F letter grade really give parents the information that they need about how a school is doing? One stakeholder in yesterday’s report captured the challenge of assigning schools letter grades:

“How do you make something that is simple enough to be understood, like an A through F rating system, but also incorporate a number of different factors that are complex enough to capture all of the things we want schools to do? Everything from math and reading to also discipline data or enrollment data or attendance data or all these other sort of facets of that system. So how do you make something that is usable and understandable, but also nuanced?”

Another stakeholder pointed out how the A-F grade represents what matters to the developers of the metrics:

“I sort of feel like the single rating of either A through F or on a number is sort of the worst impulses of accountability. Because not only are you saying what matters by its inclusion in that, but how much it matters, by how it’s weighted. So man, that takes a lot of faith in yourself that you can specify how much you should care about academics relative to attendance, relative to these other things.”

We agree- and have addressed before how although the intention was to make growth scores weight more heavily in the school performance reports and associated letter grade determination.  When schools where students make the largest improvements in achievement can still be saddled with a low grade due to the characteristics of the population they serve, we are sending the message that growth doesn’t really matter.   And as long as the performance index results in schools serving more advantaged students getting higher letter grades, we also send the message to parents that what makes a good school is not the learning that happens inside the building, but how large the houses are that surround the school.

We urge educators and parents to focus on the academic growth indicator, and view the purpose of a school performance report as the beginning of an ongoing conversation about how to continually increase student learning. 

Arkansas Discipline Update

In The View from the OEP on September 19, 2018 at 11:36 am

Last week, the Student Discipline Task Force submitted their report to the Arkansas State Board of Education, and OEP was pleased to present the annual report on student discipline.

OEP’s report examines student discipline in Arkansas public schools. We identify trends and a number of key student outcomes related to student discipline in the Arkansas public schools. While the data are only limited to what schools report, there are several meaningful findings from this work. While we recommend the full report and this introductory policy brief, today we wanted to share the highlights of what we found.


What are trends in reported student infractions and associated consequences?

  • There has been an 87% increase in reported discipline infractions since 2012-13, with over 270,000 discipline referrals in 2016-17. We believe the increase in referrals likely reflects greater focus on reporting discipline infractions as opposed to an increase in misbehavior in Arkansas schools.

discipline1

  • Over 80% of discipline referrals are for insubordination, disorderly conduct, or “other” infractions.
  • The majority of the increase in infraction referrals has been for “other” infractions. In 2016-17, additional reporting categories were included, but over a third of infractions remained identified only as “other”.

discipline2

  • Over 93% of discipline consequences are for out-of-school suspension (OSS), in-school suspension (ISS), or “other” action. There has been a decline in reported reliance on OSS, ISS, and corporal punishment over time.
  • The majority of the increase in consequences has been for “other” actions. In 2016-17, additional reporting categories were included, but about 19% of consequences remained identified only as “other”. While trends away from exclusionary discipline might indicate benefits for students, knowing more about what the “other” consequences are is important for understanding whether this represents a meaningful change for students.

discipline3


Are schools complying with Act 1329, which bans the use of OSS as a consequence for truancy?

  • The use of OSS for truancy declined from about 14% of all truancy cases in 2012-13 to about 7% of cases in 2016-17.
  • In 2016-17, 76 schools reported at least five or more truancy infractions and reported using OSS in at least 10% of those cases. Many of these were concentrated in a few districts.

discipline4


Are there racial or programmatic disproportionalities in school discipline?

  • Disproportionalities by race, free- and reduced- price lunch eligibility, and special education status exist both in terms of the number of referrals for infractions of various types, as well as in the likelihood of receiving exclusionary discipline, conditional on referral for a particular type of infraction. For example, black students receive 117.6 referrals per 100 students, relative to only about 37-40 for white students, Hispanic students, or students of other races. Then, conditional on being written up for any infraction, Black students receive OSS, expulsions, or referrals to ALE in about 25% of these cases, relative to only about 15% for students of other races.

discipline5


Which types of schools are High-Exclusion schools?

  • Certain types of schools in the state are more likely to administer lengthy exclusionary punishments: schools with greater proportions of black students, high schools and middle schools (relative to elementary schools).
  • There also appears to have been a decline in the severity used, on average, between 2014-15 and 2016-17.

What is the relationship between student absenteeism and exclusionary discipline?

  • There is a moderate correlation between student absenteeism and OSS days received, with the strongest correlations between grades 7 and 10.
  • Students marked as chronically absent in those grades received 0.5 to 0.64 more days of OSS on average, compared to those not chronically absent.
  • This suggests that schools seeking to tackle absenteeism may consider discipline reforms as one possible solution.

discipline6


What is the relationship between educational attainment and exclusionary discipline?

  • Exclusionary discipline in high school (and particularly ninth grade) is associated with lower likelihood of high school graduation and lower likelihood of enrolling in college, conditional on a variety of student characteristics as well as baseline achievement in eighth grade.
  • The magnitude of these relationships decline after controlling for the behaviors (types of infractions) reported, although there is still a small relationship detected in some cases.

discipline7

 


Act 1329 has provided an opportunity to examine student discipline the the state.  The issue is complex, but one of the Student Discipline Task Force recommendations was that schools consider alternatives to exclusionary discipline practices when addressing student behaviors. The Board expressed concerns that OSS continues to be used as a consequence for truancy, and discussed the importance of continued effort to support schools and communities in reducing the use of exclusionary discipline for students. We look forward to continued discussion about improving learning environments for Arkansas students!

 

Chronic Absenteeism in Arkansas

In The View from the OEP on September 12, 2018 at 12:56 pm

September is Attendance Awareness Month, and this week we learned that Little Rock School District is aiming to cut chronic student absenteeism by partnering with The Campaign for Grade Level Reading, ArKids Read, Heart of Arkansas United Way, and Optimist Club of Greater Little Rock. The idea is that academic performance will improve if students come to school.

But what IS chronic absenteeism?

Chronic Absenteeism is when students are missing a lot of school.  How many absences are chronic? It depends on who you ask.

From a national perspective, schools were first required to submit information on chronic absence to the Office of Civil Rights in the 2013-14 school year.  The OCR defined chronic absence as missing 15 days or more of the school year.  Data from 2015-16 showed an increase in chronic absenteeism, but researchers believe it is due to improved reporting.  Unfortunately, chronic absence data will now be collected through the US Department of Education’s Ed Facts Division.  These data will not be comparable, because chronic absence is defined differently.  Ed Facts will define chronic absenteeism as missing 10 percent or more of school days.

Chronic Absenteeism Impacts Kids

Chronic absence links to poor academic performance, delayed graduation, and higher dropout rates, and it correlates strongly with school climate issues such as bullying and poor transportation.

According to the new report Data Matters; Using Chronic Absence to Accelerate Action for Student Success, Arkansas had a lower percentage of students chronically absent than in the nation as a whole. While the national rate of chronic absenteeism was 15.5%, only 14.1% of Arkansas students missed 15 days or more of school during the 2015-16 year.

Hedy N. Chang, one of the authors of the report, stated that attendance is strongly associated with academic success. Regular absences are an alert that students may need additional support and an investment of resources to have the opportunity to learn and thrive, Chang said. Barriers to good attendance include illness, trauma, unreliable transportation, negative school experiences such as bullying, lack of engagement and relevance to a child, poor discipline that pushes kids out and family failure to realize importance of school attendance. Although researchers have found a connection between poverty and absenteeism, there isn’t a strong correlation in Arkansas between school poverty rates and chronic absenteeism rates.

The map below shows the 2015-16 reported rates of chronic absenteeism in Arkansas school districts.  You can access an interactive map that drills down to the school level here.

Absence png

Chronic Absenteeism Impacts Schools

Chronic absenteeism has become a national education metric because the 2015 Every Student Success Act (ESSA) required states to include an indicator of School Quality and Student Success (SQSS).  This ‘fifth indicator’ indicator allowed states to place a value on elements of learning that are not typically measured on assessments. States had a substantial amount of freedom to decide which SQSS indicator(s) to include. Arkansas, like the majority of the states, selected chronic absenteeism as one of the measures of School Quality and Student Success used in the state accountability system.

Chronic absenteeism is one of several measures used to indicate SQSS in Arkansas.  Schools are awarded a point for each student who is present at least 95% of the school year, and a half a point for each student who missed between 6 and 9% of the school year.  Schools receive no points for students who are absent 10% of the year or more.  These attendance points are combined with the other SQSS indicators, so attendance doesn’t actually have a large impact on the overall score, but schools may feel incentivized to overlook reporting absences. When data are not valid and reliable, we can’t use them to support students.

As noted with the national attendance data collection, monitoring will be needed to ensure that good data are collected so we can reliably use it to determine relationships with student success in our schools. High rates of absenteeism signify the need to dig deeper to understand the underlying challenges.

Improving attendance rates requires an intentional shift away from punitive action and blame that have no evidence of yielding sustained improvements in attendance.  Arkansas has been working on moving away from such punitive action with Act 1329, banning Out of School Suspension as a consequence of truancy.  As OEP will report to the State Board on Friday, however, over 1,000 referrals for truancy resulted in OSS in the most recent school year.

Like Little Rock’s initiative, communication is key. Attendance Works provides a framework for improving student attendance in which educators starting with positive engagement and problem-solving to identify and address barriers to getting to school.

We celebrate all the schools that are examining the root causes of why students are missing school.  Let us know how we can help!

 

September Happenings!

In The View from the OEP on September 5, 2018 at 11:31 am

kids

This week, we wanted to give you a heads up about a bunch of education-related events happening in September!

ESSA School Index reports will be open for Private Viewing on My School Info. from September 18 through 24, 2018.  School and district leaders should take advantage of this time before the public release to get prepared to explain to their stakeholders

  • what the ESSA School Index says about their school(s),
  • what the plans are to continue to improve, and
  • how stakeholders can support the work.

Remember that A-F school grades and rewards and recognition money will also be based on the ESSA school index, so a clear understanding and pro-active communication plan seems like a good idea to us!

Thanks to the hard work and planning of ADE staff, we have this information early in the school year so you can use it to inform your practices!  If you have questions about your report, or how to communicate the results, we are happy to help – just email us at oep@uark.edu.

National Merit lists should be released soon as the PSAT selection criteria for 2019 graduates were just released. Arkansas students need a score of 214 to be selected as a Commended Student.


We also wanted to be sure you were aware of several interesting conferences scheduled for September:

Education Innovation Summit: September 27th and 28th in Rogers, $300

This is the fourth year for this conference and they have a great lineup of international, national and local speakers! The conference is a partnership between Office of Innovation for Education (OIE) and ADE and the speakers include Derek Wenmoth, from New Zealand, Susan Patrick from iNACOL and  Stephen Spaloss with City Year.

There are a bunch of breakout sessions from practitioners implementing the work of innovation, along with policy sessions, design sessions, and opportunities to work in small groups with experts in mentor sessions.  If you haven’t been before, you can check out videos of past conferences here.

Data and Policy Symposium: September 27th 8am-1pm in Little Rock, FREE

ForwARd Arkansas, in partnership with the Institute for Chief Data Officers at UA Little Rock are bringing together national experts to discuss the importance of creating a longitudinal data system to track educational outcomes in the state of Arkansas. The keynote will be provided by John Easton, former director of the Institute of Education Sciences at the U.S. Department of Education.   Access to quality, integrated longitudinal data to track outcomes between Pre-K, K-12, post-secondary education/training and workforce participation is essential to inform future planning and resource allocation.

Arkansas Association of Gifted Educators: September 27th in North Little Rock, $105 for Members. Topics include: How GT fits with the Science of Reading Act, Strategies to Identify and Service Students of Low Income, and Closing the Identification Gap.

Arkansas Association of Federal Coordinators: September 19th-21st in Hot Springs $225 for Members. Topics include: Evaluating the Effectiveness of Programs and Budgeting, Utilizing School Index Reports to Analyze Effectiveness of Title Schools, and ADE and Legislative updates.


Meetings:

State Board of Education: September 13th and 14th*

*OEP will be presenting the latest schools discipline research on the 14th at 9

Education Caucus: September 24th at noon

Topic: Student transportation -or- teacher salaries

House and Senate Interim Committees on Education: September 24th and 25th

Topic: 2018 Adequacy Report and issues related to Educational Adequacy

        A draft of the 2018 Adequacy Report can be found here

Does Arkansas Have a Teacher Shortage?

In The View from the OEP on August 29, 2018 at 1:16 pm

pencils2

Amid national conversations about teacher shortages, we’ve been wondering if Arkansas has a teacher shortage.  We’ve been doing some research, and are excited to release our Policy Brief and Arkansas Education Report on Teacher Supply in Arkansas.


Does Arkansas have a teacher shortage?

Well, it’s difficult to tell.  One measure of teacher supply could be the number of teachers available to students, and Arkansas seems to do okay here.  According to ADE’s Recruitment and Retention Report, Arkansas students have greater access to teachers than their peers across the country. In Arkansas there is one classroom teacher to every 14.3 students, compared to the national average of 16.1 students per teacher (NCES).

Another measure of teacher supply could be the number of licensed educators in the state.  We have a bunch in Arkansas! According to Arkansas Department of Education (ADE), there were 33,228 certified teachers employed in Arkansas schools in 2017-18, and 60,317 people in Arkansas with a teaching license of any type as of 2017-18. That’s 81% more licensed teachers than are currently employed in the public schools.

A third measure of teacher supply considers the number of students enrolled in educator preparation programs, and the number of education program graduates entering the workforce. The ADE references the decline in the number of enrollees in education preparation programs as particular cause for concern.

All of these measures, however, focus on the overall supply of teachers in the state, and do not address current teacher supply realities faced by districts. It is possible there could be a shortage in some regions and subjects, but a surplus in others.


What do districts say?

We use a more intuitive and immediate measure of teacher supply: a ratio of the number of applicants for each open teaching position. This is the first study to define teacher supply in this way. By examining the ratio of applications to vacancies at the district level, we get a more direct, localized, measure of teacher supply and can investigate the relationship district characteristics may have on supply.

There is no centralized source to obtain the number of teaching vacancies and applicants in Arkansas, because each district posts their own position announcements and handles applications independently. In fact, over 46% of districts use only paper applications for teacher positions.

To gather information on the number of teaching vacancies and associated applications from school districts, we administered a survey to all districts in the Spring of 2017.  Overall, 74% of districts responded and the respondents were representative of statewide districts on examined characteristics.

We examined the reported teacher supply by student characteristics, district enrollment, district location, and beginning teacher salary. We also examined the relationship between teacher supply and the grade level and subject area of the vacancy.


What did we find?

  • There is not an overall teacher shortage across the state, but teacher supply is unequally distributed.
  • On average, districts reported receiving 6 applications per teacher vacancy. 
  • Districts that have the most favorable teaching supply are larger districts with enrollment greater than 3,500. On average, these districts get 8 applications per vacancy.
  • Urban and suburban districts, as well as districts in the Northwest appear to have a significant advantage in attracting teaching applicants.
  • Districts that face a greater challenge in attracting teaching supply are those in the Central, Southwest, and Southeast regions, and those in rural areas.
  • Beginning teacher salary is not found to be significantly related to district teacher supply, although districts who pay the most recieve more applications per vacancy.

size


What do we recommend?

Today the Learning Policy Institute released a report about six evidence-based policies that states are taking to solve teacher shortages:

  • Service scholarships and loan forgiveness (Arkansas does this!)
  • High-retention pathways into teaching (Arkansas does this!)
  • Mentoring and induction for new teachers
  • Developing high-quality school principals
  • Competitive compensation (Arkansas does this!)
  • Recruitment strategies to expand the pool of qualified educators

Arkansas is highlighted several times throughout the report, and we applaud the work that the state is doing to attract and retain teachers.  We do, however, worry that the current method Arkansas uses to identify teacher supply focuses more on the overall intended (future) supply, than on the current supply districts experience through the number of applications they receive.

Issues related to district level teacher supply may be different from statewide challenges and policies to address them must  be considered. Rather than focus on overall supply, Arkansas should consider examining teacher supply at a more localized level and examine ways to better match prospective teachers to positions.  To that end, we suggest the following recommendations:

  • To better understand how teacher supply is distributed across districts, the state should consider collecting application and vacancy information at the district level.
  • To make it easier for applicants to find district vacancies and districts to find applicants, a statewide online application process could be used. Teachers may be more likely to complete an application online than go ‘old-school’ and mail in a paper application.
  • Starting the hiring process earlier, especially for low-supply districts, could increase both the quantity and quality of candidates.
  • Examining ways to purposefully place student teachers in districts, and developing more district-university partnerships where they are limited or may not exist, would also facilitate getting teachers to where they are most needed.
  • Expand communication of any incentives available for teachers, especially those in small districts and districts in the Southeast and Southwest regions of the state.

Teachers are critical to Arkansas’ success, and there is a lot of great work being done to support them. We want to make sure we have quality teachers in all our clasrrooms.  A deeper understanding of variations in teacher supply throughout the state can maximize the impact of the policies being used to solve any teacher shortages, and provide reliable data about their effectiveness.

National Opinions on Teacher Salary

In The View from the OEP on August 22, 2018 at 11:34 am

This week the education journal Education Next released its annual poll examining nation-wide attitudes toward major issues in K-12 education.  The poll surveyed more than 4,600 respondents (a nationally-representative sample), covered 10 main topics, and compared the results with those of prior years. Today we wanted to focus on questions in the EdNext poll about teacher salaries and consider the issue through an Arkansas lens.

Teacher Salary_EdNext2018

  • Respondents underestimated annual average teacher salaries by over  31%. The average response for a yearly salary of a public school teacher in their local district was $40,181, which was more than $18,000 less than the actual average teacher salary of $58,297.
  • Parents underestimated local teacher salaries by 30%, an amount similar to the general public.
  • Teachers’ guesses about annual teacher salary were closer to the actual amount than were those of parents or the general public, but still underestimated teacher salaries by over 20%.

Average Arkansas Teacher Salaries: The average salary for classroom teachers in Arkansas was $49,615 for school districts including charters (2016-17). In June, the Bureau of Legislative Research produced a brief and report on Arkansas teacher compensation, which indicated that Arkansas teacher salaries are near the middle of the pack when compared to surrounding states and those in the broader region as well.  You can check salaries for your district here. As mentioned here, OEP examined teacher salaries in depth in this report and policy brief.  We found that differences in teacher salary are greatest within regions of the state, and are most associated with student-to-teacher ratios and enrollment.

Teacher Salary Change_EdNext2018Scale

  • Respondents widely supported increasing teacher salaries, with 67% of the general public, 74% of parents, and 86% of teachers indicating that they felt teacher salaries should increase.

Teacher Salary Given_EdNext2018Scale

  • When provided with the actual amount that teachers are making, however, respondents were less supportive of increasing teacher salaries.  When told how much teachers earn, 49% of the general public and 53% of parents indicated that salaries should increase, a decline of 20% from when the question was asked without salary information included. Teachers, however, were only slightly less supportive of salary increases when actual salary information was provided.
  • Compared to last year’s results, however, the respondents were 13 percentage points more likely to support increases in teacher salary when provided the actual salary information.  This may reflect the impact of teacher strikes and widespread discussions of teacher salary throughout the nation.

Changes to Arkansas Teacher Salaries: Changes to Arkansas teacher salaries are being discussed, with gubernatorial candidates proposing increases.  Statutory minimum teacher salary has increased in each of the last four years, and the vast majority (87%) of districts in the state have a minimum salary that is higher that legally required. We support attracting and retaining high-quality teachers, but we want to make sure that changes in teacher salary are implemented in a way that makes a difference for Arkansas students.


National perspectives on education like those presented in the EdNext poll are interesting, but education policy is generally set at the state or local level.  We thought it was helpful to compare the national results to what is happening in Arkansas. If you would like to read more about the national perspectives, you can go here, and if you would like to know more about what is happening in education in Arkansas- you are already where you need to be!

ACT Aspire: The Summer Sequel with a Bummer Ending

In The View from the OEP on July 12, 2018 at 4:47 pm

This summer it seems like all the movies are sequels: and the preliminary scores from the spring administration of the ACT Aspire for the 2017-18 school year seems like a sequel too!

Unfortunately, even though student performance hasn’t changed, ESSA Weighted Achievement scores will decline for most schools!! This bums us out because we hate it when systems send inconsistent messages about how well schools are doing. 

In today’s blog we share new data visualizations, review ACT Aspire performance patterns by subject overall and by grade, and explain why your school’s ESSA achievement scores will likely be lower this year than last year.


Show me the Data!

We are excited about some new interactive data visualizations of performance patterns across the state. Maps are available for both district and school-level, and you can select specific districts/schools and see how they scored. Unlike other data currently available, we have combined the grade level results to provide an OVERALL score for each district/ school.

We use the OEP GPA as this overall indicator of performance. The OEP’s GPA is a weighted measure of student performance that gives the most credit to students who have exceeded expectations and the least credit to those that are in need of support. In this GPA measure, we treat the ACT Aspire test scores similar to the familiar grade point average for individual students: 1.0 is the lowest score, indicating that all students in a districts were In Need of Support, while 4.0 is the highest score, indicating that all students in a districts were Exceeding Expectations on the ACT Aspire.

ACT Aspire_map

As you can see in the maps above, districts in the upper left hand half of the state are more likely to be blue, indicating higher performance. This is not surprising since we are showing performance on the ACT Aspire, which is highly correlated with the percentage of students in the district participating in the Free/ Reduced Lunch program.

If you want to see more detailed district- and school- level information, both by grade level and OVERALL, you can find it on the OEP website here! Also, check out our interactive data visualizations of district and school performance. You can select specific districts/schools (no limit) and see how they scored, or limit the view to particular regions or levels of FRL participation.


ACT Aspire Performance:

The figure below presents the percentage of Arkansas students who met or exceeded expectations in each content area by year since we began administering the ACT Aspire in 2016. Overall, the percentage of students meeting or exceeding benchmarks stayed exactly the same in mathematics, science, English, and reading. This year, unlike prior years, writing scores were not provided. Note: From here on, we use the Benchmark percent meeting or exceeding benchmarks so we can compare Arkansas performance with national results.

Figure 1. Percentage of Arkansas students meeting or exceeding expectations on the ACT Aspire, by content area, 2016 to 2018.

ACT_Aspire_18_Text

While overall ACT Aspire results did not change at all, there were some changes by grade level within each content area. We examine each content area by grade level and consider the national average performance as well.


Math: Performance in math declined somewhat in the middle grades, with 6th grade evidencing the greatest decline from the prior year (-6 percentage points). Since 6th grade experienced an increase of 7 percentage points in 2016-17, this year’s performance is similar to that of 6th graders in 2015-16. There were increases in the percentage of 8th, 9th, and 10th graders meeting or exceeding expectations. This continues a pattern of improvement in these grades, which is good news, since these grades have lower math performance than earlier grades. Math performance in 7th and 8th grades was above the national average (indicated by the red diamonds).

Figure 2. Percentage of Arkansas students meeting or exceeding expectations on the ACT Aspire Math, by grade, 2016 to 2018.

ACt_Math_Grade_2018


Science: Performance in science was generally consistent with prior performance, but declined somewhat in the middle grades, with 6th grade again evidencing the greatest decline from the prior year (-3 percentage points). Even with the decline, however, 6th grade maintained the highest performance across the grades. Science performance in 4th, 7th, and 8th grades was above the national average (indicated by the red diamonds).

Figure 3. Percentage of Arkansas students meeting or exceeding expectations on the ACT Aspire Science, by grade, 2016 to 2018.

ACT_Science_2018


English: Performance in English was generally consistent with prior performance. Scores for 8th grade students increased by 2 percentage points. English performance in grades 3-8 was above the national average (indicated by the red diamonds).

Figure 4. Percentage of Arkansas students meeting or exceeding expectations on the ACT Aspire English, by grade, 2016 to 2018.

ACT_English_Grade_2018


Reading: Performance was generally consistent with prior performance. The greatest increase was for 5th graders, with a 4 percentage point increase. Reading performance in grades 3-8 was at or near the national average (indicated by the red diamonds).

Figure 5. Percentage of Arkansas students meeting or exceeding expectations on the ACT Aspire Reading, by grade, 2016 to 2018.

ACT_Reading_Grade_2018


Plot Twist:

Every good sequel has a plot twist, and this year’s ACT Aspire results are no different. The one that has everyone scratching their head (okay- maybe just us here at OEP) is the updated cut-points for ELA and STEM readiness benchmarks. As you can see in the figure below, these new cut points (which are, according to ACT Aspire, more aligned with the performance expectations of the ACT) decreased the percentage of students meeting readiness benchmarks. In ELA, the change was a decrease of 8 percentage points from last year, but in STEM the new cut points resulted in a 25 percentage point decrease.

Figure 6. Percentage of Arkansas students Meeting Readiness Benchmarks on the ACT Aspire.

ACT_readiness_2018

The decrease in STEM readiness is startling, especially considering that we saw NO CHANGE in statewide math or science performance, but won’t have an impact on schools because STEM readiness is not used in accountability.

The decrease in ELA readiness, however, will impact schools because ELA scores are used to calculate the Weighted Achievement score in the ESSA school index. As we’ve discussed before, although stakeholders intended for growth to ‘count more’, Achievement continues to be the primary factor driving school letter grades. On a positive note, however, these changes to the ELA cut points should not impact growth scores in any way!

Decreases in ELA readiness are apparent in grades 4-10, with declines in the double digits for 6th, 7th, and 8th grades. This means that schools serving students in these grades are going to receive LOWER achievement scores on the ESSA index even though performance in English and reading was unchanged. It is unfortunate that this change was made, as Arkansas schools were just gaining familiarity with the ACT Aspire and the new ESSA metrics.

ACT_ELA_2018_arrow


So- what are the big takeaways from the preliminary ACT Aspire results? 

  • Performance on the ACT Aspire is related to school/ district poverty rates
  • Performance is generally the same as last year
  • Arkansas is scoring at or near the national average in most subjects/ grades
  • ELA cut points for meeting readiness benchmarks were changed this year by ACT Aspire, so schools should expect to have lower Weighted Achievement scores than last year on the 2018 ESSA reports
  • Growth scores should be unrelated to changes in the ELA and STEM cut points.
  • You can use the data resources from OEP to see overall district and school values
  • Data visualizations can help us see statewide patterns in performance and compare performance to other schools/ districts of interest.

We are happy to be able to share these resources with you and looking forward to seeing the Growth Scores on the ESSA reports in September. Stay tuned to OEP for more info.

Update 7/23: The cut scores for 2018 A-F school letter grade assignment have been lowered to compensate for the new ACT Aspire ELA cut points. You can find the new cut points here. 

 

 

School Report Cards for NWA and Pulaski County (new!)

In The View from the OEP on June 27, 2018 at 2:48 pm

NWARC        PCRC

Today we are pleased to release the 2017 Northwest Arkansas and Pulaski County Education Report Card.  These report cards provide an easy to understand overview of how students in the area schools are performing. OEP has produced a regional review of Benton and Washington county schools for several years, but this year is the inaugural release of a report card that focuses on the school systems in Pulaski County.

The report cards are in a ‘dashboard’ format that makes it easy for educators, school administrators, parents, and policymakers to see how school districts and charter schools are performing. Performance on key measures is broken down by Elementary, Middle, and High School levels and compared to regional and state scores.  For large districts, the report cards also include individual school data, where percentile ranks make the achievement and growth scores easy to interpret.

Springdale

The Report Cards put district-level information about academic growth, academic achievement, school quality, and A-F letter grades into a one-page context for quick interpretation. The performance data used in the report card are from the 2016-17 school year, the most recent data available at this time.

These key metrics of school performance are reported by the ADE at the school level in ESSA reports, but we feel they are important to consider from a district level to examine how effectively the school system as a whole is educating students. Additional information on ACT scores and high school graduation rates, which are important outcomes for students at the end of the K-12 journey, are also included.  To help make the connection between district resources and student success, we also include the district’s student to teacher ratio and amount of money that each district spent per-pupil.


That’s a lot of information!  What is the most important?

We believe that the growth scores are the best indicator that districts are doing what’s really important: helping all students learn. Growth scores are less related to student characteristics than achievement scores, as districts serving fewer At Risk students don’t always have higher growth scores.  The Growth Score indicates how much the district’s students in grades 3-10 improved over time on state assessments in English language arts and mathematics. This score also includes how well non-native speakers are progressing toward English language proficiency.

An average district, where students are growing academically just as predicted, will have a growth score of 80. In some districts, however, students are demonstrating greater increases in their academic performance from one year to the next than we would have predicted. To have a ‘good’ growth score, to be in the top 25% of schools in the state, Elementary schools need a growth score of 83 or higher, Middle level schools need a growth score of 82 or higher, and High Schools need a growth score of 81 or higher.

legendgrowth_sdale


How Are NWA Schools Doing?

Overall: Northwest Arkansas students are demonstrating greater growth in achievement and earning higher scores on the ACT Aspire than are the students in the state overall. Schools in NWA also have higher School Quality and Student Success scores, higher graduation rates, and are more likely to receive an “A” or “B” rating than are other schools across the state.

Academic Growth: Springdale School District had the highest overall growth score among the traditional districts, and was the only one where students demonstrated high academic growth at all levels: Elementary, Middle, and High School. Springdale students at the Elementary and Middle levels demonstrated the greatest academic growth in NWA, and nearly 70% of Springdale schools are in the top 10% of schools in the state for academic growth. Haas Hall Academy and Haas Hall Bentonville had the highest growth scores at the High School level. Many NWA districts had high growth at one or two levels, and we recommend they focus on identifying how they are supporting student learning at the schools where students are not demonstrating high growth overall.

Academic Achievement: Haas Hall Academy and Haas Hall Bentonville have the highest point-in-time achievement in the NWA area, with both NWA Classical and Arkansas Arts Academy joining in outperforming traditional districts in achievement. Bentonville School District had the highest achievement score among the traditional districts, and students at the Elementary levels demonstrated the greatest academic achievement in NWA. Since point-in-time achievement is so reflective of student demographics, we want to point out that among NWA districts where more than half of the students are eligible for the free/ reduced lunch program, Rogers, Gentry, and Siloam Springs reported the highest achievement.

School Quality: Lincoln School District had the highest School Quality/ Student Success Indicator Score at the Elementary Level. Bentonville School District had the highest Indicator Score at the Middle Level and Haas Hall Academy and Haas Hall Bentonville had the highest School Quality/ Student Success Indicator Score at the High School level.


How Are Schools in Pulaski County Doing?

Overall: Pulaski County students are demonstrating greater growth in achievement at the Elementary and High School levels than are the students in the state overall, and academic growth for students in Middle level schools is similar to the state average. This is particularly noteworthy since schools in Pulaski County serve a higher percentage of students who are likely to be at risk for not achieving in school than are served by the state overall. Pulaski County schools have lower academic achievement, School Quality and Student Success scores, and graduation rates than students in the state overall. Due to lower scores in these areas, Pulaski County schools are less likely to receive an “A” or “B” rating than are other schools across the state.

Academic Growth: eStem students demonstrated the greatest academic growth overall, with students at the Elementary, Middle, and High School levels receiving high growth scores. North Little Rock School District had the highest overall growth score among the traditional districts, reflecting high growth at the High School level. Little Rock School District had the highest growth at the Middle level and Pulaski County Special School District had the highest growth at the Elementary level.

Academic Achievement: eStem also had the highest point-in-time achievement of the Pulaski County area schools, followed closely by Academics Plus. Pulaski County Special School District had the highest achievement score among the traditional districts, overall and across all school levels. Since point-in-time achievement is highly correlated with student demographics, we want to point out that among districts where more than half of the students are eligible for the free/reduced lunch program, LISA Academy and Jacksonville Lighthouse reported the highest achievement.

School Quality: eStem had the highest School Quality/ Student Success Indicator Score at the Elementary level, Exalt Academy had the highest score at the Middle level, and Quest Academy was the highest scoring in School Quality at the High School level. Among the traditional public schools, Jacksonville had the highest School Quality/ Student Success Indicator Score at the Elementary level, Little Rock School District and Pulaski County Special School District tied at the Middle level, and North Little Rock was the highest scoring in School Quality at the High School level.



What’s the Takeaway?

In both the NWA and Pulaski County region, there are educational settings where students are demonstrating high growth by making larger academic gains than predicted based on their past performance. We want to point out that high academic growth can be found at all different types of schools:

  • schools like Haas Hall Bentonville that serve few “at risk” students,
  • schools like Covenant Keepers where 95% of students participate in the free/reduced lunch program and 30% are non-native English speakers,
  • schools like Janie Darr Elementary in Rogers which has high academic achievement,
  • schools where academic achievement is low,
  • open-enrollment charter schools, like eStem High and
  • traditional public school districts, like Springdale.

Growth scores for schools in NWA and Pulaski County also appear unrelated to resources like per-pupil expenditure or student:teacher ratio. Here at the OEP, we think growth scores are a meaningful reflection of increased student learning, and that high growth scores can be achieved by any type of school.

  • To have a ‘good’ growth score, to be in the top 25% of schools in the state, Elementary schools need a growth score of 83 or higher, Middle level schools need a growth score of 82 or higher, and High Schools need a growth score of 81 or higher.
  • If your school or district received a growth score of 80, students are demonstrating average growth in their academic performance on the state assessments in English language arts and mathematics.
  • If your school or district received a growth score below 78, students in your school or district are less likely to demonstrate academic growth than in the majority of schools in the state, and you should look for the reason.   Remember that unlike achievement, student characteristics like poverty are not highly related to growth.

On a side note: this is the first year for School Quality and Student Success scores to be reported. The School Quality score reflects a variety of indicators, and there may be a lack of consistency in how different districts report them, so here at OEP we are not freaking out about those scores.

If you want to know more about your school’s performance, check out myschoolinfo and type in your school name.  Under the “Reports” tab you can find the “ESSA report” for your school.

We hope that these report cards stimulate meaningful discussion about the educational settings within the communities, and look forward to hearing your thoughts. We invite you to share these report cards with those who are curious about the state of education in Northwest Arkansas or Pulaski County.

For more information about current education issues, check out OEP’s Policy Briefs and Blog.  If you are interested in digging into data, head on over to our website, where you can dive into all of the publicly available data on demographicstest scores, and finances.  The more we can be informed, share the good news, and look for ways to improve, the better Arkansas education will be.

If you would like a printed copy of a report card, please send us an email at oep@uark.edu and let us know which one and where you want it sent!

Summer is fun, but the Fall holds Promise

In The View from the OEP on June 20, 2018 at 2:10 pm

El Dorado Promis logo

We’re reaching the middle of summer—the time when college dorm room furniture starts popping up in prominent store displays and kids run past shelves of notebooks on sale because it’s too soon to be thinking about going back to school. While we’d all love an endless summer, classes, homework, and late night studying will again be the norm for college students across the state. For many students and families, the start of college brings anxiety not just about schoolwork, but about the rising cost of tuition and fees. According to the College Board, 60% of Bachelor’s recipients in the 2015-16 school year graduated with some debt. Concern about college debt may deter students from attending college, or it may lead students to work instead of study while in college. Thanks to the El Dorado Promise, hundreds of students coming to campus won’t have this anxiety impacting their postsecondary decisions.

The El Dorado Promise, announced in 2007, guarantees a full scholarship to all students who attend El Dorado School District from kindergarten through 12th grade, and a partial scholarship to students who attend at least from 9th-12th grade. The scholarship is capped at the maximum cost of tuition fees for an in-state resident at any public university in Arkansas—in the 2017-18 school year, this was a little over $9,000/semester. Students can combine this scholarship with other forms of financial aid, like the Arkansas Lottery Scholarship or Pell Grants, up to the full cost of attendance at any accredited university in the country. The Promise has impacted thousands of students from El Dorado since its establishment by Murphy Oil in 2007, and has now been around long enough for us to be able to analyze its impact.

In a recent paper and policy brief, we at the OEP asked what impact the El Dorado Promise has had on students’ postsecondary outcomes. Specifically, we wanted to know whether the Promise increased the rate at which El Dorado graduates enrolled in college and completed a Bachelor’s degree. To do this, we worked with folks at the El Dorado Promise office and the National Student Clearinghouse to gather data on whether students had received a Promise scholarship, if they went to college, and if they earned a BA within 6 years of graduating high school.

We can’t know the impact of a program like the Promise just by looking at how postsecondary enrollment and graduation rates have changed over time, because there are multiple factors that impact students’ college decisions—for example, during the Great Recession more people went to college to delay entering the workforce, and as the economy gradually recovered more people went straight to work after high school. So, we conducted what’s called a difference-in-differences analysis. We compared students who were eligible for the Promise (e.g. attended the district from at least 9th-12th grade) with students who weren’t eligible for the Promise (e.g. transferred to El Dorado in 10th grade or later). We examined the difference in college enrollment and completion rates between these two groups before the Promise was introduced (students who graduated between 2004 and 2006) and the difference in enrollment and completion rates between these two groups after the Promise was introduced (between 2007 and 2016 for enrollment, and 2007 and 2011 for completion). The descriptive results are shown in Figure 1. From this, it looks like the Promise was associated with a 16.5 percentage point increase in enrollment. That’s great news for El Dorado students!

pre and post promise chart

But, we wanted to be confident in our results, since students who were and were not eligible for the Promise could have been different in other ways besides mobility. We put this basic analysis into a regression framework, so we could control for things like high school GPA and student demographics. When we did this, we found that the Promise led to an 11.4 percentage point increase in enrollment, and a 10.7 percentage point increase in 6-year BA completion! 

In past work by folks at the OEP, we’ve found that the El Dorado Promise attracted families to the area, increasing enrollment in El Dorado Public Schools, and that the Promise led to a change in culture throughout the district—students, teachers, and parents are more focused on ensuring all students are ready for college. These efforts are paying off—the Promise is helping students get to and through college!

There’s a lot to celebrate about summer vacation—getting to spend time with your kids and/or friends, a chance to sleep in, and a reason to get outside. But there’s also a lot to look forward to in the Fall—including, for many students, a Promising postsecondary experience.