University of Arkansas Office for Education Policy

No time for field trips?

In The View from the OEP on March 13, 2019 at 11:38 am

Photo source:  Crystal Bridges Museum of Art

As Arkansas schools enter the final month before state testing, teachers may be focusing instructional time on test prep, foregoing other ‘non-tested’ subjects and activities, but new research finds that students who attended art-related field trips demonstrated increased engagement in school, higher levels of social-emotional skills, and, unexpectedly, higher scores on standardized tests!

The study is a longitudinal, randomized controlled trial, the gold standard for research.    Conducted by Jay Greene, distinguished Professor and head of the Department of Education Reform in the College of Education and Health Professions,  and members of the University of Arkansas National Endowment for the Arts Research Lab that he directs, the study randomly assigned fourth and fifth grade public school students in Atlanta, Georgia to attend three field trips throughout a school year. Students went to an art museum, a live theater production, and a symphony performance at the Woodruff Arts Center. A control group of students within the same schools did not attend the field trips.

You might not think that attending three field trips would lead to measurable, positive outcomes for students, but it did! You can read the working paper on the social emotional effects by lead author Angela Watson here, and the paper by lead author Heidi Holmes Erickson that addresses students’ engagement and academic outcomes here,  but here’s the highlights:

  • Students who were randomly selected to attend the field trips showed significantly higher levels of social-perspective taking through survey items like, “How often do you attempt to understand your friends better by trying to figure out what they
    are thinking?” and “When you are angry at someone, how often do you try to ‘put yourself in his or her shoes?”.  (Effects reflect the limited sample of students with higher academic performance who were more likely to be able to read and interpret the questions)
  • Students who were randomly selected to attend the field trips showed significantly higher levels of tolerance through the survey item, “I think people can have different opinions about the same thing.”
  • Students who were randomly selected to attend field trips reported more positive school engagement. They were less likely to agree that ‘school is boring’, and they had fewer disciplinary infractions in middle school than their control group peers.
  • Female students who were randomly selected to attend field trips were less careless in their survey answering, a measure of conscientiousness.  Female students in the second year of field trips demonstrate even greater levels of conscientiousness, while female students who are not included in a second year of field trips exhibit the same level of conscientiousness as female students who never attended one of the field trips.

Researchers also examined academic outcomes, hypothesizing that there would be no differences in standardized test scores between the students who attended the field trips and those who did not, as three days away from traditional classroom instruction was unlikely to affect students’ academic performance on math or reading exams one way or the other.

  • BUT- students who were randomly selected to attend field trips performed significantly better on their end of year standardized tests in math and English Language Arts than students in the control group.

Greene and his research team are continuing the research with another group of students, and will learn more about the students’ long-term outcomes as they observe them through middle school, high school, and beyond.

In the meantime, here at OEP we think that schools should consider the importance of field trips and arts experiences in a well-rounded education.

Making college accessible, one field trip at a time

In The View from the OEP on March 6, 2019 at 11:55 am


In the upcoming months, many high school seniors around the country will commit to attending a college or university. According to data from the Arkansas Department of Education, about 40-50% of high school graduates in Arkansas enroll in an in-state 2- or 4- year college. As students look forward to this important milestone, we thought it was a good time to take a step back to think about all of the decisions students have to make to be in a place where they’re deciding which college to attend. In particular, we want to focus on when and how students first start to get a realistic picture of what it is like to be a college student, and how those early experiences relate to students’ preparation for college.

Over the past two years, we at the OEP have been working with junior high schools and middle schools in the area to give eighth grade students information about college and to provide opportunities to visit the University of Arkansas-Fayetteville campus three times to learn more about the college experience. On these visits, students toured campus, participated in a college-readiness workshop, worked with academic departments, toured a dorm, and participated in an athletic event. (Oh, and of course they were able to experience the joys of a campus dining hall). These visits complement what schools are already doing to prepare students for their futures, such as offering career readiness courses, encouraging students to job shadow a professional in an interesting field, or making connections between coursework and potential careers.

Why did we focus on a campus experience, over and above providing information about college? The college-going process is complex, opaque, and confusing. Colleges can be difficult places to navigate, from making new friends, to adjusting to new academic expectations, to finding your way around campus. By creating an opportunity for students to navigate this type of environment and to meet successful students with similar backgrounds, we hope to make college seem less intimidating and more achievable.

Why did we focus on eighth grade? We knew we wanted to focus on early exposure to college, because research suggests many students get off a “college-preparatory” track in middle school. At the same time, students in the eighth grade are about to enroll in their first high school courses and to start building a high school GPA—in other words, they’re close enough to college for the message to resonate.

Of course, since we’re always interested in measuring the different ways schools are helping students, we had to evaluate these visits. So, within each of our partner schools, we randomized participating students to one of two groups. One group received an informational packet detailing postsecondary options in the state, discussing specific actions to take throughout high school to prepare for college, and educational requirements for different types of careers. The other group received the same packet of information and participated in the three campus visits. Then, we compared students’ responses to a survey asking about their attitudes towards and knowledge about college, as well as their course-taking decisions in ninth grade. You can read the full working paper here, but here are our main takeaways:

  1. Students who participate in the visits know more about college than students who just read the information on their own—students know more about the cost of college, what characteristics colleges look for in applicants, and how to earn college credit in high school, among other topics, if they have an experience to go along with the information.
  2. Students who participate in the visits have more conversations with school personnel about college than students who just receive printed information—in these conversations, students are talking about their college readiness, ACT scores they’ll need for their dream school, and other college-related topics.
  3. Students who participate in the visits are more academically diligent than students who just receive information—students who get a taste of the demands of college are more likely to fully complete a survey task.
  4. Students who participate in the visits are more likely to enroll in advanced math, science, and social science courses in ninth grade—students who participate in the visits are more likely to take advanced Geometry, pre-AP Biology, and pre-AP Civics in ninth grade, for example, than students who just receive printed information about college.
  5. Students who participate in the visits are less likely to want to go to a technical school after high school—as students gain a more realistic picture of the demands and benefits of a postsecondary education, they may be less likely to want a technical certificate and instead be more interested in other paths post-high school.

We’ve still got a lot of questions about how we can encourage students to prepare for college, particularly as more and more jobs require some sort of postsecondary training and college graduation rates stay flat. But it seems like providing field trips to a college campus is one strategy schools can pursue to help students think about all of their options for the future. If your school is interested in organizing a campus field trip (for any grade!), please reach out to us at the OEP! We would love to help you organize a fun, informational visit for your students that affirms their potential to succeed in any path they choose.


Discussion of the Proposed Arkansas Teacher Retirement System COLA Changes and Stress Testing

In The View from the OEP on February 27, 2019 at 1:19 pm

The Arkansas legislature is considering several bills that would affect the Arkansas Teacher Retirement System (ATRS), but two in particular have drawn the attention of teachers’ groups.

Here at OEP, we think it is important to understand these bills and the implications for current and future educators. We address the bills below, starting with HB1206, which would affect COLAs, before moving to HB1173 which would require plans to perform and publicly report the results of stress testing. HB1206 was withdrawn by its author this morning, but since COLAs will continue to be part of the policy dialog, it is still worthwhile to review the proposed changes. HB1173 is currently with the Joint Committee on Public Retirement and Social Security Programs, and is expected to be considered soon.

COLA Changes

What are COLAs? Cost of living adjustments (COLAs) are annual increases to retirees’ benefit payments that are meant to keep retirees from losing purchasing power due to inflation. When a teacher retires, he/she begins receiving a monthly retirement check based on years of service and average salary over his/her last few years on the job. If this base benefit amount remained constant throughout retirement, the teacher would lose purchasing power over time due to inflation. Prices of everything from groceries to healthcare tend to increase over time, requiring more dollars to purchase the same amount of goods and services. The purpose of COLAs is to increase retirees’ benefit payments at roughly the same rate as inflation so that they get consistent value from their checks throughout their retirement.

Because COLAs keep retirees’ monthly checks from losing value over time, they are a common part of plans that provide lifetime payments (i.e., annuities), including Social Security. Many public retirement systems’ COLAs were designed for a world in which inflation is consistently around 3 percent annually. Since the mid-90s, however, inflation has been far below that level. For example, average annual price inflation (i.e., December to December change in CPI-U as measured by the Bureau of Labor Statistics) for the south region since 1995 has been 2.13 percent and over the last 10 years it has been just 1.75 percent.

When COLAs provided by public retirement systems outpace inflation, retirees’ purchasing power over time is increased, rather than just maintained. Committing resources to COLAs that exceed actual inflation drives up plan cost and leaves less money to pay down pension debt and/or maintain benefit levels for new workers.

As governments across the country have struggled to get a handle on growing pension debt and rising retirement costs, many have made changes to COLAs. Since 2009, 30 states have reduced COLAs, and increasingly, governments are taking the logical step of linking COLAs to actual inflation (see NASRA report on COLAs). Many jurisdictions have also linked the provision of COLAs to their plans’ fiscal condition (e.g., COLAs can only be provided if the plan is greater than 90 percent funded). These types of changes not only keep plan costs in check but also ensure that COLAs fulfill their intended purpose of offsetting the negative effects of inflation on retirees’ purchasing power.

What changes were being proposed?

HB1206 (withdrawn today) would have modified the annual cost of living adjustments (COLAs) that retirees receive.  Under current law, ATRS retirees receive an automatic 3 percent COLA each year that is calculated using their starting benefit amount. HB1206 would have altered this by allowing the ATRS board to choose to provide an annual COLA, but the amount would capped at “the lesser of three percent (3%) or the percentage change in the Consumer Price Index (CPI), South Region as determined by the United States Department of Labor over the one-year period ending in the December immediately preceding the date for which the redetermined amount is being calculated.” In other words, the proposed change would have made the COLA discretionary and would have reduced the potential COLA amount in years when actual price inflation is below 3 percent.

So, why make the proposed change? While we don’t have any particular insight into Representative House’s way of thinking, the change was likely proposed because over the last 10 years the COLA specified in current Arkansas law would have significantly outpaced inflation. Figure 1 shows the value of $1,000 in hypothetical benefit payments growing at actual price inflation over the 10 years between 2009 and 2019 (black line) compared to COLAs under current Arkansas law – a simple 3 percent annually (orange line). To keep retirees purchasing power constant, benefit payments would have needed to increase by about 19 percent over this period, or by $190 for each $1,000 in payments. However, if COLAs had been given according to current Arkansas law, retirees’ benefit payments would have increased by 30 percent or by $300 for each $1,000 in payments. If inflation persists at its currently low level, the state would be committing more resources than necessary to maintain retirees’ purchasing power – resources that could be used to pay down pension debt and prepare the plan for the next downturn.

Figure 1: 10-Year Comparison of Different COLA Structures.

Screen Shot 2019-02-26 at 2.49.14 PM

Figure 1 also includes a blue line showing how the proposed change would have performed over the past 10 years. The COLA structure proposed in HB1206 would have tracked actual price inflation much better over this period, undershooting it by just a bit. However, while the proposed COLA structure would have performed well in today’s low inflation environment, it would significantly undershoot inflation if it were to rise. Figure 2 below is analogous to Figure 1 except it uses the last 20 years of inflation data instead of 10 years. As you can see, over this longer 20-year period the proposed COLA structure would have fallen short of actual inflation by a little more than the current law would have overshot it. To keep retirees purchasing power constant, benefit payments would have needed to increase by about 52 percent over this period, or by $520 for each $1,000. Current law would have increased benefit payments by around 60 percent while the proposed change would have increased payments by around 40 percent.

Figure 2: 20-Year Comparison of Different COLA Structures.

Screen Shot 2019-02-26 at 2.50.30 PM

The overall takeaway from these exhibits is that neither current law nor the proposed change would track actual inflation particularly closely when modeled using recent data. The current law increasingly overshoots inflation as it falls below 3 percent, and while the proposed COLA change would have done better at low levels of inflation, it would significantly undershoot inflation if it were to rise toward 3 percent.

So what should we do?

Given the low inflation of the past 10 years, it is reasonable for the Arkansas legislature to want to adjust the structure of COLAs. Why commit more resources to COLAs than is necessary to maintain retirees’ purchasing power? It is also a positive step to make COLA’s more responsive to actual price fluctuations by linking them to CPI. Such a change would continue to protect retirees while also more effectively managing plan costs. However, the COLA structure that was proposed in HB1206 could potentially fall well short of inflation if it were to rise above recent levels, likely necessitating additional modifications down the road.

If the legislature’s goal is to provide retirees with reasonable inflation protection while also not over-committing resources in periods of low inflation, then they should consider COLA structures that better track CPI within some boundaries. For example, if the legislature kept the capped, CPI-linked structure of HB1206 but changed from a simple to a compound COLA (i.e., apply the percentage increase to last years’ benefit amount rather than to the initial benefit), then COLAs would track CPI much more closely (see green line in Figure 3 below).

Regardless of the specifics, we strongly encourage the legislature to maintain adequate inflation protection for retirees and consider linking COLAs to actual inflation so that they more flexibly adjust to changing economic circumstances. We also reiterate our recommendation that any future changes to benefits, like COLAs, or contribution rates should only be made in the context of having a clearly defined funding policy and cost-sharing plan. 

Figure 3: 20-Year Comparison of Different COLA Structures with Proposed New Design.

Screen Shot 2019-02-26 at 2.51.15 PM


Stress Testing

Why is Stress Testing Important?

As noted in an earlier post,  stress testing, as proposed in HB1173, is vital to the prudent and sustainable management of public pensions. We are glad that ATRS agrees that stress testing is important, and of course, the plan already performs some level of stress testing, as the executive director points out in his recent legislative summary. However, on an admittedly quick scan of ATRS’s publicly available documents, we were unable to find any stress testing results that provide projections of future cost under multiple scenarios. We could certainly be missing something, but if stress testing results are not readily available to all stakeholders, including plan members, legislators, and taxpayers, then the value of the exercise is significantly diminished. ATRS has a responsibility to be transparent with and accountable to a broad set of stakeholders.

Given the huge implications for the public school workforce and large potential call on taxpayer resources, it is perfectly reasonable for the Arkansas legislature to place more specific stress testing requirements on ATRS than are included in the general guidelines of the Government Accounting Standards Board (GASB) or the Actuarial Standards of Practice (ASOP). ATRS’s complaints about added cost, frankly, ring hollow given the stakes for teachers and the state and school budgets. Based on our experience, it’s very hard to see how the requirements proposed in HB1173 would meaningfully increase cost above what the plan is already paying their actuaries to do.

Sounds good!  So what’s the problem?

To us, ATRS’s opposition to HB1173 appears to be more about fighting to maintain as much independence as possible rather than any concern about cost, etc. However, in our view, this strategy is shortsighted. Ultimately, it is the legislature and the taxpayers they represent that backstop the plan and ensure that teachers get the benefits that they were promised. Given the challenging times retirement plans face, ATRS should seek out increased productive engagement with the legislature so that legislators have greater ownership and a stronger sense of responsibility to fully fund the plan when the inevitable next recession hits.

We have also been disappointed by the overheated rhetoric of some employee groups regarding stress testing. Arguably their members face the greatest potential impacts from unexpected cost increases, a significant risk given ATRS’s higher than average discount rate and many experts’ investment return expectations. It seems the substantive area of disagreement is with the specific scenarios required in HB1173, which the Arkansas Education Association (AEA) calls “worst case scenarios” in the post linked above. Rather than reject the useful exercise of stress testing out of hand, we encourage the AEA to propose improvements to HB1173 that would make the scenarios acceptable to provide their members and legislators with vital information that would help them plan for whatever the future may hold.

The Takeaway

Retirement policy can be very challenging. It is technically complex, politically charged, and has many legal uncertainties. We are thrilled that Arkansas has done better than most states managing its teachers’ retirement system, but costs have still risen and public retirement plans face some significant challenges going forward. Here at OEP, we think HB1206 (now withdrawn) and HB1173 propose positive improvements that would help ATRS meet the retirement needs of former, current, and future teachers, even if the details need a little work.