University of Arkansas Office for Education Policy

More Data Is Better!

In The View from the OEP on January 9, 2012 at 8:35 am

We were pleased to see our OEP database highlighted in the state’s largest newspaper on the first day of 2012. Not only did the Arkansas Democrat Gazette highlight the very important nationally norm-referenced student achievement data for 1000+ Arkansas schools, but it also highlighted the real limitations of these data in an article on the opposite page.

The database used is a simple one that we’ve compiled at OEP.  We were prompted to do this by a local reporter who asked the straightforward question — how is my local school doing?  We directed her to the state database with the 2010-11 ACTAAP Benchmark results.  After reviewing the state-provided database, our local reporter may have found, for example, that students in her local school were doing well in 5th grade math, average in 4th grade literacy, but were faring poorly in 3rd grade math.  So, she asked again, how is my local school doing?  How do I make sense of this information?

The problem with the state-reported school data is that the data are generally presented on a grade by grade basis — there are no summary scores for overall school or district performance.  And …. since making sense of educational data is a key part of our mission in the OEP, we used the state data and some simple tools for data analysis and computed overall scores for schools and districts based on student performance on the Arkansas Benchmark Exams and on the nationally norm-referenced Iowa Test of Basic Skills.  These school-level and district-level summary scores are the primary value that the OEP databases add.

Now, our reporter (and others), can more easily answer the important question — how is my local school doing? 

Should we do this?  Well, it is not obvious to all that such summary figures should be computed and presented as some Arkansans have taken issue with our publishing these figures.  The general argument or criticism goes something like this: “how can you describe an entire school with a single number?”  And the answer is, you can’t.  However, overall school scores in math (for example) and overall school scores in literacy (as another example) are important and meaningful and tell an important part of the story of what is going on at each school.  And it would be silly to pretend that people don’t want to know about overall school performance.

Of course there is a great deal of variation in student performance within schools, and those digging into a particular school’s performance level should look at scores by subject, by grade level, and by student subgroup.  Clearly these are important details, but the overall story matters as well and we think it is important to share this information in an accessible way with those interested in the education of Arkansas’ kids.

There is another criticism levied at those of us who publish school performance data.  Some complain that, if we share these school by school results, some in the public will misuse these results to unfairly blame kids, blame teachers, or blame schools.  Thus, the argument goes, let’s not use or disseminate such information for fear of how some might use the data.  We at the OEP couldn’t disagree more strongly with these sentiments — we believe more information is better than less information and that the users of school data are more than sophisticated enough to consider multiple pieces of data at one time!  Obviously, information on the achievement of Arkansas students is important and indeed critical to discussions on how (or if) we need to improve our system.

We understand that school data are imperfect but the alternative of NO DATA is far worse. We can’t simply close our eyes and hope everything is going just fine; we owe our students more than that!

%d bloggers like this: