Minnesota online testing glitches didn’t affect students’ scores, analysis finds
Christopher Magan, Pioneer Press, August 22, 2013 – As Minnesota rushes to implement more online student proficiency testing, state educators are breathing a sigh of relief that problems with the current tests had minimal impact on performance.
The Human Resources Research Organization reported Wednesday that computer slowdowns, freezes and other problems experienced by thousands of students did not have a widespread impact on their scores.
State leaders are expected to release the results of the Minnesota Comprehensive Assessments, or MCAs, next week.
Over two test days in April, students experienced delayed delivery of test questions, the analysis found. But the exact number of students affected isn’t known because data provided by the testing vendor American Institutes of Research, or AIR, does not include students who were kicked off the system or couldn’t log in.
The problems forced education officials to suspend testing and then extend the time students could take the tests.
Jon Cohen, vice president of assessment for AIR, acknowledged a slowdown with its computer servers April 16, but said its equipment appeared to be functioning normally April 23.
The online tests are designed to handle glitches, whether on individual computers, in entire school buildings or across the system, with little impact on students, Cohen said. The conclusion that performance was not affected shows the system was working properly, he said.
Charlene Briner, state Department of Education chief of staff, said the analysis should give educators more confidence in the test results. The Human Resources Research Organization is a well-respected firm that has done statistical analyses for the state before, she said.
“I’m confident, given the data they had, the work is solid,” Briner said.
Brenda Cassellius, state education commissioner, wrote letters to school leaders and parents hoping to address their concerns about the impact on students. She also reassured them everything would be done to avoid future glitches.
“Every student has a right to an uninterrupted testing experience, which is why I am instructing AIR to take all preventative measures to make sure this does not happen again,” Cassellius wrote.
Yet, online testing troubles still worry educators in Minnesota and across the nation.
Jason Molesky, director of program evaluation in Lakeville, said students in his district experienced repeated testing problems last spring. Molesky agrees with the department’s analysis but remains concerned about future online testing.
“It certainly impacted individuals,” Molesky said. “I don’t think it was as widespread as we initially thought. Last year certainly gave me pause as to whether this is the right way to go.”
Robert Schaeffer, spokesman for the National Center for Fair and Open Testing, said more state leaders should be as cautious as educators like Molesky.
Across the country, states are scrambling to implement online proficiency tests, many of which are aligned with national education benchmarks called the Common Core.
Recent failures should serve as a warning, Schaeffer said.
“When 50 states are up and trying to run online assessments, the consequences could be much larger,” he said.
Instead, he said states should slowly build up online testing systems to make sure they have infrastructure in place to provide the same assessment to every student. Internet access varies across the country, especially in rural areas.
Other states, including Indiana, Kentucky and Oklahoma, also had issues with online proficiency tests last spring.
A review of online testing glitches in Oklahoma, performed by the same company Minnesota hired, found the disruptions had little impact on students’ scores, a news release from the state’s department of education said. Oklahoma uses CTB/McGraw-Hill to administer state tests.
Indiana also uses CTB/McGraw-Hill, and a recent report by the National Center for the Improvement of Education Assessment also found problems similar to those in Minnesota and Oklahoma did little harm to students’ scores.
Despite problems, Minnesota lawmakers voted to require nearly universal online proficiency testing by 2015. State leaders expect to have proposals in September from firms that want to administer the growing system.
In the meantime, state education officials are examining the state’s more than $60 million contract with AIR to decide whether the company should have to pay damages for delays and other problems.
When scores are released next week, Briner and other educators hope all the factors that may have affected scores will be considered. Besides computer glitches, students took more rigorous reading tests last spring for the first time.
“I think it’s important to look at information over time,” Briner said. “It’s also important to remember this is just a snapshot of a student in a particular subject on a particular day.”
Online: View the testing report and the commissioners letters at education.state.mn.us/MDE/SchSup/TestAdmin/index.html.