International Journal of Adult and Continuing Education ISSN 2241-4517 Vol. 3 (12), pp. 001-005, December, 2017. © International Scholars Journals
Full Length Research Paper
Improving Data Driven Decision Making through Assessment Literacy for Respondents
Naomi Jeffery Petersen, Randall S. Davies and Bruce Spitzer
Indiana University South Bend, South Bend, Indiana, U.S.A.
Accepted 17 July, 2017
Abstract
This study examined the use of a college course evaluation instrument in an effort to better understand and improve the assessment data derived from the instrument. The purpose of the analysis was to examine the dimensionality and reliability of the instrument but, more importantly, to understand what the data really tells us and whether assessment literacy would improve data usability. Analysis of the results suggests that the instrument tended to produce internally consistent data. However, the instrument measured predominately one aspect of course quality. In addition, based on results of the assessment literacy exercise, respondents seem to use very different criterion for rating; the constructs being measured generally did not match what was intended; and the questions being asked did not always match the scale being used. As a result the usefulness of any data interpretation and subsequent decisions was deemed suspect. The results of this study also suggest that simple assessment literacy interventions by themselves do not seem to drastically change the ability of raters to score items reliably. A much more comprehensive effort would be needed to produce results that would be beneficial for long term, data-driven decision making.
Key Words: Assessment Literacy, Course Evaluation, Data-Driven Decisions.