A HO-IRT BASED DIAGNOSTIC ASSESSMENT SYSTEM WITH CONSTRUCTED RESPONSE ITEMS

ABSTRACT

The aim of the present study was to develop an on-line assessment system with constructed response items in the context of elementary mathematics curriculum. The system recorded the problem solving process of constructed response items and transfered the process to response codes for further analyses. An inference mechanism based on artificial intelligence was implemented with the system to diagnose the bugs in the problem solving process automatically. To examine the performance of the system, a “Multiplication of Fraction” test was constructed and administered to 158 six graders in Taiwan. The results showed that the mean of classification accuracies of the bugs is above 97%, which implies that the proposed system identifies leaning bugs accurately and efficiently. In addition to bug identification, a high-order item response theory (HO-IRT) was applied to estimate the overall and domain abilities. The correlations between the abilities estimated with HO-IRT and the number of bugs were highly correlated, which suggests that the more learning bugs children possessed the lower his/her mathematic abilities would be.