EFFECTIVENESS OF AUTOMATED CHINESE SENTENCE SCORING WITH LATENT SEMANTIC ANALYSIS

ABSTRACT

Automated scoring by means of Latent Semantic Analysis (LSA) has been introduced lately to improve the traditional human scoring system. The purposes of the present study were to develop a LSA-based assessment system to evaluate children’s Chinese sentence construction skills and to examine the effectiveness of LSA-based automated scoring function by comparing it with traditional human scoring. Twenty-seven fourth graders and thirty-one six graders were assessed on single-character sentence making test (subtest 1) and two-character words sentence making test (subtest 2). The outcomes of LSA-based automated scoring methods in three Chinese semantic spaces generated from three type weighting functions were compared to the traditional human scoring. The results showed that LSA-based automated scoring in three different Chinese semantic spaces and traditional human scoring were highly correlated in single-character sentence making test and moderately correlated in two-character words sentence making test. The Chinese semantic space generated from Log-IDF outperformed the other two types of weighting function in the present study.