NCI Evalution: Assessment of Higher Order Thinking Skills via Short Free Text Answer

NCI a hybrid approach of Natural Language Processing (NLP), Convolutional Neural Network (CNN) and Information Theory are applied to assess higher order thinking skills via short free text answer. Data of students' examination scripts are collected from 3 domains namely Information Technology, Engineering and Management with a total of 479 answer scripts. All data are of tertiary level examinations. These answer text are firstly, processed by applying NLP. Next, the pattern of known-information is generated by applying CNN-seq concept and finally based on the pattern the amount of known-information is computed by applying information theory approach. Result shows that the proposed techniques achieved an excellent achievement of 0.89 and above for all domains and all level of higher order thinking skills. LSA based testing achieved from 0.47 to 0.83 agreement. This result supports that the proposed hybrid technique can improve automated short free text assessment in a given scope of domain and competency level.