Evaluating instructors’ classification of programming exercises using the revised Bloom’s taxonomy

It is a well-known fact that different instructors understand Bloom’s taxonomy differently, thus they classify given test exercises into different levels of the taxonomy. The article reports on the impact of training instructors on the use of the taxonomy for programming courses with two innovations: encouraging instructors to classify each exercise into several cognitive processes necessary to solve it, and providing them with a wide range of examples of classified programming exercises. We evaluated the impact of these innovations in two training sessions. In the first session, the revised Bloom’s taxonomy was introduced to the participants, and in the second session, the innovations were presented. In both sessions, participants were asked to classify ten exercises. The results show that classification into several cognitive processes was natural to instructors. However, other results were mixed. On the one hand, accuracy was increased, and confidence was increased for experienced participants. On the other hand, variation was increased, and confidence was decreased for non-experienced participants. Results show that our approach assists instructors in classifying exercises, but higher increases of expert instructors suggest that longer training is necessary for non-expert instructors.