Educational evaluation of cardiothoracic courses: can we do it justice?

Abstract

Objectives
Educational evaluation is the systematic appraisal of the quality of teaching and learning. It drives the development and changes the content and delivery of educational encounters. Our aim was to examine the feasibility to evaluate a European Association of Cardiothoracic Surgery (EACTS) course using Kirkpatrick’s 4 levels of evaluation.

Methods
The Open and Endovascular Aortic Therapy Course delivered in March 2012 was evaluated using Kirkpatrick’s four levels of evaluation.

Results
The evaluation of the Aortic Therapy Course at level I demonstrated the satisfaction of the participants with the course and content in general as well as each lecture individually. Evaluation at Level II demonstrated an improvement in post course multiple choice questions pretest scores compared with scores of the same test (68±7.8 vs 58±9.2; p<=0.1). Level III involved a follow up questionnaire to all the participants 1 year following attendance of the course. 71.4% of the participants felt that attending the course changed their practice while 53.3% felt it allowed them to introduce a new procedure into their practice and in 93.3% it allowed them to better deal with more complex cases. For the Level IV evaluation, a questionnaire was sent to the participants’ supervisors 1 year post course completion. 100% of the respondents (n=2) answered that there was no change in the trainee’s practice, a more complex procedure wasn’t introduced into common practice and the trainee did not better deal with a more complex case following the attendance of the Open & Endovascular Aortic therapy course.

Conclusion
A comprehensive evaluation of an educational course is feasible although Level IV was only evaluated indirectly and superficially.