Feasibility, validity and reliability of Procedure Based Assessment as a work based assessment tool in cardiothoracic surgery: A pilot study

Abstract

Background: The change in surgical training from an apprentice model to competency based model has necessitated the introduction of a number of work based assessment tools. The Procedure Based Assessment (PBA) involves assessment of trainee performance at the highest level of competence at the “does” level of competence with the trainee performing whole operations in the work place.

Methods: Review of the evidence for the use of workplace based assessment tools and in particular procedure based assessment with an aim to assess the feasibility, validity and reliability of a PBA of Coronary Artery Bypass Surgery (CABG) in Cardiothoracic Surgery in a pilot study.

Results: The literature in support of PBA as a work assessment tools is lacking and sparse. The application of PBA during ten observations of trainees performing coronary artery bypass grafts and assessment by two trainers demonstrated the feasibility, acceptability and reliability of CABG PBA assessment too. Face and content validity was confirmed although construct and predictive validity of the assessment form could not be confirmed. The tool didn’t impact significantly on time resources and was seen to be useful by the majority of trainees and trainers as formative  assessment.

Conclusions: PBA is a useful tool in formative assessment of trainees providing a structured and reproducible form of feedback and evaluation of surgical competence. The form in use may benefit from the additional of a numerical scoring system to improve its construct validity in order to differentiate the levels of progress and be able to compare the standing of a trainee at different time points in their training.