How Do Airline Captains Assess Safety-Critical Pilot Performance and Collaboration? An Exploration of Assessment Criteria related to Captains' Situational Awareness and Knowledge
Author(s)
Weber, David
Roth, Wolf-Michael
Mavin, Timothy
Dekker, Sidney
Year published
2013
Metadata
Show full item recordAbstract
Assessing the performance of commercial airline pilots has proven difficult, necessitating a closer look at how pilot performance is assessed in practice. The aim of this study is to explore the basis on which experienced captains assess their peers' safety-critical pilot performance. It is examined whether all assessors determine the same criteria, which for some assessors lead to unacceptable risk. Using the performance assessment grid implemented by their employer as a means for organizing their task, three pairs of airline captains assessed a captain and first officer in two video scenarios. The findings show that ...
View more >Assessing the performance of commercial airline pilots has proven difficult, necessitating a closer look at how pilot performance is assessed in practice. The aim of this study is to explore the basis on which experienced captains assess their peers' safety-critical pilot performance. It is examined whether all assessors determine the same criteria, which for some assessors lead to unacceptable risk. Using the performance assessment grid implemented by their employer as a means for organizing their task, three pairs of airline captains assessed a captain and first officer in two video scenarios. The findings show that assessors use varying criteria to arrive at the same assessment or apply the same or very similar criteria to arrive at different assessments. Addressing certain criteria does not necessarily lead to the same assessment scores; rather, criteria receive different weights in the composition of the total score. Implications are drawn with respect to inter-rater reliability and the diversity of observations
View less >
View more >Assessing the performance of commercial airline pilots has proven difficult, necessitating a closer look at how pilot performance is assessed in practice. The aim of this study is to explore the basis on which experienced captains assess their peers' safety-critical pilot performance. It is examined whether all assessors determine the same criteria, which for some assessors lead to unacceptable risk. Using the performance assessment grid implemented by their employer as a means for organizing their task, three pairs of airline captains assessed a captain and first officer in two video scenarios. The findings show that assessors use varying criteria to arrive at the same assessment or apply the same or very similar criteria to arrive at different assessments. Addressing certain criteria does not necessarily lead to the same assessment scores; rather, criteria receive different weights in the composition of the total score. Implications are drawn with respect to inter-rater reliability and the diversity of observations
View less >
Conference Title
SimTecT2013 : Crossing Boundaries
Publisher URI
Subject
Vocational Education and Training Curriculum and Pedagogy
Technical, Further and Workplace Education
Education Assessment and Evaluation