Two instructors observe the same crew, in the same scenario, demonstrating the same behaviours. One sees acceptable performance. The other sees a competency deficiency. Both are experienced. Both know the grading system. Both can explain their decision.
A competency-based assessment system is only as reliable as the instructors applying it.
Concordance is more than the same grade
Effective concordance is consistency through the assessment process: observation, interpretation, competency attribution, grading and the resulting training decision. Identical numbers can hide very different diagnoses.
When concordance breaks down
01 — Everyone agrees, but everyone is wrong
A group may show strong agreement with each other while remaining misaligned with the organisation's validated assessment standard. Agreement and alignment need to be understood separately.
02 — The “strict instructor”
Operational data may show one instructor grading lower than others. That is a signal, not a conclusion. Different instructors may have seen different crew performance. Common assessment material helps separate instructor variation from crew variation.
Don't turn a data pattern into a people problem before understanding what created the data.
03 — Same grade, different diagnosis
Three instructors may all award Grade 2 while attributing the performance to Situation Awareness, Problem Solving and Decision Making, or Leadership and Teamwork. Numerical agreement can therefore hide poor competency diagnosis.
04 — The competency nobody grades
A competency that is rarely identified, or almost always sits in the middle of the scale, may expose unclear observable behaviours, weak scenario exposure, grading ambiguity or instructor reluctance to differentiate performance.
05 — The experienced instructor entering CBTA
Years of instructional experience remain valuable, but they do not automatically equal years of competency-based assessment experience. CBTA adds behavioural diagnosis to traditional manoeuvre and tolerance assessment.
Concordance is a process, not an annual event
Eight practical tips
Use the same performance. Common material gives a cleaner basis for comparison. Assess first, discuss second. Capture independent judgement before group influence. Separate agreement from alignment. A group can consistently apply the wrong standard. Look behind the grade. Compare behaviours, competency attribution and rationale. Look for trends. Isolated differences rarely tell the whole story. Challenge the system before blaming the instructor. Review descriptors, scenarios and reference assessments. Use difficult cases. Borderline cases reveal interpretation differences. Reassess. Standardisation only closes the loop when improvement is measured.
Don't ask only “Did our instructors give the same grade?” Ask whether they observed the same behaviours, understood their significance and reached a comparable training conclusion.
The real value of concordance
The objective is not identical instructors or identical judgement. It is professional judgement exercised within a common, defensible assessment standard. Differences are signals that can improve both instructor performance and the training system itself.