What a Difference a Rater Makes: Rater-Method Variance and Conscientiousness-Work Performance Correlations

Abstract

Conscientiousness has been demonstrated to reliably predict workplace performance, but previous research has relied almost exclusively on self-other correlational designs, which may inadvertently reduce observed correlations. However, these have been preferred because the main alternative, obtaining ratings of multiple measures from the same source, raises concerns about common-method variance and associated bias. To address these problems, a structural equation modeling analysis of meta-analytic, multitrait-multirater data was conducted to estimate construct-level correlations of conscientiousness with performance. Construct-level correlations were substantially greater than scalecorrected self-other correlations of conscientiousness with overall performance and citizenship performance. Despite substantial method variance, self-self correlations provided the most accurate estimates of construct-level correlations. Implications for understanding the relationship between conscientiousness and performance, especially citizenship performance, are discussed. Reviewers should reconsider what rating methods are appropriate for specific questions, and researchers should use a broader range of sources when estimating construct-level relationships.