There are two basic theory to the conceptualizations of job analysis accuracy:- Classical Test Theory and Generalizability Theory. Classical test theory suggests that a "true score" esists for a given job and that true scores are stavle over time. Any measurement variation error are eliminated or aggragated across time or sources. Through this, researchers have commonly aggregated job analysis information in order the get the "true score" for every particular job, then the quality of job analysis datas are indexed with estimation of interrater reliability.
However, classical test theory can only estimate one source of error at a time. To overcome this, generalizability theory which concerns with the dependability of behavioral measures are adopted. The dependability involves the accuracy of generalizing from an observed score to the average score over different kind of situations. However, this theory allows one to segment the sources into multiple sources or facets, such as the methods of data collection and sources of data.
The ability to estimate multiple sources of measurement error simultaneously enable us to examine how different methods, sources of data, location, incumbent abililty and other possible factors impact the accuracy of job analysis data.
Cronbach's (1995) method of differentiating the accuracy scores have four components:- elevation, differential elevation, stereotype accruracy, and differential accuracy. Elevation refers to the way raters use response scales and is a function of the differences between the average of a person's ratings and the average expert score. Differential elevation refers to how close an average job rating across all dimensions to an average expert rating would be. The abiltiy to predict the profile of dimension means across the jobs are reflected by stereotype accruracy while diffrential accuracy reflects the abliity to predict the differences between jobs and individual dimensions.
Morgeson and Campion (1997) came up with...