Search results
Classical test theory, also known as true score theory, assumes that each person has a true score, T, that would be obtained if there were no errors in measurement. A person’s true score is defined as the expected score over an infinite number of independent administrations of the scale.
- Classical Test Theory
- Item Response Theory
- Similarities Between Classical Test Theory and Item Response Theory
- Differences Between Classical Test Theory and Item Response Theory
- Linear Test Design
- So… How to Choose?
CTT is an approach that is based on simple mathematics; primarily averages, proportions, and correlations. It is more than 100 years old, but is still used quite often, with good reason. In addition to working with small sample sizes, it is very simple and easy to understand, which makes it useful for working directly with content experts to evalua...
IRT is a much more complex approach to analyzing tests. Moreover, it is not just for analyzing; it is a complete psychometric paradigm that changes how item banks are developed, test forms are designed, tests are delivered (adaptive or linear-on-the-fly), and scores produced. There are many benefits to this approach that justify the complexity, and...
CTT & IRT are both foundational frameworks in psychometrics aimed at improving the reliability and validity of psychological assessments. Both methodologies involve item analysis to evaluate and refine test items, ensuring they effectively measure the intended constructs. Additionally, IRT and CTT emphasize the importance of test standardization an...
Test-Level and Subscore-Level Analysis
CTT statistics for total scores and subscores include coefficient alpha reliability, standard error of measurement (a function of reliability and SD), descriptive statistics (average, SD…), and roll-ups of item statistics (e.g., mean Rpbis). With IRT, we utilize the same descriptive statistics, but the scores are now different (theta, not number-correct). The standard error of measurementis now a conditional function, not a single number. The entire concept of reliability is dropped, and repl...
Item-Level Analysis
Item statistics for CTT include proportion-correct (difficulty), point-biserial (Rpbis) correlation (discrimination), and a distractor/answer analysis. If there is demographic information, CTT analysis can also provide a simple evaluation of differential item functioning(DIF). IRT replaces the difficulty and discrimination with its own quantifications, called simply b and a. In addition, it can add a c parameter for guessing effects. More importantly, it creates entirely new classes of statis...
Scoring
CTT scores tests with traditional scoring: number-correct, proportion-correct, or sum-of-points. IRT scores examinees directly on a latent scale, which psychometricianscall theta.
CTT and IRT differ in how test forms are designed and built. CTT works best when there are lots of items of middle difficulty, as this maximizes the coefficient alpha reliability. However, there are definitely situations where the purpose of the assessment is otherwise. IRT provides stronger methods for designing such tests, and then scoring as wel...
There is no single best answer to the question of CTT vs. IRT. You need to evaluate the aspects listed above, and in some cases other aspects (e.g., financial, or whether you have staff available with the expertise in the first place). In many cases, BOTH are necessary. This is especially true because IRT does not provide an effective and easy-to-u...
Abstract. Despite theoretical differences between item response theory (IRT) and classical test theory (CTT), there is a lack of empirical knowledge about how, and to what extent, the IRT- and CTT-based item and person statistics behave differently.
- Xitao Fan
- 1998
Abstract. Classical Test Theory (CTT) is fundamentally a test-level theory, not an item-level theory. The CTT true score simply indicates the expected value of the observed score over repeated testings under the same conditions.
Feb 16, 2018 · Summary. This chapter presents an overview of classical test theory (CTT), strong true-score theory (STT) as a variant of CTT with stronger assumptions and a focus on items instead of test scores, and generalizability (G) theory as an extension of CTT.
- Christine E. DeMars
- 2018
Jun 1, 1998 · Despite theoretical differences between item response theory (IRT) and classical test theory (CTT), there is a lack of empirical knowledge about how, and to what extent, the IRT- and CTT-based item and person statistics behave differently.
