Item Response Theory (2024)

In medicine and health sciences we often measure constructs that are not directly observable, such as quality of life, pain, or depression. Measurement of such constructs requires use of questionnaires, which validity and reliability is assessed using statistical measurement models. The Item Response Theory (IRT) provides statistical models which model the relationship between the construct and the questionnaire responses. IRT models are increasingly being used in addition to the Classical Test Theory (CTT) model. Both models have different assumptions and the analyses can complement each other. CTT, for example, focuses more on the validity and reliability of sum scores, while IRT focuses more on the validity and reliability of individual items in a questionnaire. In addition, IRT has several advantages over CTT.

An important practical advantage of the IRT based measurement instruments is the flexibility for use in research and clinical practice. For instance, IRT models can be used to create short form questionnaires tailored for specific target groups. Furthermore, a computerized adaptive test (CAT) can be developed which selects the most informative questions for each individual during the administration, based on the previous responses of the individual.

This course is an introductory course on IRT with a focus on its use in the development and validation of questionnaires used in medical and health sciences. We focus on the conceptual understanding of IRT and less on the statistical details. We will cover the underlying principles of IRT and the conceptual differences between IRT and CTT. Then, we will study the theoretical pillars of IRT (e.g. the concept of latent trait, statistical modeling of items and responses, assumptions of IRT). Once the ground is set, we will move on to the practical issues in applications of IRT, such as checking the model assumptions, evaluating the model fit and estimating reliability of measurements. Finally, we will focus on more advanced issues such as differential item functioning, the principles behind item banking and computerized adaptive testing. As an example we will be using the Patient-Reported Outcomes Measurement Information System (PROMIS), as the items included in these questionnaires were selected under the IRT model and are widely used in clinical settings.

After attending the course, students will have an understanding of the item response theory (IRT) in the context of medical measurement. Practically, the participant will be able to evaluate a scientific article which uses IRT and perform IRT analyses.

These main goals are considered to be the sum of the following sub goals:

1. The participant should be able to explain the concept of latent traits and how the concept of latent trait relates to responses on items in terms of building statistical models.
2. The participants should be able to explain differences in classical test theory (CTT) and item response theory (IRT).
3. The participant should be able to explain the basic assumptions of IRT and evaluate their implications in medical measurement context, as well as check these assumptions and judge their impact on model parameters and interpretation of results.
4. The participant should be able to explain the properties of different IRT models, which data they can be applied to and what the consequences are of selecting different models or estimation methods.
5. The participant should be able to estimate IRT model parameters from data and should be able to interpret the results.
6. The participant should be able to interpret model, item and person fit and evaluate the impact of misfit on IRT model parameters.
7. The participant should be able to explain differential item functioning and its implications in medical measurement context, as well as test whether measurement invariance holds for an instrument.
8. The participant should be able to explain what test information is and how it is related to reliability. The participant should be able to calculate reliability and understand how it relates to item selection and computerized adaptive testing.

Target audience

The course is designed for healthcare practitioners and researchers who are active in medical, allied health, psychological, or behavioural research and who deal with the development, evaluation, and interpretation of health measurements using IRT.

Course prerequisites

Participants are expected;

1. to have basic knowledge of epidemiological end statistical methods and clinimetrics, equivalent to the EpidM courses ‘Epidemiological design; basic principles (V10), ‘Regression Techniques (V30)’, and ‘Clinimetrics (V40)’. This is expected to include the following knowledge:
   • Knowledge of correlations, and the basics of multivariable (logistic) regression analyses
   • Basic knowledge of the principles of confounding and effect modification (bias)
   • Knowledge of the concepts and methods of assessing validity (including the basic principles of (uni)dimensionality and factor analysis) and reliability (including the concept of internal consistency, measurement error and standard error of measurement)
2. to have basic knowledge of the statistical programming language R and the environment RStudio. This knowledge is expected to include at least the topics covered in the chapters 1-7 in the online book ‘Introduction to R and RStudio’ (https://bookdown.org/introrbook/intro2r).

If you participate in the course, an instruction video will be available a few weeks before the starting date.

Course material

The course materials (lectures, assignments, feedback of the assignments etc) are available on Canvas, our digital learning environment. The documents will remain available on Canvas for at least one year.

In order to do the computer practicals, you will need R and R studio, which you can download for free at https://cran.r-project.org/

Literature

Epidemiology Master’s students are required to do the exam in order to complete this course. For other students (not enrolled in Epidemiology Master), the exam is optional and costs €160,- per registration.

Students who complete the exam with a pass grade will receive an exam statement mentioning the academic credits (ECs).

You can register for the exam on the Exams page. Registration will close 3 weeks prior to the exam.

Please note that you need to pass the exam in order to receive credits (EC).

A certificate of participation will be granted to all students who have attended at least 80% of the classes. The number of contact hours will be stated on the certificate.

Only for Dutch medical specialists!

On the final day of the course, you need to sign the attendance list if you wish to obtain KNMG accreditation credits (PE-points).

To qualify for these credits, there is an attendance requirement of 100%.