An Overview of the Generalizability Theory and

its Implications in Medical Education

Mortaz Hejri S¹ (MD, MSc, PhD candidate), Janani L² (PhD), Jalili M³* (MD)

¹ Department of Medical Education, Tehran University of Medical Sciences, Tehran, Iran

² Department of Biostatistics, School of Public Health, Iran University of Medical Sciences, Tehran, Iran

³ Department of Medical Education, Department of Emergency Medicine,

Tehran University of Medical Sciences, Tehran, Iran

Original Article

Received: 6 Apr 2015, Accepted: 21 Jun 2015

Abstract

Introduction: All measurements, including medical students’ assessment, are potentially prone to error from various sources. Considering the importance of maintaining the quality of assessment methods in medical education, finding methods to estimate the error and to decrease it, is essential. Many developments have been achieved in measurement theories during recent years, which can be used in this way.

Methods: In this study, following reviewing the classical test theory (CTT) and its limitations, we discuss the generalizability theory (GT) using some examples of the Objective Structured Clinical Examination.

Results: According to the CTT, students’ scores in exams are not their true scores and are accompanied with some levels of error. Determining reliability of tests helps us to assess how much the random error has been affected the score. In the CTT, it is impossible to consider all error sources simultaneously. To cope with this limitation, the GT specifies share of various sources of error based on the ANOVA models. This theory estimates reliability of tests and offers some recommendations for improving their generalizability.

Conclusion: In CTT, specification of effect of various sources of errors is not possible, whereas GT can be used to identify share of each error source. Hence, the test reliability coefficient can be estimated and also a more reliable test may be planned.

Key words: measurement, classical test theory, generalizability theory, reliability  

Please cite this article as follows:

Mortaz Hejri S, Janani L, Jalili M. An Overview of the Generalizability Theory and its Implications in Medical Education. Hakim Health Sys Res 2015; 18(2): 146- 152.