The methodological challenges of measuring student outcomes in higher education

The main objective of this study is to empirically test a number of theory-based models (i.e., fixed effects [FE], random effects [RE], and aggregated residuals [AR]) to measure the generic knowledge as well as the degree attainment rates and early labor outcomes gained by students in different programs and institutions in higher education. Our results show the importance of addressing the issue of student selection into programs and institutions in order to reduce selection bias, and they provide suggestive evidence in favor of using FE models. Our findings also confirm our hypotheses that rankings of specific college-program combinations change depending on the different educational and labor outcome measures considered. This finding emphasizes the need to use complementary indicators related to the mission of the specific postsecondary institutions that are being ranked. Given the sensitivity of the models to different model specifications, it is not clear whether they should be used to make any high-stakes decisions in higher education. They could, however, serve as part of a broader set of indicators to support programs and colleges as part of a formative evaluation.

Read More »