Publications

Article Published: Making the Cut: Comparing Methods for Selecting Cut-Point Location in Logistic Regression 1

Dr. WeissJanuary 9, 2020January 9, 20201 Comment

Drs. Brandi A. Weiss and William Dardick published an article titled, "Making the cut: Comparing methods for selecting cut-point location in logistic regression" in the Journal of Experimental Education: Measurement, Statistics, and Research Design (DOI: 10.1080/00220973.2019.1689375).

Abstract

Classification measures and entropy variants can be used as indicators of model fit for logistic regression. These measures rely on a cut-point, c, to determine predicted group membership. While recommendations exist for determining the location of the cut-point, these methods are primarily anecdotal. The current study used Monte Carlo simulation to compare mis-classification rates and entropy variants across four cut-point selection methods: default 0.5, MAXCC, nonevent rate, and MAXSS. Minimal differences were found between methods when group sizes were equal or large between-groups differences were present. The MAXSS method was invariant to group size ratios, however, yielded the highest total misclassification rate and highest amount of misfit. The 0.5 and MAXCC methods are recommended for use in applied research. Recommendations are provided for researchers concerned with small group classification who may use the MAXSS method. EFR and EFR-rescaled were less influenced by cut-point location than classification methods.

Keywords: logistic regression; model-fit; cut-point methods; entropy; classification; misclassification.

Article Published: Entropy-Based Measures for Person Fit in Item Response Theory

Dr. WeissApril 10, 2017April 4, 2018Leave a comment

Drs. William Dardick and Brandi Weiss published an article titled Entropy-based measures for person fit in item response theory in Applied Psychological Measurement (DOI:10.1177/0146621617698945).

Abstract

This article introduces three new variants of entropy to detect person misfit (E_i, EM_i, and EMR_i), and provides preliminary evidence that these measures are worthy of further investigation. Previously, entropy has been used as a measure of approximate data–model fit to quantify how well individuals are classified into latent classes, and to quantify the quality of classification and separation between groups in logistic regression models. In the current study, entropy is explored through conceptual examples and Monte Carlo simulation comparing entropy with established measures of person fit in item response theory (IRT) such as l_z, l_z*, U, and W. Simulation results indicated that EM_i and EMR_iwere successfully able to detect aberrant response patterns when comparing contaminated and uncontaminated subgroups of persons. In addition, EM_i and EMR_i performed similarly in showing separation between the contaminated and uncontaminated subgroups. However, EMR_i may be advantageous over other measures when subtests include a small number of items. EM_i and EMR_iare recommended for use as approximate person-fit measures for IRT models. These measures of approximate person fit may be useful in making relative judgments about potential persons whose response patterns do not fit the theoretical model.