Format

Send to

Choose Destination
J Appl Meas. 2013;14(4):389-99.

Assessing DIF among small samples with separate calibration t and Mantel-Haenszel χ² statistics in the Rasch model.

Author information

1
National Council of State Boards of Nursing (NCSBN), 111 E. Wacker Drive, Ste. 2900, Chicago, IL 60601-4277, USA, awoo@ncsbn.org.

Abstract

The National Council Licensure Examination (NCLEX) program has evaluated differential item functioning (DIF) using the Mantel-Haenszel (M-H) chi-square statistic. Since a Rasch model is assumed, DIF implies a difference in item difficulty between a reference group, e.g., White applicants, and a focal group, e.g., African-American applicants. The National Council of State Boards of Nursing (NCSBN) is planning to change the statistic used to evaluate DIF on the NCLEX from M-H to the separate calibration t-test (t). In actuality, M-H and t should yield identical results in large samples if the assumptions of the Rasch model hold (Linacre and Wright, 1989, also see Smith, 1996). However, as is true throughout statistics, "how large is large" is undefined, so it is quite possible that systematic differences exist in relatively smaller samples. This paper compares M-H and t in four sets of computer simulations. Three simulations used a ten-item test with nine fair items and one potentially containing DIF. To address instability that may result from a ten-item test, the fourth used a 30-item test with 29 fair items and one potentially containing DIF. Depending upon the simulation, the magnitude of population DIF (0, .5, 1.0, and 1.5 z-score units), the ability difference between the focal and reference group (-1, 0, and 1 z-score units), the focal group size (0, 10, 20, 40, 50, 80, 160, and 1000), and the reference group size (500 and 1000) were varied. The results were that: (a) differences in estimated DIF between the M-H and t statistics are generally small, (b) t tends to estimate lower chance probabilities than M-H with small sample sizes,

PMID:
24064579
[Indexed for MEDLINE]

Supplemental Content

Loading ...
Support Center