Send to

Choose Destination
See comment in PubMed Commons below
Acad Med. 2017 Nov;92(11S Association of American Medical Colleges Learn Serve Lead: Proceedings of the 56th Annual Research in Medical Education Sessions):S21-S25. doi: 10.1097/ACM.0000000000001914.

Formative Assessment in an Integrated Curriculum: Identifying At-Risk Students for Poor Performance on USMLE Step 1 Using NBME Custom Exam Questions.

Author information

J.M. Brenner is associate dean of curricular integration and assessment, Hofstra Northwell School of Medicine, Hempstead, New York. J.B. Bird is assessment and evaluation analyst, Hofstra Northwell School of Medicine, Hempstead, New York. J.M. Willey is professor and chair, Department of Science Education, Hofstra Northwell School of Medicine, Hempstead, New York.



The Hofstra Northwell School of Medicine (HNSOM) uses an essay-based assessment system. Recognizing the emphasis graduate medical education places on the United States Medical Licensing Examination (USMLE) Step exams, the authors developed a method to predict students at risk for lower performance on USMLE Step 1.


Beginning with the inaugural class (2015), HNSOM administered National Board of Medical Examiners (NBME) Customized Assessment Service (CAS) examinations as formative assessment at the end of each integrated course in the first two years of medical school. Using preadmission data, the first two courses in the educational program, and NBME score deviation from the national test takers' mean, a statistical model was built to predict students who scored below the Step 1 national mean.


A regression equation using the highest Medical College Admission Test (MCAT) score and NBME score deviation predicted student Step 1 scores. The MCAT alone accounted for 21% of the variance. Adding the NBME score deviation from the first and second courses increased the variance to 40% and 50%, respectively. Adding NBME exams from later courses increased the variance to 52% and 64% by the end of years one and two, respectively. Cross-validation demonstrated the model successfully predicted 63% of at-risk students by the end of the fifth month of medical school.


The model identified students at risk for lower performance on Step 1 using the NBME CAS. This model is applicable to schools reforming their curriculum delivery and assessment programs toward an integrated model.

[Indexed for MEDLINE]
PubMed Commons home

PubMed Commons

How to join PubMed Commons

    Supplemental Content

    Full text links

    Icon for Wolters Kluwer
    Loading ...
    Support Center