OBJECTIVES: To describe the conceptual development of a measure for assessing pharmacist knowledge of herbal and dietary supplements. METHODS: A standardized approach to constructing a multiple-choice competency examination following 8 pre-specified criteria (eg, specifying the target spectrum of herbal and dietary supplements) was used to create an item bank. The quality of each item was evaluated by 5 herbal and dietary supplement content experts based on specific criteria in 3 rounds of review. RESULTS: From 122 initial items, 56 items were retained for the item bank representing 4 content areas: efficacy/effectiveness, safety, drug-supplement interactions, and regulation. The experts tended to agree that the constructed items represented a wide range of difficulty. CONCLUSION: The initial development of a conceptually based item bank/measure of pharmacist herbal and dietary supplement knowledge lays the groundwork for a large-scale validation study. The measure should be useful as a standalone tool and as a component of a knowledge, attitude, and behavior survey for the assessment of pharmacist traits related to herbal and dietary supplements.