Development and application of the Commitment to Profession of Medicine Scale using classical test theory and item response theory

Aim To determine the level of professional commitment of medical students by developing and applying a new scale. Methods The study enrolled 999 students of Çanakkale Onsekiz Mart University, School of Medicine. Factor analysis, reliability analysis, and item analysis were performed based on the classical test theory and item response theory. The data obtained through scale application were analyzed using factorial ANOVA. Results The Commitment to Profession of Medicine Scale was identified as a unidimensional scale consisting of nine items. The scale in its present form explained 51% of the variance in commitment to profession of medicine. The reliability was 0.88. The scale application revealed that female students had higher commitment than male students. The highest level of commitment was observed in third- and first-year students, students with the lowest level of family income, and students whose ideal profession was medicine. Conclusion There are many factors affecting professional commitment levels of university students. Therefore, it is of great importance to examine students' commitment at an early stage. In addition, the experiences of students during university years are important since they directly affect the commitment level. Considering these factors, teachers should support their students and strive to increase their commitment levels.

The main goal of medical education is to equip students with professional competence and prepare them for lifelong learning. Medical students should gain the motivation and skills necessary to sustain the qualifications they acquired, to acquire new qualifications, and to commit themselves to professional values (1).
Commitment to profession has been defined in several ways: as an attitude toward the profession and the work being carried out (2), as a psychological relationship between the profession and the individual, and as emotional reactions to the profession (3). Aranya et al used the term "professional bonding" (4), which relates to emphasizing identity, putting in effort in the work one performs, and adherence to professional goals, values, norms, and ethical principles (5).
Commitment of an individual to one's job can be described as exhibiting performance in cognitive, emotional, and physical context (6)(7)(8) and emotional states creating positive excitement (9). Meyer et al identified three features of commitment to profession (10): emotional commitment, commitment to continuity, and normative commitment. Emotional commitment means being emotionally willing to pursue a profession. Individuals who are emotionally committed to their profession will have the chance to progress in their career and grow professionally. Commitment to continuity means pursuing a profession in the case of a conflict between leaving and staying. Normative commitment means that the rules of the profession have become a part of the individual's personality. Individuals with high commitment to profession exhibit their talents at a highest level, strive to develop their career, and do not quit the profession (10,11). For this reason, it is very important to assess the level of commitment to the profession of medicine among medical students. Tools that measure commitment are available across all professions, especially in those related to educational sciences and management. However, there is no scale measuring the professional commitment to medicine. Therefore, this study aimed to develop and apply a selfreport measurement tool that assesses medical school students' commitment to the profession of medicine.

Data collection instrument
A measurement tool was developed by using a previously described method (12)(13)(14). First, we identified the characteristic that was to be measured by the tool. In order to create an item pool, relevant databases were searched using the words "commitment, " "medicine commitment, " and "commitment to health" in the title or abstract. The retrieved publications were examined, and possible items were included in the scale. In addition, a focus group semi-structured interview was conducted with 10 students from different classes of the medical school. The participants provided consent for using the interview recordings. The recordings were analyzed, and possible items were included in the scale. From both sources, a pool of 38 items was created and was discussed by a panel team consisting of two medical education experts, one measurement and evaluation field expert, and two researchers. The panel agreed on a 26-item draft scale. The responses to the items were "strongly disagree, " "disagree, " "not sure, " "partly agree, " and "strongly agree. "

Research group
After validity and reliability testing, the scale was administered to 999 students of Çanakkale Onsekiz Mart University, School of Medicine (COMUSM) on September 23, 2019. The obtained data were divided into three data sets, and each was analyzed using a different approach: 1) explanatory factor analysis (EFA), item response theory (IRT) analysis, and reliability analysis, 2) confirmatory factor analysis (CFA), and 3) scale application and comparison analyses (Table 1). The minimum number of data needed for factor analysis and the number of participants in the process of developing a measurement tool was determined according to previous studies (15)(16)(17)(18)(19)(20). We made certain that the study enrolled at least ten times as many participants as is the number of items in the measurement tool.

Data analysis
The randomness of missing data was tested by using SPSS with "Estimate Mean. " The missing data were found to be random (P > 0.05) and, since the items in the draft scale showed a multivariate normal distribution, were completed by the mean rank of the respective variable. The suitability of the data file for factor analysis was tested with Kaiser Meyer Olkin (KMO) and Bartlett's test of sphericity. Possible factorizations that may occur in the factor analysis were tested with Varimax rotation of the axes. Fit indices for CFA were used to assess the model fit.
The validity and reliability of the scale were assessed with IRT methods. To be able to use IRT methods for the items requiring a rated response (ie, strongly agree, partly agree, disagree, strongly disagree), unidimentionality was assessed with EFA and local independence with the Q3 statistic (21). The IRT calibrations were established using the "mirt v. 1.30" (22) package of the R v. 3.5.0 software.

Validity and reliability as assessed with CTT methods
The structure of the 26-item scale was assessed with EFA using the principal axis factoring (23). The analysis re-vealed a three-factor structure, but this structure was not verified by CFA.
In an effort to reduce the number of factors and find a simpler solution, the scree plot of the factor analysis was examined, and the factor analysis was reconducted with a distinct factor where the slope was the steepest. The factor analysis with a single factor showed a KMO value of 0.913 and a Bartlett's test of sphericity value of 1538.019 (sd = 36, P < 0.001), which are considered satisfactory values (24,25).
In this single-factor structure, the items making the greatest contribution were 2, 4, 7, 8, 13, 16, 18, 19, and 21 (Table 2, Figure 1). The nine items exhibited a single dominant factor character with an eigenvalue of 4.592. The Cronbach alpha reliability coefficient of the nine items was 0.876 (Table 2), indicating a high level of reliability (25).
A CFA was performed to verify the structure obtained with EFA (Table 3, Figure 2). Some of the obtained fit-indices indicated perfect fit and some indicated acceptable fit when compared with those available in the literature. This confirmed the structure obtained with EFA.

Validity and reliability as assessed with IRT methods
The use of IRT requires unidimensionality and local independence assessment. Unidimensionality assessment with EFA showed that the scale exhibited a nine-item, unidimensional structure. Local independence was assessed with the Q3 statistic as suggested by Yen (21). Q3 showed that none of the nine items impaired local independence. Item calibrations were found for the nine items using the IRT-based generalized partial credit model (GPCM). Follow- ing GPCM, S_χ 2 , degrees of freedom, root mean square error of approximation (RMSEA), and level of significance statistics of the items were calculated ( Table 4).
The RMSEA values of the items were less than 0.08, indicating a good item fit. Based on this result, it was decided that the single-factor, nine-item scale obtained with EFA satisfied the model fit as per GPCM. The "a" and "b" parameters and standard errors of the items that satisfied model fit as per GPCM were estimated ( Table 5). The estimations made according to GPCM (Akaike information criterion, Bayesian information criterion, log likelihood, χ 2 , P < 0.001) indicated the fit of scale items.
The item-characteristic curves ( Figure 3) showed that the items included in the scale together with their options were significant and had a good performance for different levels of ability. The discriminative ability of the response categories of the Item 13 was relatively lower compared with the other items. The response categories of the scale items were understood by the participants and had a discriminative function.
According to the item information functions, the scale items gave more information for respondents who had a low commitment level. The most informative items were 2, 7, and 8. Other items were moderately informative ( Figure  4, Figure 5).
The Commitment to Profession of Medicine gives information on "professional commitment. " The scale was most informative for individuals with a commitment level in the interval between -3 and 0.5. The marginal reliability coefficient of the scale was 0.875, which is almost the same as the Cronbach alpha internal consistency value calculated according to CCT.

The application of medical students' Commitment to the Profession of Medicine Scale
Female students had a higher commitment level than male students ( Table 6). The highest commitment level was observed among third-year students, followed by   first-, second-, sixth-, fifth-, and fourth-year students. Students who had a family income between 0 and 456 € had the highest commitment level, followed by those who had a family income of 914 € and over and those who had a family income between 457 and 913 €. Students who followed medical literature besides taking notes for their lessons had a higher commitment level than those who did not. The highest commitment level was observed among the students whose "ideal profession was medicine, " followed by those who "chose medicine for other reasons, " those who "chose medicine due to its high social status, " those who "chose medicine because it is a profession with guaranteed employment and income, " and finally those who "chose medicine because their university examination score was sufficient. " The levels of commitment to profession with respect to students' sex, study year, family income, following medical literature besides taking notes for their lessons, and the reason for choosing medicine were compared using the factorial ANOVA method with Bonferroni post-hoc test (Table 7, Table 8).
Female students had a significantly higher commitment (F (1-496) = 24.529, P < 0.05), with a moderate effect size (η 2 = 0.05) (33). Commitment to profession significantly differed according to the study year (F (5-496) = 5.582, P < 0.05), with a moderate effect size (η 2 = 0.05). First-year and third-year students had higher commitment to profession than fourth-year students. Third-year students had higher commitment to profession than fifth-year students. Students whose family income was between 0 and 456 € had a significantly higher commitment to profession than those whose family income was between 457 and 913 € (F (2-496) = 3.796, p≤0.05), with a small effect size (η 2 = 0.02).
The interaction of the basic effects of study year and family income created a significant difference in commitment to profession (F (10-496) = 2.245, P < 0.05), with a moderate ef-     fect size (η 2 = 0.04). First-year students with family income of 0-456 € had a higher commitment level than secondyear students with family income of 457-913 €. Secondyear students with family income of 0-456 € had a higher commitment level than second-year students with family income of 457-913 €.
Commitment to profession significantly differed among students with different reasons for studying medicine (F (4-522) = 35.978, P < 0.05), with a large effect size (η 2 = 0.22). Students who reported that medicine was their ideal profession had higher commitment than students who listed other reasons for studying medicine.  Commitment to profession did not significantly differ among students who followed medical literature besides taking notes for lessons, and the interaction of this variable with the reasons for studying medicine did not create a significant difference (P > 0.05).

DISCUSSION
In this study, using both CTT and IRT, we developed a onedimensional, nine-item Commitment to Medical Profession Scale for university students. The scale application showed that female students had higher commitment levels than male students. Higher commitment levels in women were also observed in studies among nurses (34,35) and teachers (36), the latter showing that women were more committed than male teachers in all domains except commitment to work group. Contrary to this, higher commitment levels among men were found among students (37), administrative staff of a state university (38), and nurses (39). In some studies, commitment level did not significantly differ between the sexes (40,41).
Another important result of our study was the relationship between the professional commitment levels and family income. Namely, students with low or high family income were more committed to profession than students with moderate family income. Similarly, Saruc et al (41) showed that the professional commitment level of social workers increased with the increase in income level.
We also observed higher professional commitment of students who followed scientific literature besides taking course notes. These findings show that students who develop their professional skills were more committed to their profession. Since best practices and patient care standards are constantly evolving, physicians are expected to follow the recent developments in the field (42,43) and undergo continuous training in order to provide the best care for the patients. Within this context, another study reported that nurses believed that continuing education was as an investment into specialization in the field (44).
Considering all this, students should be encouraged to get involved in professional development events.
Another important finding was that professional commitment level of first-year and third-year students was higher than that of fourth-year students. In addition, the commitment level of third-year students was significantly higher than that of fifth-year students. Therefore, the students' commitment level decreased during the course of the studies. In other words, it was negatively affected by students' experiences during university education. These experiences could include formal internships, jobs, and social work, as well as mentoring and counselling they received (45).
Students who stated that they chose this profession because it was their ideal job had higher commitment level. This indicates that choosing a profession based on students' life goals is more important than basing this choice on other factors. Moreover, career choice is one of the biggest dilemmas and challenges in students' lives (46) and is affected by many factors, such as family, passion, salary, and experiences (47). In other studies, students reported their career choice to be significantly affected by salary (48,49) but also by cultural factors, career aims, professional status, as well as family factors (50).
A strength of this study is a high participation rate -only 11 (1%) students refused to participate. Despite the high number of participants, the fact that the study was carried out in only one medical school in Turkey limits its generalizability. In order to ensure that the validity and reliability of the scale did not depend on the group, IRT methods were applied alongside with CTT methods. On the other hand, repeating the validity and reliability assessment on data obtained from different groups can contribute to the development of the scale.
In conclusion, the results of this study revealed different factors affecting professional commitment levels of university students, the most significant of which were sex, family income, attending professional development events, fol- lowing the recent developments in the field of medicine, a high social status of medicine, and salary. Therefore, it is important to evaluate students' professional commitment level at an early stage. In addition, universities should create opportunities for students to take part in experiences that positively affect their commitment, such as professional development trainings. Teachers should also strive to support students in their professional development and enable access to medical literature.
Funding None.
Ethical approval given by the Committee of COMU Clinical Research Ethics (1900075321).
Declaration of authorship both authors conceived and designed the study; AMAK acquired the data; CT analyzed and interpreted the data; CT drafted the manuscript; AMAK critically revised the manuscript for important intellectual content; both authors gave approval of the version to be submitted; both authors agree to be accountable for all aspects of the work.