Box 1: The importance of response categories with respect to the validity and reliability of tools.()
Left: Printed from Lozano et al () the interrelation between number of response categories (here: number of alternatives: 2 – 9) and sample size in relation to Cronbach alpha (α). From the graph it is clear that reliability of the unidimensional scale decreases with fewer responses; i.e., fewer responses decreases the variability and as such the reliability (here expressed by Cronbach alpha). The percentage of variance explained by the first factor decreases as the number of responses is reduced, regardless of the correlations between the items and sample size.() This might be extrapolated to multiple dimensions or factors.
Right: Hypothetical visualization of the `perceived' range of response (never to always/often/usually) with number of response categories.(, ) The fewer response categories, the more forced a choice might `sense'. In addition, assigning a label and a numerical value influences the psychometrics of a tool (e.g., the perceived range, the summated score – e.g., a zero). For instance, the label might blur the hypothetical equal spacing; when 3 responses are given `poor' – `good' – `very good' the differentiation between good and very good might not always be clear-cut, for example respondents circling both (possible solution: replace `very good' by `excellent'). Another example, `often', `occasional' and `sometimes' might have a different value (interpretation) from person to person, or even culturally (e.g., add the frequency when possible to avoid misperception and thus inaccurate responses). With hypothetically equal spacing (or perceived value) among responses `sometimes'(*) would fall in the middle, but depending on the perception of the labels or the assigned numerical value substantial shifts might occur; i.e., horizontally being less or more distinctive, and vertically being gradual or abrupt. It is of utmost importance that each person in each study perceives the responses in identical ways.
An analogy would be a grade at school; i.e., mathematical they are at equal distance but some might perceive a 7 closer to 8 and 9, whereas other see it more towards 6, or the difference between a 6 and 7 on a 10 point grade might be perceived large whereas on a 20 point grade it might be perceived small.
A: 7-point scale from 1–7 with 1: never, 2: rarely, 3: occasionally, 4: sometimes*, 5: often, 6: almost always, and 7: always.
B: 5-point scale from 1 – 5 with 1: never, 2: just a few times, 3: sometimes*, 4: quite often, and 5: always.
C: 5-point scale from 0 – 4 with 0: never, 1: just a few times, 2: sometimes*, 3: quite often, and 4: always.
D: 5-point scale from 0 – 3 with 0: never, 0: rarely, 1: occasionally*, 2: often, and 3: very often.
E: 3-point scale from 1 – 3 with 1: never/rarely, 2: sometimes*, and 3: usually.
F: 3-point scale from 0 – 2 with 0: never/rarely, 1: sometimes*, and 2: usually.