• We are sorry, but NCBI web applications do not support your browser and may not function properly. More information
Logo of bmcfpBioMed Centralsearchsubmit a manuscriptregisterthis articleBMC Family Practice
BMC Fam Pract. 2010; 11: 73.
Published online Oct 6, 2010. doi:  10.1186/1471-2296-11-73
PMCID: PMC2967516

The association between demographic factors, user reported experiences and user satisfaction: results from three casualty clinics in Norway

Abstract

Background

User reported experiences and satisfaction are increasingly used as basis for quality indicators in the health sector. However, there is limited understanding of factors associated with user reported experiences and satisfaction with casualty clinics.

Methods

A random sample of 542 patients that had contacted any of three casualty clinics from mid April to mid May 2008 was mailed a questionnaire. A reminder was sent to non-respondents after six weeks. Descriptive statistics for four user reported experiences scales and 20 single items are presented. Multivariate regression analysis was used to assess associations between background variables and user reported experiences, and between user reported experiences and user satisfaction.

Results

225 (41.5%) patients, carers and guardians returned a completed questionnaire. Users reported most positive experiences with the doctor services and the nursing services at the casualty clinics; on a scale from 0 to 100, where 100 is the best possible experience the doctor scale was 82 and the nursing scale 81. Users reported least positive experiences with the organization of the casualty clinic, with a scale score of 65. Self perceived health was associated with user satisfaction, while self perceived health and age were associated with user reported experiences with organization of the clinics. A range of user reported experience domains were related to user satisfaction, after controlling for socio-demographic variables, including experiences with doctor services at the clinics, organization of the clinics, information and self perceived incorrect treatment.

Conclusions

Users report positive experiences with the three casualty clinics, with organization as the aspect with largest improvement potential. The importance of age and health status for users' experiences and satisfaction with casualty clinics was shown, but a range of user reported experiences with the clinics were the most important predictors for user satisfaction.

Background

Patient satisfaction and experiences are an important part in the evaluation of health care quality [1,2]. The purpose of patient satisfaction surveys is frequently related to quality improvement, but reports of general satisfaction have limited value in quality improvement processes [3,4]. Rather than simply asking about patient satisfaction, patient experiences studies identify concrete health care aspects that are important as measures of service quality from the perspective of the patients and hence contribute to their overall satisfaction. This requires an extensive development phase to secure the content validity of the questionnaire and the comparison of the domains of care included in patient experiences questionnaires with more general measures of patient satisfaction as part of validity testing. To inform quality improvement it is common to benchmark results against results for other units or changes over time. This requires case-mix analysis and potential adjustments for case-mix differences. Therefore, quality improvement based on patient satisfaction surveys requires both an assessment of the association between patient reported experiences and patient satisfaction, and between background variables and patient reported experiences of concrete aspects of health care.

A systematic review of the patient satisfaction literature showed that among socio-demographic factors, age and health status are consistently related to patient satisfaction [5]; older respondents and healthier respondents generally have higher satisfaction, while the evidence related to gender, ethnicity and socio-economic status is equivocal. The importance of age and self-perceived health for patient experiences was also found in a recent US study among community health centers, but the authors also identified other important factors like education and race/ethnicity [6]. The generalizability of these findings to the primary care out-of-hours field is uncertain. We found a few studies about the association between demographic factors and patient satisfaction with primary care out-of-hours services [7-10]. One study found that health status and socio-economic status are significantly related to patient satisfaction, but not age and gender [8], while another found that age and ethnicity are related to patient satisfaction [10]. All in all, findings are equivocal and the only pattern that emerges across these studies is that most socio-demographic factors seem to be only weakly related to patient satisfaction.

The systematic review on patient satisfaction also found consistent evidence that the most important health service factor affecting patient satisfaction is the patient-practitioner relationship [5]. Health service factors might be measured by means of patient reporting or evaluation on items and scales for patient experiences, or by other variables concerning organization and type of services. One study about the association between patient experiences and patient satisfaction relating to primary care out-of-hours services found that the doctor's assistant's attitude on the phone, opinion of GP's treatment and waiting time were strongly related to overall satisfaction [7]. Another study confirmed the importance of waiting time for patients' satisfaction [10]. The association between organization and service variables has been assessed in several primary care out-of-hours studies. Two studies found that patients receiving telephone advice were less satisfied than other patients [7,10], but other studies did not find such an association [8,9].

The studies above show that the association between socio-demographic factors and patient satisfaction are equivocal within the primary care out-of-hours field. To some degree the existing evidence contradicts findings from the general patient satisfaction literature. The literature also identifies a need for more research regarding the importance of health service factors for patients' satisfaction including factors related to patient reported experiences. Therefore, our study had two primary aims: i) to assess the association between socio-demographic factors and user reported experiences with primary care out-of-hours services; ii) to assess the association between user reported experiences and user satisfaction, controlling for socio-demographic factors.

The analyses were based on a study of three casualty clinics in Norway in 2008. Norway has a two-level public health care system with a small private sector. Four Regional Health Authorities (RHA) under the Ministry of Health and Care services have responsibility for the hospital sector. The 431 municipalities are responsible for organizing primary health care including out-of-hours services [11]. There were 262 out-of-hours districts in Norway in 2006 covering single or several municipalities through inter-municipality co-operatives. Emergency medical services are usually managed at GP offices during office hours, and by municipality maintained out-of-hours duties by GPs during evenings, nights and weekends [11]. Most of the out-of-hours services are located in a casualty clinic in the host municipality, but some use GPs' surgeries. In this paper we use the term casualty clinic when referring to the physical clinics, while primary care out-of-hours services refers to all services offered by the clinics including office visits, telephone advice and home visits.

Methods

Three casualty clinics were chosen to represent different organization types and size of such clinics. The clinics were recruited through the Watchtower project [11]. The study population included patients that had been in contact with one of the three casualty clinics from 15 April to 13 May 2008. For patients younger than 16 years and patients not able to answer themselves their carers and guardians were asked to fill out the questionnaire. The survey included a random sample of 200 patients from each clinic having telephone contact, consultation visit or home visits by a doctor.

The casualty clinics distributed the questionnaires by mail to the patient's home address. We aimed to send out the reminders at two weeks, but due to practical circumstances relating to clinical administration and postal service delays reminders were sent after six weeks. The questionnaires were returned to the Norwegian Knowledge Centre for the Health Services.

The Norwegian Social Science Data Services approved the survey which was in accordance with the Helsinki Declaration of 1975, as revised in 1983. The leader of each casualty clinic had the opportunity to exclude individuals from the survey based on ethical considerations.

Questionnaire and variables

The questionnaire used in this study has undergone a rigorous process of development and evaluation, including a literature review, qualitative interviews with patients, guardians and carers, and input from an expert group of out-of-hours staff [12,13]. These activities ensured the content validity of the questionnaire. Most of the questionnaire's core items had low levels of missing data, indicating the acceptability and relevance of the questions to patients and guardians. The psychometric tests showed that the questionnaire has satisfactory internal consistency and construct validity [13].

The questionnaire comprises five parts: parts A and E are completed by all patients; part B concerned telephone contact with the clinic; part C concerned consultation at the casualty clinic; part D concerned home visit from a doctor. The questionnaire comprised a total of 42 items including background questions about patients and guardians. The reference period used for survey questions was "during the last visit/encounter".

Based on factor analysis four scales were identified [13]: telephone contact (4 items), nursing services (4 items), doctor services (4 items), and organization (3 items). The questions are described in table table1.1. Item-scale correlations were above 0.5 for all scales. Cronbachs alpha was 0.91 for telephone contact, 0.90 for doctor services at the casualty clinic, 0.93 for nursing services at the casualty clinic, and 0.82 for organizing the casualty clinic, all of which are regarded as satisfactory [13]. Items relating to experiences of care have a five-point scale of Not at all, To a small extent, To some extent, To a large extent and To a very large extent. Items were transformed to scores ranging from 0 to 100 where 100 is the best possible. Items comprising scales are summed and transformed into percentage scores. Patients with missing values on more than half of the items in a scale were excluded.

Table 1
Mean scale and item scores for experiences with the casualty clinics on a 0-100 scale

Analysis

Descriptive statistics for respondents, the four scales and single items are presented (n, mean, SD). Multivariate regression analysis was used to assess the association between background variables and five dependent variables including four user reported experiences scales and one item about global satisfaction. Age and health have been found to be consistently related to patient satisfaction [5] and were included in the regression. We also included other background variables with some empirical evidence of an association with patient satisfaction, including education [6], gender [14,15], extent of urgency [15] and marital status [16]. Length of stay has been found to be associated with inpatient satisfaction [17], and in our study we used the number of times in contact with the out-of-hours clinic the last two years as an equivalent to this in the outpatient setting. We pooled the guardian and patient sample and included a respondent variable in the regression.

Finally, we assessed the association between user reported experiences and global satisfaction by means of a multivariate regression analysis, controlling for background variables. We controlled for the same background variables as in the first regression. The telephone and nursing scales, which were relevant to a fewer number of respondents, were excluded from this analysis due to small sample sizes. SPSS (15.1) was used to analyze the data.

Results

Data collection

Of the 600 patients included in the study, 58 were excluded due to cancellations and factors such as unknown addresses. Of the 542 patients who were sent a questionnaire, 225 (41.5%) responded. Table Table22 shows the respondents characteristics; 148 (68.2%) were patients, 53 (24.4%) were guardians of children under the age of 16, and 16 (7.4%) were carers of patients above 16 years of age. Approximately two thirds of the respondents were women (71.7%), the average age was 46 years, and more than half of the respondents (58.1%) had first contacted the casualty clinic by telephone and then had a consultation at one of the three clinics.

Table 2
Respondent characteristics (n = 225)a

Statistical analysis

Table Table11 shows mean scale and item scores for experiences with the casualty clinics. The users reported most positive experiences with the doctor services and nursing services at the casualty clinic. The former scale had an average score on 81.9, the latter 81.4. The telephone contact scale had a mean score of 80.6, while organization at the casualty clinic had a mean score of 64.7. The items about health care personnel being understandable and self perceived incorrect treatment had the highest scores for the individual items. The two questions about waiting time at the casualty clinic had the lowest scores.

Table Table33 shows the results of multivariate regression analysis with background variables as independent variables and the four patient experiences scales and one item about global satisfaction as dependent variables. Explained variance for the regression models ranged from 5.6% (doctor services) to 11.4% (organization). Only a few background variables had a significant association with the dependent variables. Self perceived health was significantly associated with global satisfaction (p = .03), while age (p = .02) and self perceived health (p = .04) were significantly related to the scale about organization of the casualty clinic.

Table 3
Multivariate linear regression models: association between background variables and the five dependent variables

Table Table44 shows the results of multivariate regression analysis with user satisfaction as the dependent variable and user reported experiences as predictors, controlled for background variables. Explained variance for the regression model was 73.2%. Most patient reported experiences were significantly associated with user satisfaction, while two were close to significant (information on tests and unanswered questions). The most important predictors were doctor services (p < .00) and incorrect treatment (p < .00).

Table 4
Multivariate linear regression model: association between background variables, user reported experiences and global satisfaction (n = 154)

Discussion

In general, age and self perceived health are the most consistent socio-demographic factors related to patient satisfaction [5]. However, within the primary care out-of-hours field the few identified studies revealed inconsistencies [7-10]. One study found the importance of health status and socio-economic status, but not age and gender [8], another found the importance of age and ethnicity [10]. The only clear finding was that most socio-demographic factors seem to be only weakly related to patient satisfaction. Our study identified self-perceived health as a significant predictor for patient satisfaction and one of four experiences scales, and age as significant associated with user experiences with organization at the clinics. This follows the general patient satisfaction literature [5], and these variables have partial empirical support in the primary care out-of-hours literature. However, the small number of studies and lack of consistent findings means that associations should be assessed in future studies. Furthermore, to use this information in case-mix adjustments further work is necessary including assessing the variation of these variables across the unit of analysis [6]. Future studies in the primary care out-of-hours field might use these findings to test hypotheses about associations between health, age and patient satisfaction, but the effects of case-mix will depend on both the strength of association and the variation between the units in question.

The majority of the user reported experiences domains had a significant association with user satisfaction, after controlling for user characteristics. The most important predictor was the experiences the users had with the doctors at the casualty clinic. This concurs with findings from the systematic review of the patient satisfaction literature [5] and shows that the most important user experience domain for user satisfaction is the relationship between the user and the caregiver. The final regression model found that more than 70% of the variation in global satisfaction was explained, which also gives strong support to the validity of the user experiences questions as an indirect measure of user satisfaction. Since ratings of general satisfaction have limited value in quality improvement processes [3,4], the approach of asking about experiences with health care providers is used as a means to identify concrete improvement areas.

The primary aims of this paper were to assess the associations between socio-demographic variables and user satisfaction/experiences, and between user-reported experiences and global satisfaction. Naturally, the sample of three clinics is inadequate to represent the population of clinics in Norway. Also, the number of clinics is too small for multilevel regression, making it difficult to separate individual and clinic level effects. This means that individual level effects might be overestimated, especially if the intraclass correlation coefficient (ICC) is substantial. Future studies in Norway should include a representative sample of casualty clinics, and enough clinics to allow empirical assessment of effects at different levels.

The questionnaire is developed specifically for primary care out-of-hours users and includes items of relevance for measurement of user reported experiences with out-of-hours care. It is based on a literature review, interviews with users of out-of-hours services and consultation with an expert group that was designed to ensure the content validity of the questionnaire [12,13]. Compared to questionnaires such as the Clinician & Group CAHPS survey or the EUROPEP questionnaire, this questionnaire can be used with different types of contact with the out-of-ours services (telephone contact, and/or at the casualty clinic, and/or home visit from the doctor). This questionnaire is also designed for patients that have had one contact with the doctor which is in contrast to questionnaires specific to general practice care that relate to consultations with the patient's usual general practitioner.

The consideration of any differences found through the comparison of user experiences and satisfaction of out-of-hours care with those for general practice care more generally might inform quality improvement initiatives. The EUROPEP questionnaire has been used in large scale surveys of general practice in Norway [18]. However, the differences in the content of items, including items scaling, and scales composition rules out any comparison. Moreover, the study findings relate to three clinics and hence are not representative of all out-of-hours clinics.

Low response rates are a common problem in patient experience surveys in general [5], and also in the primary care out-of-hours field [7,19-22]. The response rate in our study was 41.5% which is similar to other surveys in this field [7]. A low response rate may cause non-response bias if non-respondents differ systematically from respondents [5]. Some studies have found differences on socio-demographic variables between respondents and non-respondents [5,20-22], but a Dutch study on patient satisfaction with out-of-hours primary care found that overall satisfaction did not differ much between respondents and non-respondents [7]. This corresponds to findings in studies conducted by the Norwegian Knowledge Centre for the Health Services that shows small differences between respondents and non-respondents in user experiences surveys [23-26]. Therefore, we expect small effects related to non-response in our study.

Conclusions

Users report positive experiences with the three casualty clinics, with organization as the aspect with largest improvement potential. The importance of age and health status for users' experiences and satisfaction with casualty clinics was shown, but a range of user reported experiences with the clinics were the most important predictors for user satisfaction.

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

All authors were involved in the development of the questionnaire. KD planned the paper together with the other authors, carried out the statistical analysis, conducted the study together with administrative staff, and drafted most of the paper. OAB planned the paper together with the other authors, wrote parts of the paper, and revised the draft critically and approved the final version. AG planned the paper together with the other authors, revised the draft critically and approved the final version. OF planned the paper together with the other authors, revised the draft critically and approved the final version. HHI planned the paper together with the other authors, revised the draft critically and approved the final version. SH planned the study together with the other authors, revised the draft critically and approved the final version. All authors read and approved the final manuscript.

Pre-publication history

The pre-publication history for this paper can be accessed here:

http://www.biomedcentral.com/1471-2296/11/73/prepub

Acknowledgements

The authors would like to thank the patients, guardians and carers who completed the questionnaire. We would also thank the members of the casualty clinics (Arendal, Kvam and Tromso) that recruited patients, and Saga Hogheim for help with data collection. This study was funded by the Norwegian Knowledge Centre for the Health Services.

References

  • Donabedian A. The quality of care. How can it be assessed? JAMA. 1988;260:1743–1748. doi: 10.1001/jama.260.12.1743. [PubMed] [Cross Ref]
  • Kelly E, Hurst J. OECD Health Working Papers 23-2006. Paris: OECD; 2006. Health Care Quality Indicators Project: Conceptual framework paper. full_text.
  • Jenkinson C, Coulter A, Bruster S, Richards N, Chandola T. Patients' experiences and satisfaction with health care: results of a questionnaire study of specific aspects of care. Qual Saf Health Care. 2002;11(4):335–339. doi: 10.1136/qhc.11.4.335. [PMC free article] [PubMed] [Cross Ref]
  • Fitzpatrick R. Capturing what matters to patients when they evaluate their hospital care. Qual Saf Health Care. 2002;11(4):306. doi: 10.1136/qhc.11.4.306. [PMC free article] [PubMed] [Cross Ref]
  • Crow R, Gage H, Hampson S, Hart J, Kimber A, Storey L, Thomas H. The measurement of satisfaction with healthcare: implications for practice from a systematic review of the literature. Health Technol Assess. 2002;6(32):1–244. [PubMed]
  • Johnson ML, Rodriguez HP, Solorio MR. Case-mix adjustment and the comparison of community health center performance on patient experience measures. Health Serv Res. 2010;45:670–90. doi: 10.1111/j.1475-6773.2010.01101.x. [PMC free article] [PubMed] [Cross Ref]
  • Van Uden CJ, Ament AJ, Hobma SO, Zwietering PJ, Crebolder HF. Patient satisfaction with out-of-hours primary care in the Netherlands. BMC Health Serv Res. 2005;5(1):6. doi: 10.1186/1472-6963-5-6. [PMC free article] [PubMed] [Cross Ref]
  • Glynn LG, Byrne M, Newell J, Murphy AW. The effect of health status on patients' satisfaction with out-of-hours care provided by a family doctor co-operative. Fam Pract. 2004;21(6):677–683. doi: 10.1093/fampra/cmh616. [PubMed] [Cross Ref]
  • McKinley RK, Stevenson K, Adams S, Manku-Scott TK. Meeting patient expectations of care: the major determinant of out-of-hours primary medical care? Fam Pract. 2002;19(4):333–338. doi: 10.1093/fampra/19.4.333. [PubMed] [Cross Ref]
  • Salisbury C. Postal survey of patients' satisfaction with a general practice out of hours cooperative. BMJ. 1997;314(7094):1594–1598. [PMC free article] [PubMed]
  • Hansen EH, Hunskaar S. Development, implementation, and pilot study of a sentinel network ("The Watchtowers") for monitoring emergency primary health care activity in Norway. BMC Health Serv Res. 2008;8:62. doi: 10.1186/1472-6963-8-62. [PMC free article] [PubMed] [Cross Ref]
  • Garratt AM, Danielsen K, Hunskaar S. Patient satisfaction questionnaires for primary care out-of-hours services: a systematic review. Br J Gen Pract. 2007;57(542):741–747. [PMC free article] [PubMed]
  • Garratt AM, Danielsen K, Hunskaar S. The Patient Experiences Questionnaire for Out-of-Hours- Care (PEQ-OHC): data quality, reliability and validity. Scand J Prim Health Care. 2010;28:95–101. doi: 10.3109/02813431003768772. [PMC free article] [PubMed] [Cross Ref]
  • Hargraves JL, Wilson IB, Zaslavsky A, James C, Rogers G. Adjusting for patient characteristics when analyzing reports from patients about hospital care. Med Care. 2001;39(6):635–641. doi: 10.1097/00005650-200106000-00011. [PubMed] [Cross Ref]
  • Danielsen K, Garratt AM, Bjertnaes ØA, Pettersen KI. Patient experiences in relation to respondent and health service delivery characteristics: a survey of 26,938 patients attending 62 hospitals throughout Norway. Scand J Public Health. 2007;35:70–7. doi: 10.1080/14034940600858615. [PubMed] [Cross Ref]
  • Groven G, Holmboe O, Helgeland J, Bjertnæs ØA, Iversen H. Brukererfaringer med poliklinikker for voksne i psykisk helsevern: nasjonale resultater i 2007 og utvikling fra 2004. Oslo: Norwegian Knowledge Centre for the Health Services; 2008. Report No.: PasOpp-rapport nr 5. 2008.
  • Rosenheck R, Wilson NJ, Meterko M. Influence of patient and hospital factors on consumer satisfaction with inpatient mental health treatment. Psychiatr Serv. 1997;48:1553–61. [PubMed]
  • Grol R, Wensing M. Patients evaluate general/family practice. The EUROPEP instrument. Centre for Quality of Care Research, Nijmegen. 2000. pp. 9–70.
  • McKinley RK, Manku-Scott T, Hastings AM, French DP, Baker R. Reliability and validity of a new measure of patient satisfaction with out of hours primary medical care in the United Kingdom: development of a patient questionnaire. BMJ. 1997;314(7075):193–198. [PMC free article] [PubMed]
  • Moll van Charante E, Giesen P, Mokkink H, Oort F, Grol R, Klazinga N, Bindels P. Patient satisfaction with large-scale out-of-hours primary health care in The Netherlands: development of a postal questionnaire. Fam Pract. 2006;23(4):437–443. doi: 10.1093/fampra/cml017. [PubMed] [Cross Ref]
  • Salisbury C, Burgess A, Lattimer V, Heaney D, Walker J, Turnbull J, Smith H. Developing a standard short questionnaire for the assessment of patient satisfaction with out-of-hours primary care. Fam Pract. 2005;22(5):560–569. doi: 10.1093/fampra/cmi050. [PubMed] [Cross Ref]
  • Campbell JL, Dickens A, Richards SH, Pound P, Greco M, Bower P. Capturing users' experiences of UK out-of-hours primary medical care: piloting and psychometric properties of the Out-of-hours Patient Questionnaire. Qual Saf Health Care. 2007;16(6):462–468. doi: 10.1136/qshc.2006.020172. [PMC free article] [PubMed] [Cross Ref]
  • Guldvog B, Hofoss D, Pettersen KI, Ebbesen J, Ronning OM. PS-RESKVA -(Patient Satisfaction, Results and Quality -- patient satisfaction in hospitals) Tidsskr Nor Laegeforen. 1998;118(3):386–391. [PubMed]
  • Bjertnaes OA, Garratt A, Iversen H, Ruud T. The association between GP and patient ratings of quality of care at outpatient clinics. Fam Pract. 2009;26:384–90. doi: 10.1093/fampra/cmp043. [PMC free article] [PubMed] [Cross Ref]
  • Bjertnaes OA, Garratt A, Helgeland J, Holmboe O, Dahle KA, Hanssen-Bauer K, Røttingen JA. Parent assessment of outpatient child and adolescent mental health services. Tidsskr Nor Laegeforen. 2008;128(9):1041–1045. Norwegian. [PubMed]
  • Bjertnaes O, Garratt A, Botten G. Nonresponse bias and cost-effectiveness in a Norwegian survey of family physicians. Eval Health Prof. 2008;31:65–80. doi: 10.1177/0163278707311874. [PubMed] [Cross Ref]

Articles from BMC Family Practice are provided here courtesy of BioMed Central

Formats:

Related citations in PubMed

See reviews...See all...

Cited by other articles in PMC

See all...

Links

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...