NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Henriksen K, Battles JB, Keyes MA, et al., editors. Advances in Patient Safety: New Directions and Alternative Approaches (Vol. 2: Culture and Redesign). Rockville (MD): Agency for Healthcare Research and Quality (US); 2008 Aug.

Cover of Advances in Patient Safety: New Directions and Alternative Approaches (Vol. 2: Culture and Redesign)

Advances in Patient Safety: New Directions and Alternative Approaches (Vol. 2: Culture and Redesign).

Show details

The PeaceHealth Ambulatory Medication Safety Culture Survey

, MD and , PhD.

Author Information

, MD and , PhD.*

Center for Medical Education & Research, PeaceHealth Oregon Region (Dr. Stock); Director of Survey, Research, and Development, PeaceHealth System Office (Dr. Mahoney)
*Address correspondence to: Ronald Stock, MD, Medical Director, Center for Medical Education & Research, PeaceHealth Oregon Region, 722 East 11th P.O. Box 10482, Eugene, OR 97440; e-mail: gro.htlaehecaep@kcotsr.

Objective: The objective of this project was to construct a measure of medication safety culture in ambulatory settings. Methods: A 16-item survey was created to measure the degree to which a culture of medication safety exists within ambulatory clinics. The instrument was tested with two administrations separated by 12 months in three ambulatory clinics and evaluated with Mplus factor analysis, internal consistency reliability, and discrimination ability. Results: Of 105 staff, 62 (60 percent) returned surveys in the first administration and 80 (77 percent) in the second. The measure had good internal consistency reliability, with a Cronbach alpha of 0.94 and 0.90 for the two administrations and 0.90 to 0.96 across the three clinics. The measure demonstrated good sensitivity and discrimination between clinics. Five subdomains of medication safety culture were identified: (1) leadership, (2) learning culture, (3) quality improvement, (4) physician responsibility, and (5) safety as a priority. Conclusion: The measure is psychometrically strong and capable of assisting in the improvement of medication management safety.

Introduction

One of the biggest challenges in health care is providing safe, effective care, and one of the most significant areas of opportunity for improvement is medication safety. It is well known that many adverse drug events (ADEs) occur within the hospital setting. However, little is known about the incidence of ADEs in the ambulatory setting. This knowledge gap exists despite the fact that medication prescribing is the most frequently used therapeutic intervention, with nearly two-thirds of office visits concluding with a prescription for medication.1 The risks for medication errors and subsequent ADEs in the outpatient setting can be a result of (or a combination of) physician/provider-related, health system/practice process-related, or patient-related factors. To best understand these factors, it is important to examine the processes involved in each of those three domains. In the health system/practice domain, a key measurable component is “safety culture,” which includes the management behaviors, safety system processes, and staff perceptions of safety that exist within the health care environment.2

The Institute of Medicine (IOM) has recommended that health care organizations improve patient safety culture.3 In their report, Preventing Medication Errors,4 the IOM notes that developing an organizational culture of medication safety in the health care setting is a key component to improving medication safety outcomes and preventing ADEs. A number of safety culture (or climate) surveys have been developed to assess the degree to which a safety culture exists within an organization. Most have measured safety attitudes and perceptions at the individual level based on five common dimensions of the patient safety climate: (1) leadership, (2) policies and procedures, (3) staffing, (4) communication, and (5) reporting.5

Although many of these surveys were developed for general use, most have been utilized primarily within the hospital setting.6, 7, 8, 9, 10, 11, 12, 13, 14 Despite the known importance of patient safety in outpatient clinics, few surveys have been developed specifically for the ambulatory setting. Recently, the safety attitudes questionnaire was adapted to assess provider safety attitudes in the ambulatory setting.15 Others have attempted to better understand the theoretical framework of patient safety in primary care.16, 17 However, we are unaware of any survey that has been developed specifically to assess medication safety culture in the ambulatory clinic or hospital setting.

In order to improve medication safety in the ambulatory clinic setting, a better understanding of the safety culture or climate specifically related to medication management is needed before interventions to improve safety can be developed. Only through assessment and feedback to clinical work groups or office teams can safety culture and ultimately patient safety be transformed.

This report describes the development of an ambulatory medication safety culture survey, its psychometric properties, and findings from testing in three adult medicine primary care practices.

Methods

Study Design

An ambulatory clinic-focused survey measuring the degree to which a culture of medication safety is present in a clinical practice was developed using components of previously published patient safety culture surveys.6, 7, 8, 9, 10, 11, 12 Initially, an 18-item survey was constructed for testing. Baseline data were collected for three clinics (collected in June 2004 for two clinics; August 2004 for a third). All data were collected prior to an intervention to improve medication management in the participating clinics. A followup survey was sent out in June 2005 for all three clinics. Psychometric analysis of the instrument was performed using 142 completed surveys.

Participant Clinics

Three free-standing ambulatory primary care clinics were chosen to participate in the study based on their interest in improving medication safety and experience in quality improvement projects. The following clinic sites participated in the study: the Senior Health and Wellness Center (SHWC), Eugene, OR, with four geriatric providers and two nurse practitioners; the Center for Senior Health (CSH), Bellingham, WA, with seven adult medicine and geriatrician providers; and Health Associates Peace Harbor (HAPH), Florence, OR, with 13 adult care providers.

This survey was completed prior to an intervention to improve the medication reconciliation process in the clinics and introduction of a patient-centered electronic medication list. Total number of clinic staff completing the survey in the first administration was 62 (overall response rate, 60 percent; Clinic A: N = 20, response rate, 80 percent; Clinic B: N = 16, response rate, 55 percent; Clinic C: N = 26, response rate, 51 percent). In a second administration 12 months after the intervention, the staff completed a total of 80 surveys (overall response rate, 77 percent; Clinic A: N = 20, response rate, 80 percent; Clinic B: N = 28, response rate, 96 percent; Clinic C: N = 32, response rate, 63 percent).

Survey Instrument Construction

Initially, eighteen questions specifically addressing medication safety were constructed based on existing generic safety culture surveys. Two of the 18 items were not retained for the final analysis. It was found that inclusion of these items lowered internal consistency reliability. These two items were, “I often worry about whether I have all of the information I need to make sure that a medication is prescribed safely for a patient” and “The health care providers in this clinic frequently disregard rules or guidelines for medication safety.” Table 1 lists the remaining 16 items used in the final analysis.

Table 1. Culture of medication safety items.

Table 1

Culture of medication safety items.

Psychometric Analysis

Psychometric analysis was performed on data collected from participants who completed the 18-item survey at both time periods. Analysis consisted of internal consistency reliability (Cronbach’s alpha) to determine whether the 16 analyzed items could be used to construct a single culture-of-medication safety score. Exploratory factor analysis in Mplus18 with the items treated as ordered categorical variables was conducted to understand the different domains of medication culture within the 16 items. In the Mplus analysis, maximum likelihood estimation of missing values was employed. Finally, ability to discriminate differences among clinics and change over time was assessed by a general linear model analysis.

Results

Internal Consistency Reliability

The final 16-item instrument had good internal consistency reliability and alpha did not increase by deleting any item. For all three clinics combined, Cronbach alpha was 0.94 at the first administration and 0.90 at the second administration. The internal consistency reliability was maintained in all clinic sites (Clinic A = 0.96; Clinic B = 0.90; Clinic C = 0.94). The ceiling and floor effects of the 16 items were small (floor effect = 0.7 percent; ceiling effect = 2.1 percent).

Factor Structure

An Mplus factor analysis revealed that a one-factor solution was not satisfactory with a root mean square residual (RMSR) of 0.08. An unsatisfactory one-factor solution was due to item 1 (“The culture of this clinic makes it easy to learn from the medication mistakes of others”) having a loading of 0.03, while all other items loaded 0.5 or above on the single factor.

Exploratory factor analysis in Mplus found that the best solution was a five-factor solution (RMSR = 0.03) as shown in Table 2, where 7 of the 16 items can be seen to load high on the leadership factor; another four items load highly on the quality improvement factor. Because these two factors dominate the 16 items, the fact that item 1 loads 0.004 on leadership and −0.051 on quality improvement and is the only question addressing learning culture explains why item 1 fails in a one-factor solution. Therefore, although the 16 items all measure culture of medication safety, they address five separate domains of that construct: (1) leadership, (2) learning culture, (3) quality improvement, (4) physician responsibility, and (5) safety as a priority. An overall score is calculated by taking the mean response across the 16 items where the disagree-agree response options are scored “disagree strongly = 1” to “agree strongly = 4.” The higher the score, the more a culture of medication safety is present.

Table 2. Varimax rotated factor loadings of the five-factor solution.

Table 2

Varimax rotated factor loadings of the five-factor solution.

Discriminant Validity and Sensitivity

The discriminant validity of the 16-item measure was assessed by evaluating the ability of the measure to distinguish between clinics qualitatively known to differ on their likelihood of (a) having a culture of medication safety in place and (b) the degree to which such a culture could be increased. This qualitative classification of the clinics comes from our observation and experience working with the three clinics over a 3-year period, thereby gaining some insight into the cultural dynamics of the clinical practices. Based on this knowledge, we hypothesized that clinics A and C would have more of a culture of medication safety in place at the 2004 baseline. We further hypothesized that clinics A and B would have greater capacity for improvement over time than clinic C.

To evaluate differences among the three clinics and their change over time in the culture of medication safety, a univariate general linear model analysis was conducted on culture of medication safety scores. Clinic site and year (2004, 2005) were fixed factors with no covariates.

There was a significant between-subjects effect for clinic (F = 9.65, P < 0.0001) and year (F = 17.5, P < 0.0001) and a significant clinic x year interaction (F = 14.28, P < 0.0001). As shown in Figure 1, the nature of the interaction was that Clinic A and Clinic B significantly improved in culture of medication safety from 2004 to 2005, while there was no significant change in Clinic C (95 percent CI). At baseline in 2004, there were no significant differences among the three clinics. In 2005, both Clinics A and B had a significantly higher culture of medication safety score than Clinic C (95 percent CI). The measure does appear to discriminate among the clinics in the degree of medication safety culture present and is sensitive to detect change over time in culture of medication safety after an intervention.

Figure 1. Mean culture of medication safety score by year by clinic.

Figure 1

Mean culture of medication safety score by year by clinic.

Discussion

Medication management is known to be a critical component of patient safety across the care continuum, and a focus on the ambulatory setting provides an emerging opportunity to improve medication safety. Creating a culture of medication safety in ambulatory clinics that patients consider their “medical home” will be important to ensure that safe, reliable health care is adequately managed across that continuum.

The PeaceHealth Ambulatory Medication Safety Culture Survey, although in its early stages of development, appears to be a valid and valuable tool to assist ambulatory clinic staff in their pursuit of safer medication management. Although developing useful instruments to measure medication safety culture is important, the tool development process and the use of these tools will also assist in understanding the complex nature of the safety culture within an organization. Certainly, a simple unidimensional survey tool would have some benefits. However, it is clear from the evaluation of this instrument, as supported by previous work, that the culture of safety is more complex.

In this study, the best Mplus factor analysis solution revealed five subdimensions: (1) leadership, (2) learning culture, (3) quality improvement, (4) physician responsibility, and (5) safety as a priority. This instrument contains seven items that appear to measure attributes of leadership within the clinical setting. These items suggest that leadership involves not only the actions of leaders, but also the processes/outcomes, or lack thereof, of leadership attributes.

The most heavily loaded item in the leadership dimension was, “If a member of my immediate family were to be a patient in this clinic (not my patient), I would have no concern at all about possible medication errors.” This item supports the staff perception that a criterion for a culture of medication safety is reflected by the degree with which it would be safe for one of their family members. Although this item does not specifically mention leadership, it does suggest the importance of leadership in creating an environment that staff feel is safe for those that are close to them.

The learning culture subdimension contains only one question, albeit an important attribute of a safety culture. More exploration through item development is needed to better understand what it means to be a learning organization in the context of clinic medication safety. The third subdimension of “quality improvement” assesses the environment in which the processes are defined to ensure medication safety and whether staff perceptions of continuous improvement are an attribute of that dimension.

The centrality of the provider role in patient care and safety is suggested as a theme of the physician responsibility subdimension. A core activity of patient safety is the “physician responsibility” for creating a safe environment through the positive focus on processes rather than blame. This is reinforced by moderately high loading of item 12 (mistakes not approached as personal blame) on the leadership subdimension.

Finally, the subdimension of “safety as a priority” is necessary in order for conversations about defined safety protocols to occur. If safety is not perceived to be a high priority, conversations and actions that lead to safer care will not occur. It is clear that there is still much to learn about the components of a culture of medication safety in the ambulatory environment. Better understanding of subdimensions and attributes within a culture of medication safety will ensure more accurate measurement and thus improved feedback to clinical staff endeavoring to improve the safety of care provided to patients.

This survey tool appears to have the ability to discriminate the degree of medication safety culture differences among clinics, and it has the sensitivity to detect change in the culture of medication safety over time after an intervention. Two of the three clinic sites improved their culture of medication safety after an intervention to improve medication safety was implemented. As a component of the intervention, clinic staff discussions about clinic medication safety were part of the survey feedback and were believed to have been an effective intervention. Also, staff and provider involvement in the process improvement of medication management within the clinic most likely affected the safety culture results.

Based on qualitative observations, it appeared that team members, particularly providers, who were closer to changes in the medication management process at the point of service and participated actively in the study, produced better staff engagement. This explains why two of the clinics improved their culture scores while the third did not. At the site where medication safety culture did not change over time, direct provider and patient participation in the intervention was not as active. Thus, engagement of staff and providers was perceived to be lower. In this third clinic, the accuracy of medication lists improved through the process redesign, but the culture of medication safety did not. This raises the issue of whether improving care processes leads to improved safety culture. This clinic will require continued observation to evaluate whether the medication reconciliation workflow processes will remain reliable and sustainable, as we believe that the clinic culture will influence the sustainability of work processes.

While the current instrument appears to be of value, the measure could be enhanced with further development. The factor analysis clearly indicates that more items could be constructed in the subdimensions of “physician responsibility,” “safety as a priority,” and “learning culture.” Although the subdimensions of “leadership” and “quality improvement” have sufficient internal consistency reliability to be used for creation of subdimension scores (alpha = 0.85 and 0.84, respectively), the measure would be far more useful if it provided subdimension scores for all five subdimensions. Testing in ambulatory clinic settings other than adult primary care and larger samples of providers and staff are needed for further evaluation of the generalizability of this instrument. Also, assessment of differences of responses among clinic staff disciplines would further enhance the usability of this tool for medication management quality improvement.

Conclusion

The PeaceHealth Ambulatory Medication Safety Culture Survey has strong psychometric properties. The survey was found to be an effective tool for providing feedback to clinic staff regarding the perception of medication safety in the work environment. Based on early testing, we believe the utility of this survey is strengthened by its strong psychometric properties and its development specific to the care environment and purpose of medication safety. Further development of the instrument is needed to better define survey subdimensions. Finally, caution is needed in inferring that improving medication safety culture will lead to better patient outcomes and, alternatively, that an improvement in medication safety outcomes translates into an improved culture of safety.

Acknowledgments

The Agency for Healthcare Research and Quality (AHRQ) provided funds for this study under the Safe Practices Implementation Challenge Grants Program. PeaceHealth provided matching fund contributions to the study (Grant Award Number: UC1HS14315-02).

References

1.
Barnsteiner JH. Medication reconciliation: Transfer of medication information across settings – keeping it free from error. Am J Nursing. 2005;105( 3 Suppl):31–36. [PubMed: 15802996]
2.
Guldenmund F. The nature of safety culture: A review of theory and research. Saf Sci. 2000;34:215–257.
3.
Kohn LT, Corrigan JM, Donaldson MS. Institute of Medicine. Washington, DC: National Academies Press; 1999. To err is human: Building a safer health system.
4.
Aspden P, Wolcott J, Bootman JL, et al., editors. Institute of Medicine. Washington, DC: National Academies Press; 2006. Preventing medication errors.
5.
Colla JB, Bracken AC, Kinney LM, et al. Measuring patient safety climate: A review of surveys. Qual Saf Health Care. 2005;14:364–366. [PMC free article: PMC1744072] [PubMed: 16195571]
6.
Nieva VF, Sorra J. Safety culture assessment: A tool for improving patient safety in healthcare organizations. Qual Saf Health Care. 2003;12(Suppl II):ii17–ii23. [PMC free article: PMC1765782] [PubMed: 14645891]
7.
Sexton JB, Thomas EJ. The Safety Climate Survey: Psychometric and benchmarking properties. Austin, TX: The University of Texas Center of Excellence for Patient Safety Research and Practice; Rockville, MD: Agency for Healthcare Research and Quality; 2003.
8.
Sexton JB, Thomas EJ, Helmreich RL, et al. Frontline assessments of healthcare culture: Safety Attitudes Questionnaire norms and psychometric properties. Austin, TX: The University of Texas Center of Excellence for Patient Safety Research and Practice; Rockville, MD: Agency for Healthcare Research and Quality; 2004.
9.
Weingart SN, Farbstein K, Davis RB, et al. Using a multihospital survey to examine the safety culture. Jt Comm J Qual Saf. 2004;30:125–132. [PubMed: 15032069]
10.
Singer SJ, Gaba DM, Geppert JJ, et al. The culture of safety: Results from an organization-wide survey in 15 California hospitals. Qual Saf Health Care. 2003;12:112–118. [PMC free article: PMC1743680] [PubMed: 12679507]
11.
Smetzer JL, Vaida AJ, Cohen MR, et al. Findings from the ISMP Medication Safety Self-Assessment for Hospitals. Jt Comm J Qual Saf. 2003;29:586–597. [PubMed: 14619351]
12.
Sorra J, Nieva VF. Psychometric analysis of the Hospital Survey on Patient Safety. Rockville, MD: Westat, under contract to BearingPoint; Rockville, MD: Agency for Healthcare Research and Quality; 2003. (Contract 290-96-0004)
13.
Sexton JB, Helmreich RL, Neilands TB, et al. The safety attitudes questionnaire: Psychometric properties, benchmarking data, and emerging research. BMC Health Serv Res. 2006;6:44. [PMC free article: PMC1481614] [PubMed: 16584553]
14.
Singer S, Meterko M, Baker L, et al. Workforce perceptions of hospital safety culture: Development and validation of the patient safety climate in healthcare organizations survey. Health Serv Res. 2007;42:1999–2021. [PMC free article: PMC2254575] [PubMed: 17850530]
15.
Modak I, Sexton JB, Lux TR, et al. Measuring safety culture in the ambulatory setting: The safety attitudes questionnaire-ambulatory version. J Gen Intern Med. 2007;22:1–5. [PMC free article: PMC2227589] [PubMed: 17351834]
16.
Hutchinson A, Cooper KL, Dean JE, et al. Use of a safety climate questionnaire in UK health care: Factor structure, reliability, and usability. Qual Saf Health Care. 2006;15:347–353. [PMC free article: PMC2565820] [PubMed: 17074872]
17.
Kirk S, Parker D, Claridge T, et al. Patient safety culture in primary care: Developing a theoretical framework for practical use. Qual Saf Health Care. 2007;16:313–320. [PMC free article: PMC2464946] [PubMed: 17693682]
18.
Muthen LK, Muthen BO. Mplus: The comprehensive modeling program for applied researchers guide. Los Angeles: Muthen & Muthen; 2001.
PubReader format: click here to try

Views

  • PubReader
  • Print View
  • Cite this Page
  • PDF version of this page (381K)

Other titles in this collection

Related information

  • PMC
    PubMed Central citations
  • PubMed
    Links to pubmed

Related citations in PubMed

See reviews...See all...

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...