NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

National Collaborating Centre for Primary Care (UK). Medicines Adherence: Involving Patients in Decisions About Prescribed Medicines and Supporting Adherence [Internet]. London: Royal College of General Practitioners (UK); 2009 Jan. (NICE Clinical Guidelines, No. 76.)

Cover of Medicines Adherence

Medicines Adherence: Involving Patients in Decisions About Prescribed Medicines and Supporting Adherence [Internet].

Show details

7Assessment of adherence

7.2. Introduction

Many patients take medicines over long periods of time and discussions about these medicines need to consider the patients experience of taking the medicine. This includes an assessment or discussion about whether or not the patient is taking the medicine and if they are doing this exactly as prescribed or in some other way.

A number of ways of assessing adherence have been developed. These can generally be described as direct methods or indirect methods. Direct methods are examinations of blood, urine or other bodily fluids for the presence of the medicine or a metabolite. Indirect methods do not measure the presence of the medicine but use methods such as self report from patients, pill counts, prescription reordering, pharmacy refill records, electronic medicine monitoring and therapeutic effect to form an assessment of adherence. In the context of routine clinical practice and of involving patients in decisions about medicines the GDG considered that indirect methods were the most commonly used. Self-report is the most available method for reporting adherence in a clinical context. The GDG wished to consider the advantages and disadvantages of self report in routine clinical practice to recommend how it should be used by practitioners. We conducted an evidence review to explore specifically the advantages and disadvantages of self-report for assessing adherence. Other types of measures of adherence were not explored.

7.3. What are the advantages and disadvantages of self-report in assessing patient’s adherence?

Related referencesEvidence statements (summary of evidence)
Advantages
Hawkshead (2007) 150; Gagne (2005) 151; Paterson (2002) 152; Miller (2000) 153; Turner (2002) 154; Farmer (1999) 155; Bender (1997) 156; LaFleur (2004) 157; Rand (1994) 158Self-reporting is the most simple and inexpensive method of measuring adherence.
Miller (2000) 153; Farmer(1999) 155; Paterson (2002) 152; Bender (1997) 156; Rand (1994) 158Self-reporting is quick and easy to administer, avoiding the use of sophisticated methodology or equipment.
Hawkshead (2007) 150; Bender (1997) 156Self-reporting methods which are validated can feasibly be used in clinical settings.
Hawkshead (2007) 150; Paterson (2002) 152; Farmer (1999) 155; Hecht (1998) 159; Bennett Johnson (1992) 160; George (2007) 161Self-reporting can identify those who are nonadherent. It is most likely those reporting nonadherence are being truthful.
Hawkshead (2007) 150; Miller (2000) 153; Rand (1994) 158; Bennett Johnson (1992) 160Self-reporting can gather social, situational and behavioural factors including revealing patterns of medicine use and what leads to non-compliance.
Disadvantages
George (2007) 161; Hawkshead (2007) 150; LaFleur (2004) 157; Turner (2002) 154; Miller (2000) 153 Hecht (1998) 159; Bender (1997) 156Self-reporting has the problem of over- estimating adherence.
Hawkshead (2007) 150; Gagne(2005) 151; LaFleur (2004) 157; Turner (2002) 154; Farmer(1999) 155; Bennett Johnson (1992) 160Inaccurate self-reporting can be caused by recall bias, social desirability bias and errors in self-observation.
Paterson (2002) 152; Hecht (1998) 159; Bennett Johnson (1992) 160The timeframe of the adherence recollection can affect the accuracy of the recall. Specifying the time period can help.
Hawkshead (2007) 150; Farmer (1999) 155; Hecht (1998) 159Wording of questions, the way a question is asked and the skills of the interviewer can either facilitate or be detrimental to gaining accurate responses.
Turner (2002) 154; Bennett Johnson (1992) 160Being non-judgmental, giving a preamble before adherence questions, and asking about specific behaviours can help validity.

7.3.1. Evidence to recommendations

The GDG considered that self-report is the most widely used method of assessing adherence and that although direct measures of adherence are relevant in some situations they were more interested in making recommendations for routine clinical practice. Indirect methods such as therapeutic effects and prescription ordering and refills are methods which should alert prescribers and dispensers to problems of adherence. In these situations and as part of medicine reviews health care professionals need to be able to discuss medicine-taking with patients. The GDG made recommendations on how professionals should assess adherence using the review of advantages and disadvantages of self-report.

7.3.2. Methods of the evidence review

This paper includes a narrative summary of the included evidence, structured according to the category of the intervention, following the agreed reviewing protocol:

Types of studies – We included literature reviews and systematic reviews only.

Types of participants - People prescribed medicine for a medical condition.

Duration of studies – No time limit was specified.

Types of interventions - Any interventions intended to change adherence to prescribed medicine which reviews studies which focus on self-report advantages and disadvantages.

Types of outcome measures - No outcome measures specified.

7.3.3. Evidence review

The searches mainly returned literature reviews, rather than systematic reviews, therefore details (of the various studies mentioned in these reviews) were not always given and some only mentioned the studies briefly.

Garber (2004) 162 produced a systematic review on the concordance of self-report with other measures of medicine adherence. They searched a number of databases and identified 2757 articles. The inclusion criteria included studies where at least 2 adherence measures were used, one of which was a self-report measure, the other a non self-report measure. The self-report measures included questionnaires, diaries or interviews and were categorised under these. They found 86 unique comparisons, mostly interviews (57%), questionnaires (27%) and diaries (17%). The non-self-report measures were electronic measures (35%), pill count or canister weight (26%), a plasma drug concentration (20%) a claims-based measure (13%) and a clinical opinion (6%). 43% of the pairings of self-report and nonself-report measures were highly concordant.

Concordance levels were categorised by the following: Kappa results (for categorical variables) over 0.6 were high, 0.6 to 0.4 were moderate and below 0.4 were low. Pearson correlation co-efficient (for continuous variables) over 0.8 were high, 0.8 to 0.4 were moderate and below 0.4 were low. When sensitivity and specificity of the measure was given the measure was as a likelihood ratio (LR). A positive LR greater than 10 was categorised as high, a LR+ of 3 to 10 was moderate and LR+ below 3 was low. If there was no statistical analysis given the authors used an algorithm to categorise.

In the majority (45/59) of those which were not highly concordant, the self-report measure showed higher adherence compared to the nonself-report measure, but this varied widely depending on type of self-report measure. 31% of the interviews were highly concordant with nonself-report measures. Diaries (71%) and questionnaires (55%) were much more likely to be highly concordant with non self-report measures. The difference in concordance by the type of self-report measure was significant (chi-square=8.47, p=0.01). It also depended on the non-self-report measures. Self-reporting had higher concordance with other types of non self-report measures (58%) than electronic measures (17%) (chi-square 14.3, p<0.01). Interviews showed the least concordance with electronic measures, where none of the 15 comparisons were highly concordant. The authors noted that this was a comparison between measures which could not fully evaluate the accuracy of any of the measures.

The authors concluded that questionnaire and diary methods were preferable over interviewing for measuring adherence. They note that many of the studies did not explicitly compare adherence measures statistically and those that did used simplistic analyses. They also note that the categorisation of concordance was based on arbitrary cut-off points, so different cut-off points could change the levels of concordance between methods.

In summary, questionnaires and diaries were more concordant with other measures.

George (2007) 161 conducted a literature review to assess adherence of COPD patients with disease management programs. They searched OVID and International Pharmaceutical Abstracts. They did not report the inclusion/exclusion criteria or how many studies were retrieved. The adherence measures that were included in the review were self report, inhaler weights, electronic monitoring, inhalation technique assessment, medicine/pill count, pharmacy refill data/claims data and biological assays.

They found that self-reporting of missed doses (by questionnaire) underestimated nonadherence compared to more objective measures e.g. capsule count (Dompeling, 1992), inhaler weights (Rand, 1995) and electronic monitoring (Rand, 1992; Braunstein, 1996; Simmons, 2000). Self-report was shown to have moderate reliability (25% to 67%) compared to objective measures such as canister weight (Rand, 1995) and electronic monitoring (Gong, 1988; Nides, 1993; Bosley, 1995).

Self-reporting of nonadherence of medicine for COPD has shown satisfactory reliability, when compared to objective measures (Dolce, 1991; Nides, 1993; Rand, 1995). Self-report is commonly criticised for overestimating adherence and poor reliability yet those who report nonadherence are likely to be telling the truth (Haynes, 1980; Inui, 1981; Choo, 1999; Erickson, 2001).

The author’s concluded that even though electronic monitoring is regarded as the gold standard it is more suited to a clinical trial setting. Self-reporting is the cheapest, simplest and easiest method to assess adherence. Self-report can identify the reasons for nonadherence and therefore the issues can be addressed.

Hawkshead (2007) 150 presented a narrative review of the advantages and limitations of methods for measuring adherence in hypertensive patients. No mention is given to how they searched for these studies or decided to include/exclude. The types of adherence measures were self-report, electronic monitoring, pill counts, pharmacy refill rates, bioassays/biomarkers and direct observation.

They state that self reporting is the simplest method for assessing medicine adherence and can include patient diaries, interviews during office visits and adherence-specific questionnaires. ‘Several multi-item questionnaires have been developed and tested in outpatient settings with the explicit aim of ascertaining valid and reliable estimates of adherence to antihypertensive medicines’, of which many have reported high measures of validity and reliability (Morisky, 1986; Kim, 2000; Shea, 1992; Krousel-Wood, 2005; Hyre, 2007). There are three previously validated self-reported medicine adherence instruments – the Medication Adherence Survey (MAS), the Brief Medication Questionnaire (BMQ) and the Medical Outcomes Study (MOS). Cook (2005) compared the level of agreement between these and pharmacy refill rates and found correlations between of 0.23, 0.26 and 0.21 between the refill rates and the MAS, MOS and BMQ respectively.

Validated self-report measures can feasibly be used in clinical settings and help to identify those who are nonadherent, and intervene to increase this (Harmon, 2006). The advantages stated are that self-report is simple and economical; it can also gather social, situational, and behavioural factors which can impact on adherence. The disadvantages are the possibility that there could be recall bias, over-estimation of compliance and responses which are socially acceptable. Validity can also depend on the skills of the interviewer as well as the question construction and timeframe (Farmer, 1999 and Wang, 2004). It is suggested that self-report could be combined with objective information, e.g. prescription-fill data, to improve adherence measurement.

The authors’ concluded that selecting the type of measure for clinical practice depends on the intended use of the information, the resources available, patient acceptance and the convenience of the method. A combination of methods may be best to give an accurate assessment of adherence and should be tailored to individual needs.

In summary, some self-reporting questionnaires have been validated and can be simple and feasible to use in clinical settings and identify non-adherers. However they can have biases and overestimate adherence.

Gagne (2005) 151 reported on how to improve self-report measures for nonadherence to HIV medicines, with particular attention to techniques that can be applied with questionnaires administered in clinical practice. Questionnaires are inexpensive and convenient and can be conducted in clinical and research settings, but can vary in terms of accuracy. According to many authors, forgetfulness (Brooks, 1994; Hayes & DiMatteo, 1987; Holzemer, 1999; Rand, 2000; Svarstad, 1999) and social desirability (Felkey, 1995; Gordis, 1969; Gray, 1998; Rand, 2000; Svarstad, 1999) are main factors leading to inaccurate self-reporting of nonadherence. Social desirable answers can depends on how much the patient perceives the desirability of the behaviour to be. Those behaviours perceived as undesirable are under-reported and behaviours perceived as desirable can be over-reported (Cannell 1979; Fowler, 1995). There are techniques suggested for minimising forgetfulness and social desirability (Cannell, 1979; Fowler, 1995; Sudman & Bradburn, 1974; Sudman & Bradburn, 1982) although methods to reduce these are not well-documented, are often derived from clinical practice than controlled experimental studies and their reported effectiveness is inconsistent.

Suggestions were made to reduce socially desirable answers:

  • Assuring confidentiality and that information will not be available to HCPs (Eldred, 1998; Gordillo, 1999).
  • Explaining that there are no right or wrong answers (Des Jarlais, 1999; Chesney, 1990).
  • How the question is asked (Ickovics, in Eldred, 1998; Chesney, 1999; Svarstad, 1999).
  • Wording the question to increase the likelihood of gaining certain desired answers, such as nonadherence (loading the question) (Sudman, 1982; Bradburn, 1982; Allaire, 1988).
  • Open-ended questions can avoid the pitfalls of response categories (Schwarz, 1985; Sudman, 1982).

Open-ended questions have been used in studies of HIV (e.g. Chesney, 1990) and for measuring adherence/nonadherence (e.g. Svarstad, 1999). Open-ended answers have shown to be less affected by social desirability than close-ended answers (Sudman, 1974). Sudman (1974) also found that open-ended questions were less affected by forgetfulness and recall errors.

Recall can be aided by:

  • Item wording, using familiar words and words that have only one meaning and one idea (Sudman, 1982);
  • Words should not have blame implications (Averitt, in Eldred, 1998).
  • Aided-recall techniques such as memory cues may be useful (Sudman, 1982).
  • Specifying a reference time period, especially a recent and short time frame can aid forgetfulness (Brooks, 1994; Chesney, 1999; Holzemer, 1999; Sudman, 1982).

However there is the problem of the time period being too short and not accurately representing the adherence level, as adherence varies over time (Chesney, 1997b; Gray, 1998; Kastrissios, 1998). This could be solved by using a short period of time and administering the questionnaire a number of times over the period. However, this could lead to less motivation and could be costly. Shorter periods of reference could be used when administering the questionnaire only once. According to episodic and semantic memory it may be best to ask more precise information about the past few days and less specific information from a longer time period.

The author concluded that most of the HIV literature used multiple measures of adherence. Adherence to HIV measures could be enhanced by improving self-report measures of nonadherence. Questionnaire designs may have surprisingly beneficial results.

In summary, self-reporting by questionnaire can have biases such as social desirable responses and recall bias. These biases can be minimised using certain techniques.

LaFleur (2004) 157 conducted a brief narrative review of methods to measure compliance with medicine regimens. No search or inclusion/exclusion criteria were given. They state that self-report is the most popular method for assessing compliance as it is inexpensive but is often unreliable (Myers, 1998). Self-report can include patient interviews or self-report surveys. When compared to objective measures e.g. electronic monitoring devices or medicine level monitoring of compliance self-reporting has shown to over-report compliance over 50% of the time (Spector, 1986; Gordis, 1969; Waterhouse, 1993; Straka, 1997). It is also often inaccurate for those reporting non-compliance with medicine-taking. In Kwon (2003) a comparison of self-reporting of antidepressant use with prescription claims showed a 20% difference in those reporting nonadherence to antidepressants. The reasons for any discrepancies with other measures could be that patients do not understand regimens, do not know indications for their medicine, or do not report behaviours perceived as not socially-acceptable, or forgetting of non-compliance. No references were given for these assertions.

In summary, self-report by interviews or surveys can be inexpensive but can be unreliable and over-report compliance. Those who report non-compliance can also be inaccurate. There could be biases such as social desirability, recall and not understanding medicine regimes.

Turner (2002) 154 reviewed literature, to compare various measures of adherence to antiretroviral therapy. This was a narrative review with no details of search/inclusion criteria. The types of adherence measures in the review were self-report, health care provider estimation, pill counts, pharmacy-based measures, electronic monitors and biological/laboratory markers. They state that self-reports are less complex but that there can be problems with recall over long time periods. Many studies use self-report over the past 4 days but additional questions may be needed, e.g. about weekends, as this tends to be a difficult time for adherence.

All types of self-reporting overestimate adherence compared to other measures (Arnsten, 2001; Golin, 1999; Melbourne, 1999). Even those who report missing doses tend to overestimate adherence compared to other measures (Wagner, 2000). Social desirability biases can also contribute. Those who report problems with adherence usually have poorer adherence with other measures (Haynes, 1980). Those who report nonadherence appear responsive to interventions, and are important to identify (Haynes, 1980).

The validity can be increased with a preamble before the questions about adherence in order to reassure patients that information will not be held against them and that nonadherence is common. Audio computer-assisted self-interviewing is suggested for more sensitive topics (Metzger, 2000; Gribble, 2000).

A study by Bangsberg (2000) compared adherence measured by self-report (by patient interview) with provider estimation and unannounced pill counts. A comparison of pill counts with estimates of and patient self-reporting of medicines adherence showed that the physician estimates explained 26% of the variation of pill count adherence, and patients’ estimates explained 72%. They found that the sensitivity and specificity of estimates of nonadherence (<80% of pills taken according to pill count) were 72% and 95% respectively for patient interview but only 40% and 85% respectively for provider estimates.

They conclude that self-report is more easily obtained but has relatively poor sensitivity but good specificity. However electronic measures have better sensitivity but poorer specificity.

In summary, all types of self-report overestimate adherence, even with those who report nonadherence and biases such as social desirability can occur. Certain techniques could be used to minimise these biases.

Paterson (2002) 152 conducted a brief narrative review to ascertain how adherence to antiretroviral medicine should be measured. The methods reported were electronic monitoring, pill counts, pill recognition, review of pharmacy records, patient self-report, biological parameters, and medicine monitoring and provider prediction of adherence. They noted that how a question is asked can influence self-report of adherence (i.e. in face-to face inquiry or patient-completed questionnaires). A non-judgemental stance can help and this can be achieved by a preamble before the questions to show that they are not being judged and are looking for honest answers (Turner, 2001).

Another disadvantage of self-report (face-to-face interview) is that periods shorter than 7 days are not long enough to determine the percentage of adherence likely, however some patients may not correctly report adherence for 7 day periods. They state that additional questions may be necessary to counteract this e.g. about adherence at the weekend.

One method to counteract the problems of gaining honest answers is computer-assisted self-interviewing (Bangsberg, 2001) or diary. Diaries hold an advantage as they can be inexpensive and accurate. Their disadvantage is that some may complete them retrospectively or not at all.

Paterson (2002) asserts that self-report is ‘likely to be the simplest means of assessing adherence’ and so the reliability is important to assess. Adherence was found to be ‘considerably higher’ than that measured by electronic monitoring or pill count (Liu, 2001). Self-report overestimates adherence. It is most useful in those who admit to being poor adherers (Murri, 2000). They conclude that electronic monitoring devices are the closest to a gold standard in adherence measurement.

The authors conclude that there is no gold standard for measuring adherence and that electronic monitoring, in their opinion, may be the closest, yet it has some limitations. If a patient is failing to respond to treatment, self-report or pill identification should be the first option. If they report adherence this could be confirmed by electronic monitoring.

In summary, various self-reporting measures were reported. Interviews may be too late for accurate recall or may be too early to gain useful adherence information. Diaries are inexpensive and can be more accurate as there is no recall bias, however they may not be completed or completed retrospectively. Self-report can overestimate adherence but can identify those who report nonadherence.

Miller (2000) 153 reviewed current literature of measures of adherence of antiretroviral medicines in clinical trials. The types of measures of adherence were self-report, clinician estimates, pill counts, pharmacy records, clinic attendance, plasma levels, surrogate/indirect laboratory markers and electronic monitors. They report that the simplest method of measuring adherence is self-report. But there is no standardised instrument. Self-reported surveys are quick and avoid sophisticated methodology or equipment and are inexpensive compared to other methods of measurement. They have limitations, such as significantly exceeding adherence measured by other objective methods (Bond, 1991; Stratka, 1997; Cramer, 1991). HIV studies confirm this (Golin, 1999; Arnsten, 2000; Paterson, 1999; Bangsberg, 1999). Interviews and surveys often promote socially acceptable responses (DiMatteo, 1982). Less adherent patients report higher adherence than they actually had (Bond, 1991). Memory can also affect the accuracy of reporting adherence. Most surveys use broad response categories to report the proportion of pills taken, thus small degrees of nonadherence is hard to distinguish with self-report. The information is useful, but accuracy is limited and biased towards higher adherence.

Self-reported nonadherence has been associated with worse virologic outcomes (Demasi, 1999; Bangsberg, 1999; Duong, 1999; Murri, 1999; Le Moing, 1999) and as an independent predictor of clinical response to HAART when controlling objective virologic and immunologic markers (Montaner, 1999). They assert that even though it is an imperfect measure it can provide information that explains variation in clinical response to antiretroviral therapy which is not explained by other clinical factors.

The authors concluded that each method of adherence measurement has its own strengths and weaknesses. Caution should be taken when extrapolating adherence measured in clinical trials into clinical practice. Many measures have been independent predictors of clinical outcome. They may identify slightly different nonadherent populations. The different measures may complement each other and they would recommend using more than one measure, where possible.

In summary, self-report surveys are simple and inexpensive but can overestimate adherence. Interviews and surveys can have social desirability and recall biases. Also as categories are large, small degrees of nonadherence are hard to detect. There is no standardised instrument. However it can explain variation in clinical responses to ART.

Farmer (1999) 155 conducted a review of methods for measuring and monitoring medicine regimen adherence in clinical trials and clinical practice. They searched Medline for the years 1990 to 1999 and retrieved 2630 articles regarding patient compliance. The types of adherence measures included were self-report, biomarkers, direct patient observation, pill counts, prescription record review (manual and electronic) and electronic monitoring. Types of self-report included questioning/interrogation and the use of diaries and survey instruments. They tabulated the various methods for assessing adherence and their advantages and disadvantages. Patient interviews are easy to use and inexpensive but the patient can be influenced by question construction and interviewer’s skill. Adherence questionnaires are easy to administer (on site, mail, telephone), can be validated and may explain patient behaviour. However there is a lack of continuous data and the accuracy is instrument-dependent.

Patient interviews are generally considered the most unreliable for assessing adherence (Grymonpre, 1998; Matsui, 1994; Craig, 1985; Straka, 1997; Park, 1964; Inui, 1981; Gordis, 1969). Those who report nonadherence are usually correct, whereas those who say they are adherent may not be (Cramer, 1991). However it can depend on the method used and how it is used. Assessing self-reporting is difficult mainly because there are so many methods. The interviewer’s skill and the construction of the questions can affect the accuracy and validity of self-report. The relationship and communication between the HCP and patient can statistically significantly affect compliance (Davis, 1969). Highest compliance was found with those who joked, laughed and sought suggestions from their GP. The wording of questions can affect the response, and implications of blame can encourage biased responses (Ross, 1991). Some answers are socially desirable and concealed their real behaviour (Sherbourne, 1992). It is hard to assess studies of interviews as the way they are asked could bias the result.

Stewart (1987) looked at 2 compliance questions in an interview to assess medicine-taking behaviour. Comparing the results to pill counts, the questions had a specificity of 69.8% and sensitivity of 80%, therefore an overall 74.5% accuracy. The time frame used for recall can differ, some researchers do not specify, others are 7–10 days and some are a month (Grymonpre, 1998; Dirks, 1982; Straka, 1997). To correct these problems some researchers have tried to construct a standardised questionnaire for measuring adherence. For example Morisky (1986) developed a 4-item questionnaire specific to medicine regimen adherence. It was assessed on unidimensionality, reliability and concurrent validity with blood pressure control. The instrument’s sensitivity was 81% and specificity 44%. It was not found to be efficient at predicting poor adherence (Morisky, 1986).

Svarstad (in press at time of review) developed a self-administered instrument called the Brief Medication Questionnaire. The accuracy was assessed using MEMS. There were 3 sets of questions – 5 regimen screen items, 2 belief screen items and 2 recall screen items. The sensitivity for repeat nonadherence was 80% for the regimen screen, 100% for the belief screen and 40% for the recall screen. The specificity for repeat adherence had 100% for the regimen, 80% for the belief and 40% for the recall screen. The accuracy of reported repeat nonadherence was 95% for the regimen, 85% for the belief and 40% for the recall screen.

The authors concluded that each method has strengths and weaknesses depending on the intended use. When selecting a specific method an assessment of each method’s validity should be undertaken. A combination of methods is recommended.

In summary, several methods of self-report were examined. Interviews are simple and inexpensive, but can depend on the interviewer. Questionnaires can be administered in a variety of methods, but are considered the most unreliable. Those who say they are nonadherent are usually being truthful but many who say they are adherent may not be.

Bender (1997) 156 conducted a literature review to assess nonadherence in asthmatic patients. A search of Medline was made from 1990 to 1997 of all pertinent articles, preferably controlled studies. Types of adherence measures were biochemical measurement, clinical judgement, medicine measurement, pharmacy database review and electronic medicine monitoring. Self-report measures can be collected by interview, diaries and questionnaires but no validated adherence-specific questionnaire is commonly used as they are often specific to the studies. Self-report measures are simple, inexpensive and usually brief and so they are commonly used to measure adherence. Especially in the clinical setting they are the best measure for collecting information on beliefs, attitudes and experiences with medicine regimes. Accuracy with other measures is highly variable. Spector (1986), Coutts (1992) and Gibson (1995) compared asthmatics self-reporting of inhaler usage with electronic medicine monitoring devices and they showed that asthma diaries usually overestimate adherence. Demands of the setting can influence the usefulness and reliability of the information gained from self-reporting. These can be a desire to please on the part of the patient and the Health Care Professional’s skill and sensitivity in eliciting self-reports. When collected well it can give good insight into patients’ problems with adherence. As they are unlikely to identify themselves as non-adherers unless they are this helps identify them (Coutts, 1992; Spector, 1986; Dolce, 1991; Morisky, 1990).

The authors state that while self-report may not be a sufficient measure of adherence in many settings and particularly in research, it is probably a necessary measure in all settings.

In summary, self-report measures are simple, inexpensive, brief and the best way of collecting information in the clinical setting. However diaries overestimate adherence and the demands of the setting can influence the usefulness and reliability of the measure.

Rand (1994) 158 reported in a narrative review on measuring adherence to asthma medicine regimens. They did not state search or inclusion criteria. The types of adherence measures included in the review were biochemical measures, observation of MDI technique, clinical judgement, medicine measurement and medicine monitors.

They state that self-report is the most inexpensive and quick way of measuring adherence (Soutter, 1974). The possible advantage of diary cards is that they can measure adherence across time and can reveal patterns between the disease exacerbation and compliance with the medicine. As there are many medicines used within asthma prescribing, it can help to see the adherence of certain medicines rather than just overall. It can also specifically assess overuse, inappropriate use or erratic use of medicines as well as triggering events for the need for medicine e.g. in Kesten (1991). Asthma diaries may share commonalities but there is no standardised diary as such in research. A disadvantage of asthma diaries is they may be be complex and time-consuming. Also criteria of acceptable adherence may differ from patient to patient. One way to evaluate the level of adherence is to use trained, masked, medical personnel to score the compliance. It is preferable to develop standardised compliance criteria for all raters and train them by a standardised protocol and make sure there is inter-rater reliability.

Many studies have used questionnaires to collect clinic or follow-up data for patient adherence (Bailey, 1987; Kinsman, 1980; Dolce, 1991), mainly designed for a particular research project. Many include adherence questions within a larger questionnaire, such as the 76-item Revised Asthma Problem Behaviour Checklist (RAPBC) and the 72-item Asthma Problem Behaviour Checklist (APBC). These have been found to be reliable with test-retest correlation coefficients of r=0.95 and r=0.80 respectively. However no reliability and validity information was available for those items specifically measuring adherence to medicine.

Rand (1994) points out that both asthma diaries and self-report are the most common methods for assessing asthma medicine adherence but these instruments, because they are not standardised or not published, rarely have their validity and reliability assessed. Except for Adherence Scale and Inhaler Adherence Scale (Kinsman, 1980; Dolce, 1991; Bailey, 1990), which are six-item scales based on Morisky’s work (1990). This instrument was found to have a Chronbach’s alpha of 0.76 and 0.69 and was concordant with outcome measures in the UAB adult asthma study.

The limitations of self-report have been mentioned by many authors (Masur, 1981; Mawhinney, 1991; Cramer, 1989; Rand, 1992). When compared to objective measures it varies highly on the degree of accuracy (Gordis, 1966; Mattar, 1974). Diary self-reports were compared to an electronic medicine monitoring device to measure adherence to asthmatic medicine by Spector (1986). The findings were that all patients self-reported using the inhaler on certain days, whereas the measured medicine suggested just over half (52.6%) actually did so. Adding a diary can add more complexity to the patient regime than there all ready is. It has been shown that the greater the complexity of a regime the lower the compliance (Masur, 1981). Some participants alter their records of medicine use to appear compliant (Mawhinney, 1991; Rand, 1992). This can be improved if they also have reporting by the family/partner of the patient (Paulson, 1977).

Self-reporting can also depend on the individual patient or practitioner. For example elderly patients may have memory impairment, especially when taking many medicines and therefore do not report accurately. Long-term usage may be forgotten but they may be able to recall recent usage. The skill and sensitivity of the Health Care Practitioner can also play a role in how much information is given and the reliability of it. When collected carefully it could be very good insight into the problems of a patient’s adherence. Also it is unlikely that patients will represent themselves as non-adherers (Gordis, 1976) so it will identify non-adherers correctly.

In summary, self-report is generally inexpensive and quick. Diaries can measure adherence across time and reveal any patterns and overuse of medicine. However there is no standardised diary and it can sometimes be complex and time consuming. If there is no standardised questionnaire or diary then no validity or reliability are assessed. Therefore there is variation in accuracy and it can depend on the individual or practitioner.

Bennett-Johnson (1992) 160 conducted a narrative literature review of adherence measurement in diabetes management. No search or inclusion criteria were given. The types of adherence measures used were health-status indicators, health provider ratings, behavioural observations and pill counts.

They point out that self-report of regimen adherence is often mistrusted. Patients may say one thing but do something completely different, often because of what they think the professional wants to hear. However noncompliance self-reporting appears more valid than self-reporting of compliance (Diehl, 1987). Asking about specific behaviours can lead to better adherence data (Cerkoney, 1980; Cox, 1984; Shlenk, 1984; Brownlee-Duffeck, 1987; Hanson, 1987; Hanson, 1988; Hanson, 1990). There have only been a few that have looked at the reliability of these reports (Hanson, 1987 and Hanson, 1988). If asked to report their specific behaviours over a certain time period, the data can be good quality (Glasgow, 1987; Johnson 1986). Glasgow (1987) used written diaries successfully to measure adherence and Johnson (1986) adapted the 24-hour recall interview (which is a standard dietary assessment method) to use as a general adherence assessment strategy with IDDM patients. The authors state that ‘in a series of studies the authors have demonstrated both the reliability and validity of this technique’ (Bennett Johnson, 1992). These studies referred to are Johnson (1986); Johnson (1990); Spevack (1991); Reynolds (1990); Johnson (1990) and Freund (1991). Multiple interviews are recommended to ensure representation of adherence behaviours.

One disadvantage with self-reporting is problems of memory recall. Where possible a significant other should additionally be interviewed regarding the patient’s behaviour.

The advantages of self-report are numerous, as reliable information can be obtained; interviews can be done over the telephone making them accessible; the patient does not have to do very much apart from give their time for an interview. They however do need trained interviewers, or with multiple interviews and multiple patients the process can take a lot of time and effort. No references were made for these assertions.

In summary, self-reporting of non-compliance is likely to be more valid, whereas compliance reporting is not valid. They can ask about specific behaviours and find out about what leads to non-compliance. It is easy for the patients to do and interviews can be done by telephone call. However there are biases with recall and people may say one thing but do another and there can be errors in reporting e.g. self-observation skills.

Dunbar (1989) 163 reviewed the methods to assess adherence to arthritis medicine with a review that included ‘16 representative studies of compliance’. No inclusion/exclusion criteria or search details were given. The review included self-report, clinician judgement, therapeutic outcome, direct observation, biological measures, pill counts, pharmacy refills and electronic monitors.

They noted that a major problem is the accuracy of reporting, with poor compliance usually underreported. One issue is the memory decay when assessing adherence (Farr, 1987). Effects, such as not realising the diminishment of higher adherence levels has occurred and moving past events forward in perception can all lead to inaccuracy. Motivational factors are also important, errors in reporting can be due to self-observation skills, especially when the compliance behaviour is itself variable. Misconceptions of the regimen may lead to errors through inaccurately labelling events compliant or noncompliant.

Self report has advantages in that it can identify some noncompliance in a cost-efficient manner and permits an in-depth study of the types of errors that patients can make which leads to non-compliance. It also has been shown to have reasonable sensitivity and specificity in discriminating those who comply from those who do not. In one study of medicine adherence self-report showed 100% sensitivity and 40% specificity when serum levels were used as a standard (Craig, 1985). The authors assert that ‘self-report can be a useful measure. However, it is important to attend to the collection of accurate information.’

The authors conclude that self-report can be a useful measure and interviewers’ skills are very important. Clinical measures are fraught with problems and there is no perfect measure.

In summary, self-reporting can mean poor compliance is underreported, there can be recall bias and self-observation skills may be erroneous. It is cost-efficient and can identify non-compliers.

Hecht (1998) 159 reported briefly with a narrative review on measures for HIV adherence in clinical practice. The types of adherence measures mentioned were self-reporting, medicine levels, physician judgement, MEMS, pill counts and prescription refills.

Sackett (1975) compared self-report to pill counts. Of those that reported having less than 80% adherence, 95% were found nonadherent by pill count. Those reporting that they were adherent over 80% of the time, were shown to be nonadherent by pill count 34% of the time. Gilbert and Sackett’s studies, suggest that self-report is more accurate than physician assessment. Thus if HCPs want to know if patients are taking anti-retroviral therapy, they need to ask them rather than relying on their judgement. When they say they are missing medicine, believe them, as this is mostly the truth. Patient self-report tends to overestimate adherence. Those who report missing doses infrequently may have a significant problem of nonadherence.

Hecht (1998) says that what matters is how Health Care Practitioners ask the questions. Stating it should be in a specific, non-judgmental way and one that allows them to disclose nonadherence. Therefore, questions should not imply that they are wrong if they do not take their medicine the way they are ‘supposed to’. A time period must also be specified. No references given for these conjectures. Measuring medicine levels should be regarded as a supplementary measure. Electronic pill monitoring, pill counts, reviewing prescription refills can be useful adjuncts to self-report in certain contexts, but every method has its limitations.

In summary, self-report is more accurate than Health Care Practitioners’ judgement alone. It tends to overestimate adherence. It depends on how the questions are asked and a time period must be specified.

Overall summary

This evidence review focused on the advantages and disadvantages of self-report for assessing adherence. These were primarily narrative reviews rather than systematic reviews.

These reviews reported that all measures of adherence have strengths and weaknesses. There is no gold standard. Self-report can vary in reliability, yet it was thought generally to be a useful measure of adherence. Those who report nonadherence are likely to be telling the truth. It is also good for finding out the reasons for nonadherence. In a couple of studies it was suggested that self-report could be the first measure of adherence and for those who report adherence it could be supplemented by other measures. It is primarily a clinical tool, whereas other measures may be more relevant to clinical trials.

Any questionnaire which measures self-report should be well-designed and validated. Many of the reviews reported that the success of interviews as a measure largely depended on the skills and communication of the interviewer. It could depend on the way a question is asked.

Copyright © 2009, Royal College of General Practitioners.

All rights reserved. No part of this publication may be reproduced in any form (including photocopying or storing it in any medium by electronic means and whether or not transiently or incidentally to some other use of this publication) without the written permission of the copyright owner. Applications for the copyright owner’s written permission to reproduce any part of this publication should be addressed to the publisher.

Bookshelf ID: NBK55447
PubReader format: click here to try

Views

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...