• We are sorry, but NCBI web applications do not support your browser and may not function properly. More information
Logo of jmirEditorial BoardMembershipSubmitCurrent IssueArchiveReviewSearchAboutJMIR PublicationsJMIR
J Med Internet Res. Mar 2013; 15(3): e57.
Published online Mar 14, 2013. doi:  10.2196/jmir.2059
PMCID: PMC3636293

Cost-Effectiveness and Cost-Utility of Internet-Based Computer Tailoring for Smoking Cessation

Monitoring Editor: Gunther Eysenbach
Reviewed by Jamie Banks and Jan Van Ooteghem

Abstract

Background

Although effective smoking cessation interventions exist, information is limited about their cost-effectiveness and cost-utility.

Objective

To assess the cost-effectiveness and cost-utility of an Internet-based multiple computer-tailored smoking cessation program and tailored counseling by practice nurses working in Dutch general practices compared with an Internet-based multiple computer-tailored program only and care as usual.

Methods

The economic evaluation was embedded in a randomized controlled trial, for which 91 practice nurses recruited 414 eligible smokers. Smokers were randomized to receive multiple tailoring and counseling (n=163), multiple tailoring only (n=132), or usual care (n=119). Self-reported cost and quality of life were assessed during a 12-month follow-up period. Prolonged abstinence and 24-hour and 7-day point prevalence abstinence were assessed at 12-month follow-up. The trial-based economic evaluation was conducted from a societal perspective. Uncertainty was accounted for by bootstrapping (1000 times) and sensitivity analyses.

Results

No significant differences were found between the intervention arms with regard to baseline characteristics or effects on abstinence, quality of life, and addiction level. However, participants in the multiple tailoring and counseling group reported significantly more annual health care–related costs than participants in the usual care group. Cost-effectiveness analysis, using prolonged abstinence as the outcome measure, showed that the mere multiple computer-tailored program had the highest probability of being cost-effective. Compared with usual care, in this group €5100 had to be paid for each additional abstinent participant. With regard to cost-utility analyses, using quality of life as the outcome measure, usual care was probably most efficient.

Conclusions

To our knowledge, this was the first study to determine the cost-effectiveness and cost-utility of an Internet-based smoking cessation program with and without counseling by a practice nurse. Although the Internet-based multiple computer-tailored program seemed to be the most cost-effective treatment, the cost-utility was probably highest for care as usual. However, to ease the interpretation of cost-effectiveness results, future research should aim at identifying an acceptable cutoff point for the willingness to pay per abstinent participant.

Keywords: randomized controlled trial, economic evaluation, smoking cessation, Internet, computer-tailoring, general practice

Introduction

Background

Smoking is the single most preventable cause of illness and premature death in the world and is an important risk factor for 6 of the 8 leading causes of death, including several types of cancer, cardiovascular diseases, and respiratory diseases [1]. Consequently, smoking is related to a reduced quality of life and places a burden of €4 to €7 billion on health care [2-4]. Thus, quitting smoking is important, not only to improve individual and population health, but also to reduce smoking-related health care costs.

Extensive evidence exists on the clinical effectiveness of behavioral interventions for smoking cessation [5-7]. Brief advice from a general practitioner is one of these effective smoking cessation interventions [8]. However, general practitioners and practice nurses often report a lack of time and skills to provide their patients with elaborate smoking cessation advice [9,10]. Another behavioral intervention that has proven to be effective in increasing smoking cessation rates up to 13 months is computer tailoring [5,7,11-16]. Computer tailoring entails the adaption of the content of an intervention to participants’ individual characteristics by using computer programs [17]. Most often, a questionnaire is used as a screening instrument [12,18,19]. The answers smokers provided on the questions in this screening instrument are accumulated into a large data file and are subsequently matched with relevant feedback messages that are ultimately combined into a tailored feedback letter. Tailored interventions are more effective in attracting and keeping the smoker’s attention [17,20], resulting in better processing of information [21]. A single tailored feedback message is successful in increasing cessation rates [15], but dynamically tailored feedback provided on multiple occasions can be even more effective [11,12,22]. Due to the automatic generation of the tailored feedback and the fact that computer-tailored interventions are increasingly delivered online [23,24], the integration of an Internet-based computer-tailored program in the general practice setting might limit the burden on health professionals and patients, reduce facility and administrative costs, and could potentially be time- and cost-saving. As a combination of effective interventions was expected to achieve higher abstinence rates than either of the 2 alone [25], our research team has developed a smoking cessation intervention consisting of Internet-based multiple computer tailoring and a single tailored counseling session by a practice nurse.

Despite the proven clinical effectiveness of behavioral smoking cessation interventions, information about their relative cost-effectiveness is limited. Previously, several cost-effective smoking cessation interventions have been developed [26-28]. However, the interventions studied all involved the use of smoking cessation medication and investigated the cost-effectiveness of referrals to intensive counseling combined with pharmacotherapy [27], reimbursement of smoking cessation support [28], or the smoking cessation drug varenicline [26]. With regard to behavioral smoking cessation interventions, a computer-based smoking cessation intervention for primary care professionals was successful in increasing abstinence rates and quality-adjusted life years (QALYs) among patients [29]. In addition, in response to a call in the Journal of Medical Internet Research for research to economically evaluate eHealth interventions [30], cost-effectiveness and cost-utility studies of Internet-based interventions aimed at alcohol reduction [31] and depressive symptom treatment [32,33] have been initiated. However, to our current knowledge, no such studies have yet been conducted concerning the cost-effectiveness and cost-utility of an Internet-based smoking cessation intervention.

Therefore, the objective of the present study was to compare the cost-effectiveness and cost-utility of (1) an Internet-based multiple computer-tailored smoking cessation program combined with a single tailored counseling session by a practice nurses, (2) only an Internet-based multiple computer-tailored smoking cessation program, and (3) care as usual, defined as practice nurses’ standard care regarding smoking cessation.

Methods

Design

Economic evaluation studies aim to determine the costs and effects associated with an intervention and to compare these with the costs and effects of other interventions and/or current practice [34]. They usually consist of 5 steps [35], which are listed in Textbox 1. For a more extensive and detailed description of each of these steps, refer to Multimedia Appendix 1.

The 5 steps in economic evaluation studies.

Step 1. Identification of relevant costs and effects

Based on a chosen perspective (eg, the health care perspective, health insurer perspective, or societal perspective), relevant costs and effects are identified.

Step 2. Measurement of costs and effects

Costs can be assessed prospectively by means of cost diaries, or retrospectively using cost questionnaires. Effects are usually assessed in terms of quality of life.

Step 3. Valuation of measured costs and effects

Health care and patient costs are usually valued in a monetary currency using manuals for cost analysis in health care research. Effects on quality of life are usually valued in QALYs gained or lost.

Step 4. Calculation of a cost-effectiveness ratio

When comparing two interventions, an incremental cost-effectiveness ratio (ICER) can be calculated: ICER=(Ci–Cc)/(Ei–Ec). When comparing more than 2 interventions, a net monetary benefit (NMB) should be calculated using the willingness to pay (WTP): NMB=(Ei–Ec)×WTP–(Ci–Cc). A description of both formulas can be found in Multimedia Appendix 1.

Step 5. Uncertainty analysis

To deal with the sampling uncertainty bootstrap analyses can be used, whereas a sensitivity analysis can be conducted to deal with uncertainty due to the assumptions made.

The present economic evaluation study was trial-based, embedded in a randomized controlled trial (RCT) that tested the effectiveness of an Internet-based multiple computer-tailored smoking cessation program and tailoring counseling by practice nurses. This 3-armed RCT was conducted among Dutch adult smokers and had a follow-up period of 12 months. From May 2009 to June 2010, 91 practice nurses working in different Dutch general practices throughout the Netherlands recruited smoking patients for participation in the RCT. To aid recruitment, several recruitment materials were provided (eg, desk displays, posters, and business cards). Smokers interested in participation could sign up for the study on the study website. There, information was provided about the objectives of the study, the randomization procedure, and the incentive provided when respondents completed all questionnaires (ie, a €10 gift voucher). When signing up, participants were able to choose their own username and password and were informed that no one but the research team would be able to retrieve these passwords. After providing informed consent, participants were randomized into 1 of the 2 intervention groups (multiple tailoring and counseling or multiple tailoring only) or into the usual care control group. Randomization took place at the participant level by means of a computer software randomization device.

The trial design was approved by the Medical Ethics Committee of Maastricht University and the University Hospital Maastricht (MEC 08-3-037; NL22692.068.08), and is registered with the Dutch Trial Register (NTR1351). A more detailed description of the study design has been published elsewhere [25].

Participants

Participants were eligible for participation if they smoked, were motivated to quit within 6 months, were 18 years or older, and were able to read and understand Dutch sufficiently to read study materials and participate in the trial. Moreover, they had to have access to the Internet. This resulted in a total of 414 eligible smokers.

The Interventions

Figure 1 presents an overview of the intervention components in each of the study groups.

Figure 1
Overview of the intervention elements received by the 3 groups.

The Internet-based multiple computer-tailored smoking cessation program was based on a previously developed single computer-tailored intervention [12,15] for which the I-Change model (ICM) formed the theoretical framework [36]. As was its predecessor, the attitude-social influence-efficacy model [37], the ICM is a theory of behavioral change which incorporates theoretical concepts from several sociocognitive models, including the transtheoretical model [38], the theory of planned behavior [39], social cognitive theory [40], and the health belief model [41]. According to the ICM, the most proximal predictor of behavior is the intention to perform this behavior. Intention is predicted by 3 motivational constructs, attitude, perceived social influence, and self-efficacy, which in turn can be predicted by several premotivational factors, such as awareness, previous experience with the same and related behaviors, biological factors, and sociocultural factors. To overcome barriers that increase the well-known gap between intention and behavior (eg, [42]), the ICM proposes ability factors, such as an individual’s skills to refrain from smoking, and the formation of action plans. The ICM has been used successfully to develop several other effective computer-tailored programs [12,15,18]. While filling out the first online questionnaire (ie, baseline questionnaire), all participants were asked to set a date within the next 4 weeks at which they would attempt to quit smoking. They received a total of 4 feedback letters: at baseline, 2 days after the quit date they had set for themselves at baseline, after 6 weeks, and after 6 months. Feedback was personalized and tailored to several participant characteristics: gender, attitude, social influence and self-efficacy, intention to quit smoking, action planning, and smoking behavior. Feedback letters were iterative: the second, third, and fourth feedback letters did not only concern the participant’s present state, but also referred to changes participants had made since they were included in the program. Most feedback letters consisted of 4 to 5 pages and 7 components: (1) introduction, including specific feedback on the respondent’s smoking behavior and on his/her intention to quit smoking and to maintain nonsmoking; (2) feedback on the respondent’s attitude, including perceived advantages (pros) and disadvantages (cons) about smoking and quitting smoking; (3) feedback on perceived social influence (not) to smoke; (4) feedback on the respondent’s reported self-efficacy to refrain from smoking in specific situations, including suggestions on how to cope with these situations; (5) feedback on the extent to which respondents were planning to undertake specific actions (action plans) while preparing their quit attempt; (6) feedback on how to cope with situations in which it might be difficult not to smoke (coping plans), including the formulation of personal plans in the shape of if-then statements [28]; and (7) ending. Participants could access their feedback letters directly online after questionnaire completion. Additionally, feedback letters were sent to the participant by email. In both cases, feedback letters could be printed. An example of a tailored feedback message is provided in Multimedia Appendix 2.

After receiving the first tailored feedback, participants in the multiple tailoring and counseling group were prompted to schedule a counseling meeting with their practice nurse within 6 to 8 weeks. They received this counseling session instead of the third tailored feedback letter at the 6-week follow-up. A counseling protocol was provided to assist practice nurses in guiding these counseling sessions. This protocol consisted of 3 chapters guiding on 3 different types of participants: smokers who had quit successfully, smokers who had quit but relapsed, and smokers who had not yet quit. The content of the counseling session was developed to be as similar as possible to the content of the computer-tailored feedback and was also tailored to the participant characteristics mentioned previously. After 6 months, practice nurses were instructed to call their patients to ask them about their progress toward permanent cessation and, if needed, to provide them with additional cessation support.

Participants randomized in the usual care group received smoking cessation guidance according to participating practice nurses’ standard practice, which can vary from a brief intervention consisting of a single recommendation to stop smoking to more intensive interventions [43,44].

Measurements

Self-reported online questionnaires were used to assess both costs and effects. Questionnaires were administered at baseline and at 6-week, 6-month, and 12-month follow-ups. When follow-up questionnaires were not completed by 1 week after the invitation, an email reminder was sent. At 12-month follow-up, this email reminder was followed by a phone call to collect data.

Identification, Measurement, and Valuation of Costs

The present economic evaluation study was conducted from a societal perspective. This implies that intervention costs, health care costs, and patient costs were identified as relevant. Intervention costs consisted of all costs that could be attributed to the delivery of the intervention, such as hosting costs for the Internet-based program and costs associated with counseling sessions. Costs for the development of the intervention and research-specific costs were excluded because these costs are sunk costs, costs that would already be spent before the intervention is implemented. In total, intervention costs were €57.70 per participant in the multiple tailoring and counseling group and €7.70 per participant in the multiple tailoring group. Interventions costs in the usual care group were considered zero because no intervention materials needed to be developed for this group. Health care costs related to general practitioners’ or practice nurses’ (telephone) consultations or home visits (other than the counseling session which was part of the multiple tailoring and counseling intervention), inpatient and outpatient specialist care, alternative medicine, mental health care, prescribed and over-the-counter (OTC) smoking cessation medication, hospital admissions, smoking cessation aids, and other care (eg, paramedics consultations or professional home care). Patient costs consisted of traveling and time lost due to participation in the intervention. However, for primary and secondary analyses, patient costs were not valued in monetary costs, but considered as reflected in participants’ reported quality of life [45].

Self-reported health care use was assessed during a 12-month follow-up period using a 3-month retrospective costing questionnaire that consisted of open-ended questions. Participants indicated whether they had received each type of care during the past 3 months, and if so, how often. The time participants spent using the online tailoring program was tracked by computer-registered log-in and log-out data. To assess time spent on counseling, we used a mean time of 20 minutes for face-to-face counseling sessions and an average of 10 minutes for telephone consultations. Traveling time was measured based on average travel distances to a general practitioner in the Netherlands [45].

To valuate health care usage and patient costs, the updated Dutch manual for cost analysis in health care research was used [45]. In general, standardized prices were used, but when no standardized prices were available, real costs or tariffs were used to estimate costs. In case of uncertainty, we used the lowest price. Costs of smoking cessation medication were calculated based on daily-defined dosage [46], including 6% Value Added Tax, prescription charges for prescribed medication, and clawback, a lawful discount percentage to be subtracted from medication prices by pharmacists [46]. Prices of informal care were based on shadow prices for unpaid work. The participants’ time spent on the program was valued by using the friction cost approach [45]. The index year used was 2011. Because prices in the Dutch manual for cost analysis in health care research [45] were from the year 2009, these prices were indexed to the year 2011. The consumer price indexes used were 105.38 for 2009 and 109.02 for 2011 [47]. A 13-month recruitment period and a 12-month follow-up period can both be considered as relatively short; therefore, it is unlikely that any substantial differences in health care consumption and effects existed between participants who were included at the beginning and toward the end of the recruitment period. As a result, there were no reasons to discount volumes of health care consumption or effects.

Identification, Measurement, and Valuation of Effects

The primary outcome measure used in the cost-effectiveness analysis (CEA) was prolonged abstinence measured at 12-month follow-up. This was assessed by 1 item asking whether the participant had refrained from smoking since the previous measurement at 6-month follow-up (ie, abstinence for at least 6 months; 1=no, 2=yes). Secondary outcome measures were 7-day point prevalence abstinence assessed by 1 item asking whether the participant had refrained from smoking during the past 7 days (1=no, 2=yes) and addiction level measured by the abbreviated Fagerström Test for Nicotine Dependence (FTND; 0=not addicted, 10=highly addicted) [48]. Self-reported abstinence at 12-month follow-up was cotinine validated using a saliva swab test [49]. Smoking abstinence was expressed in a probability score that a smoker would be abstinent (1=not abstinent, 2=abstinent).

The primary outcomes measure for the cost-utility analysis (CUA) was quality of life, measured in terms of QALYs. The valuation of effects on quality of life implies that utility scores need to be computed. In the present study, utilities were measured by the EuroQol EQ-5D [34,50], which is the current recommended measure for assessing quality of life by the National Institute for Health and Clinical Excellence [51] and has been used in other evaluations of smoking cessation programs [52,53] and Internet-based interventions aimed to reduce other health-related problems [32]. The EQ-5D consists of 5 health state dimensions (mobility, self-care, usual activity, pain/discomfort, and anxiety/depression) on which participants have to indicate their own health state (1=no complaints, 2=some complaints, 3=many complaints) [50]. Utility scores assessed at different points in time were transformed into an overall QALY score using the area under the curve method. The area under the curve is the duration of the health state (on the x-axis; in our case 12 months/1 year) multiplied by the quality weight for the health state (on the y-axis; utility scores). The resulting QALY score represents the number of QALYs gained or lost during the 12-month follow-up period [34]. For example, gaining 1 QALY means that 1 year is gained in perfect health, whereas gaining 0.8 QALY means that 1 year is gained in less than perfect health (utility score of 0.8) or that 0.8 years are gained in perfect health (utility score of 1).

Analyses

All analyses were conducted according to the intention-to-treat principle. Missing data for costs, EQ-5D items, overall tobacco consumption, and addiction level were replaced by mean imputation by using participants’ scores on the previous and next measurement. When mean imputation was not possible because of missing data on multiple measurement points, missing data were replaced using the last observation carried forward (preferred choice) or next observation carried backward method. Missing data for smoking abstinence were replaced using a negative scenario; participants lost to follow-up were considered still smoking.

Baseline Comparability of the Three Study Groups

To investigate the comparability of the 3 groups with regard to demographics, baseline values of outcomes, and health care–related costs over the past 3 months, 1-way analyses of variance (ANOVA) with Tukey post hoc tests and chi-square tests were conducted. To determine whether selective dropout had occurred, a comparison was made between those lost to follow-up and those who remained in the study after 12 months using 2-sided t tests and chi-square tests.

Annual Costs and Effects

The 3 groups were compared with regard to their mean annual costs using nonparametric bootstrapping (5000 times) with 95% confidence intervals in percentiles [34]. To compare the 3 groups with regard to the mean effect assessed 12 months after baseline, 1-way ANOVA with Tukey post hoc tests and chi-square tests were conducted.

Cost-Effectiveness and Cost-Utility Analyses

First, incremental costs and effects were calculated for each of the 3 treatments studied. Subsequently, an incremental cost-effectiveness ratio (ICER) was calculated to compare costs and effects between pairs of study groups according to the following formula: ICER=(Ci–Cc)/(Ei–Ec). In this formula, C i is the adjusted annual costs of the intervention group, C c is the adjusted annual costs of the control group, E i is the adjusted effects for the intervention group, and E c is the adjusted effects of the control group. With regard to quality of life, ICERs are often called incremental cost-utility ratios (ICURs). However, because ratios such as the ICER or ICUR can compare only 2 groups, to compare the 3 groups in our study a net monetary benefit (NMB) was calculated for each of 3 treatments. The NMB can be calculated by valuing the effectiveness and utility outcomes in monetary values using a threshold for society’s willingness to pay (WTP) per abstinent participant and per QALY gained [54] according to the following formula: (Ei–Ec)×WTP–(Ci–Cc). In the present study, we used a WTP of €18,000 because this is an accepted Dutch cutoff point for the WTP per QALY [55], to calculate the likelihood that each treatment would be most likely highest in cost-effectiveness/cost-utility.

Uncertainty Analysis

Sampling uncertainty around the estimates of cost-effectiveness and cost-utility was taken into account by using nonparametric bootstrap resampling techniques [34,56]. Using bootstrapping techniques with replacement n (often 1000) times, a random sample is drawn from the original dataset resulting in 1000 slightly different samples and thus slightly different ICERs. Of these 1000 ICERs, the percentage can be calculated with (1) more effects and lower costs (dominant), (2) with less effects and lower costs, (3) with more effects and higher costs, and (4) with less effects and higher costs (inferior).

To deal with the uncertainty of parameter estimates from the primary analyses, a sensitivity analysis was conducted. As described earlier, in primary analyses patient costs (ie, traveling and time costs) were not valued in monetary costs but considered as reflected in participants’ reported quality of life [45]. However, because patient costs can be considered directly related to the treatment received [45], it could be argued that these costs should be included as part of the program cost. Therefore, we tested whether an increase in program costs as a result of patient costs’ monetary valuation would lead to a change in results. For the multiple tailoring and counseling group, the inclusion of patient costs meant an increase in program costs from €57.70 to €141.89 per participant; for the multiple tailoring group, this meant an increase in program costs from €7.70 to €82.24 per participant.

Bootstrap analyses were conducted using Microsoft Office Excel 2003. All other analyses were conducted using SPSS 17.0 (SPSS Inc, Chicago, IL, USA).

Results

Sample Characteristics

Of the 414 participants who were eligible for participation, 163 were randomized into the multiple tailoring and counseling group, 132 into the multiple tailoring group, and 119 into the usual care group. No baseline differences were found between the 3 groups (Table 1). After 12 months, 231 (55.8%) of the 414 participants could be followed up. Participants lost to follow-up were significantly younger than those who remained in the study (P=.01). Of the 46 participants who reported prolonged abstinence at the 12-month follow-up, 30 successfully completed a cotinine test and all cases of self-reported abstinence were confirmed. After imputation of missing values, total cost data were available for 370 participants (89.2%) whereas effect data were available for 414 (100%) participants for abstinence measures, 409 (98.8%) for addiction level, and 384 (92.8%) for QALYs.

Table 1
Comparability of the 3 groups, multiple tailoring and counseling (MTC), multiple tailoring (MT), and usual care (UC), regarding demographics, baseline values of outcomes, and health care–related costs over the past 3 months (N=414).

Annual Costs and Effects

A significant difference was found between the multiple tailoring and counseling and the usual care groups with regard to annual health care–related costs per patient, with significantly higher costs in the multiple tailoring and counseling group (Table 2). No differences were found between the 3 groups with regard to any specific type of health care–related costs (Table 2), nor regarding effects on abstinence, QALYs, or addiction level assessed at 12-month follow-up (Table 3).

Table 2
Mean annual costsa per participant in the MTC, MT, and UC groups.
Table 3
Mean annual effect on smoking abstinence, QALY, and addiction level in the MTC, MT, and UC groups.

Cost-Effectiveness Analyses

Table 4 shows that for participants in the multiple tailoring and counseling group costs were higher, whereas effects were lower than in the usual care and multiple tailoring groups; thus, multiple tailoring and counseling was dominated by the other 2 treatments. Comparing multiple tailoring with usual care showed that multiple tailoring resulted in more costs, but also in more effects. Compared with usual care, €5100 had to be paid within the multiple tailoring group for each additional abstinent participant (Table 4).

Table 4
Incremental costs and effects per abstinent smoker and per QALY gained for the MTC, MT, and UC groups with a willingness-to-pay threshold of €18,000.

The CEA showed that until a threshold value for the WTP of €5100 per abstinent participant, usual care was probably the most efficient treatment. However, from a WTP of €5100 or higher, multiple tailoring was probably most cost-effective (Table 5). With the accepted Dutch cutoff point of €18,000 per QALY for preventive interventions [57], multiple tailoring would be the preferable treatment. These results are visually displayed in the cost-effectiveness acceptability curve (CEAC), showing the probability of each treatment being preferable to the other 2 treatments for varying levels of the WTP per additional abstinent participant (Figure 2). Sensitivity analyses supported these results (Table 5).

Table 5
Results from cost-effectiveness and cost-utility analyses based on 1000 bootstrap replications.
Figure 2
Cost-effectiveness acceptability curve for the 3 treatments studied: MTC, MT, and UC.

Results from secondary analyses showed that with 7-day point prevalence abstinence, a high probability was found (ie, 88%, with a WTP of €18,000 per abstinent participant) that multiple tailoring was the most cost-effective treatment. Regarding the level of addiction, however, it was most probable that multiple tailoring would be least efficient (Table 5).

Cost-Utility Analyses

With regard to QALYs gained, Table 4 shows that multiple tailoring was dominated by usual care because this treatment was both more expensive and less effective. Furthermore, cost-utility analyses showed that multiple tailoring and counseling was more expensive, but also more effective than usual care and multiple tailoring in increasing the number of QALYs gained. This resulted in an incremental cost of €40,300 per QALY gained when comparing multiple tailoring and counseling with usual care, and in an incremental cost of €18,367 per QALY when comparing multiple tailoring and counseling to multiple tailoring.

With a WTP of €18,000 per abstinent participant, the CUA showed that usual care would probably (ie, 64%) be the most efficient treatment (Table 5). Although decreasing this threshold value to €0 led to an increased probability that usual care would be most efficient, increasing this threshold led to a lower probability of usual care being most preferable. With a WTP of almost €40,000, usual care and multiple tailoring and counseling would be equally preferable. These results are further illustrated in the cost-utility acceptability curve (CUAC) (Figure 3). Sensitivity analyses showed similar results (Table 5).

Figure 3
Cost-utility acceptability curve for the 3 treatments studied: MTC, MT, and UC.

Discussion

Main Findings

To our current knowledge, this was the first study to determine the cost-effectiveness and cost-utility of a behavioral smoking cessation intervention consisting of Internet-based computer tailoring with and without counseling by a practice nurse. The results presented suggest that participants who received the Internet-based multiple computer-tailored program and tailored counseling by their practice nurse reported significantly more annual health care-related costs than participants who received care as usual. A potential explanation for this finding might be that smokers were prompted by the tailored feedback they received to ask for more smoking cessation guidance, eg, additional counseling sessions with the practice nurse or a prescription for smoking cessation medication. Although participants who received the Internet-based program only might have had the same tendency, the practice nurses for participants in the multiple tailoring and counseling group might have been prompted by their patients’ visit to offer them more smoking cessation help. Because the current smoking cessation guidelines in the Netherlands recommend more than 1 counseling session [44,58], this is not unlikely. Interestingly, though the significant difference in total health care-related costs could not be explained by a difference in any particular type of health care-related costs. For instance, no significant differences were found with regard to costs spent on general practitioner or practice nurse consultations or on the use of smoking cessation aids.

Furthermore, the present study showed that the Internet-based multiple computer-tailored smoking cessation program would probably be the most cost-effective of the 3 treatments under study. Although no similar studies yet exist within the field of smoking cessation, this finding is in-line with findings from recent studies toward the cost-effectiveness of Internet-based interventions aimed at other health-related behaviors or health problems [31,32]. Compared with current practice, the incremental costs per abstinent participant associated with the Internet-based multiple computer-tailored smoking cessation program were €5100. This is slightly more than what was found in previous studies [59,60]. However, one of these studies only included costs directly related to the interventions received [59], whereas we conducted our economic evaluation from a broader societal perspective. Nevertheless, the interpretation of the incremental costs per abstinent participant is difficult because no information exists on the amount of money that society is willing to pay per abstinent participant. Although a WTP of €18,000 per QALY is an accepted Dutch cutoff point [55], no such cutoff point exists with regard to smoking abstinence rates. To enable the interpretation of the incremental costs per abstinent participant, future research should aim at identifying an acceptable cutoff point for the WTP per abstinent participant.

Regarding cost-utilities, the results suggest that care as usual would probably be the most preferable of the treatments studied. A potential explanation for this finding might be that the follow-up period of 12 months was not sufficiently long for the beneficial effects of the intervention on smoking abstinence to be translated into detectable changes in quality of life, as recent ex-smokers are known to suffer from withdrawal symptoms [61]. A potential solution would be to use short-term trial data as input for a model predicting the effects of smoking cessation interventions on long-term quality of life, a technique used in several recent studies [28,29,60]. Although trial data served as input for these models, several assumptions had to be made to build these models, bringing about additional uncertainty in the results presented [34, 60]. Another solution would be to lengthen the follow-up period of clinical trials to gather longer-term data on quality of life. Although this might imply an increased burden on the participant, it might be needed to establish the intervention’s cost-utility in an as certain as possible way. However, as previous studies were able to detect a positive association between quitting smoking and quality of life during a 12-month follow-up period [62,63], additional explanations for this finding need to be sought. One such explanation might be that within Dutch general practices, care as usual for smoking cessation is rather intensive. Anecdotal evidence suggests that practice nurses usually offer 4 to 6 consultations as part of smoking cessation care (unpublished). Although participants in the multiple tailoring and multiple tailoring and counseling groups received 4 and 3 tailored feedback letters, respectively, a primarily Internet-based program might have been perceived as more distant and less intense than care as usual. As a result, participants in the usual care group might have established a better or stronger social bond with their practice nurse than participants in the multiple tailoring or multiple tailoring and counseling groups. This assumed social bond [64] might subsequently have resulted in positive effects on quality of life among usual care participants.

Strengths and Limitations

The present study aimed to contribute to the literature by examining the cost-effectiveness and cost-utility of an Internet-based smoking cessation intervention, something that has not been done before to our knowledge. In the present study, in addition to generic quality of life, disease-specific effects of the intervention (ie, smoking abstinence) were taken into account. To facilitate the comparison of the cost-effectiveness of interventions targeting different diseases, effects are usually assessed in terms of quality of life. However, to compare smoking cessation interventions more specifically, disease-specific effect measures might be more informative.

Nevertheless, the present study also had its limitations. First, it suffered from relatively high dropout rates. High rates of attrition seem to be inherent to many Internet-based interventions and dropout rates of 44% are not uncommon [24,65-67]. As a consequence, however, there was not sufficient power for us to conduct a complete-case analysis as part of the sensitivity analyses. Secondly, because we expected higher attrition rates in the multiple tailoring and counseling group, slightly more participants were randomized into this intervention group at baseline. However, attrition rates appeared to be similar among the groups, resulting in a skewed distribution of participants with 163 participants in the multiple tailoring and counseling group, 132 in the multiple tailoring group, and 119 in the usual care group. Although we do not expect that this has biased our results, this finding of no selective attrition can be considered valuable in the design of future trials.

Conclusions

The Internet-based multiple computer-tailored program seemed to be the most cost-effective treatment when smoking abstinence was the outcome measure. However, the cost-utility, using quality of life as the outcome measure, was probably the highest with care as usual. To enable the interpretation of the incremental costs per abstinent participant found in the cost-effectiveness analyses, future research should aim at identifying an acceptable cutoff point for the WTP per abstinent participant.

Acknowledgments

The study was funded by the Dutch Cancer Society (UM 2007-3834). We would like to thank Thea van Asselt for the additional assistance in conducting the cost-effectiveness and cost-utility analyses.

Abbreviations

CEA
cost-effectiveness analysis
CUA
cost-utility analysis
CEAC
cost-effectiveness acceptability curve
CUAC
cost-utility acceptability curve
FTND
Fagerström Test for Nicotine Dependence
ICER
incremental cost-effectiveness ratio
ICM
I-Change Model
ICUR
incremental cost-utility ratio
MTC
multiple tailoring and counseling
MT
multiple tailoring
NMB
net monetary benefit
OCT
over-the-counter
QALY
quality-adjusted life year
RCT
randomized controlled trial
UC
usual care
WTP
willingness to pay

Multimedia Appendix 1

Economic evaluation studies in a nutshell.

Multimedia Appendix 2

A tailored smoking cessation advice for a respondent who reported to still be smoking at six-month follow-up and whose self-efficacy to quit has decreased since baseline.

Multimedia Appendix 3

CONSORT-EHEALTH checklist V1.6.2 [68].

Footnotes

Conflicts of Interest:

Conflicts of Interest: Hein de Vries is scientific director of Vision2Health, a company that licenses evidence-based innovative computer-tailored health communication tools.

References

1. WHO Report on the Global Tobacco Epidemic, 2008: the MPOWER package. Geneva: World Health Organization; 2008. [2013-02-24]. webcite http://www.who.int/tobacco/mpower/mpower_report_full_2008.pdf.
2. Allender S, Balakrishnan R, Scarborough P, Webster P, Rayner M. The burden of smoking-related ill health in the UK. Tob Control. 2009 Aug;18(4):262–7. doi: 10.1136/tc.2008.026294. [PubMed] [Cross Ref]
3. Muennig P, Fiscella K, Tancredi D, Franks P. The relative health burden of selected social and behavioral risk factors in the United States: implications for policy. Am J Public Health. 2010 Sep;100(9):1758–64. doi: 10.2105/AJPH.2009.165019. [PMC free article] [PubMed] [Cross Ref]
4. Scarborough P, Bhatnagar P, Wickramasinghe KK, Allender S, Foster C, Rayner M. The economic burden of ill health due to diet, physical inactivity, smoking, alcohol and obesity in the UK: an update to 2006-07 NHS costs. J Public Health (Oxf) 2011 Dec;33(4):527–35. doi: 10.1093/pubmed/fdr033. http://jpubhealth.oxfordjournals.org/cgi/pmidlookup?view=long&pmid=21562029. [PubMed] [Cross Ref]
5. Lancaster T, Stead LF. Self-help interventions for smoking cessation. Cochrane Database Syst Rev. 2005;(3):CD001118. doi: 10.1002/14651858.CD001118.pub2. [PubMed] [Cross Ref]
6. Lancaster T, Stead LF. Individual behavioural counselling for smoking cessation. Cochrane Database Syst Rev. 2005;(2):CD001292. doi: 10.1002/14651858.CD001292.pub2. [PubMed] [Cross Ref]
7. Noar SM, Benac CN, Harris MS. Does tailoring matter? Meta-analytic review of tailored print health behavior change interventions. Psychol Bull. 2007 Jul;133(4):673–93. doi: 10.1037/0033-2909.133.4.673. [PubMed] [Cross Ref]
8. Stead LF, Bergson G, Lancaster T. Physician advice for smoking cessation. Cochrane Database Syst Rev. 2008;(2):CD000165. doi: 10.1002/14651858.CD000165.pub3. [PubMed] [Cross Ref]
9. Hall S, Vogt F, Marteau TM. A short report: survey of practice nurses' attitudes towards giving smoking cessation advice. Fam Pract. 2005 Dec;22(6):614–6. doi: 10.1093/fampra/cmi082. http://fampra.oxfordjournals.org/cgi/pmidlookup?view=long&pmid=16055470. [PubMed] [Cross Ref]
10. Vogt F, Hall S, Marteau TM. General practitioners' and family physicians' negative beliefs and attitudes towards discussing smoking cessation with patients: a systematic review. Addiction. 2005 Oct;100(10):1423–31. doi: 10.1111/j.1360-0443.2005.01221.x. [PubMed] [Cross Ref]
11. Borland R, Balmford J, Hunt D. The effectiveness of personally tailored computer-generated advice letters for smoking cessation. Addiction. 2004 Mar;99(3):369–77. doi: 10.1111/j.1360-0443.2003.00623.x. [PubMed] [Cross Ref]
12. Dijkstra A, De Vries H, Roijackers J. Long-term effectiveness of computer-generated tailored feedback in smoking cessation. Health Educ Res. 1998 Jun;13(2):207–14. http://her.oxfordjournals.org/cgi/pmidlookup?view=long&pmid=10181019. [PubMed]
13. Hoving C, Mudde AN, Dijk F, Vries HD. Effectiveness of a smoking cessation intervention in Dutch pharmacies and general practices. Health Education. 2010 Jun 2010;110(1):17–29. doi: 10.1108/09654281011008726. http://her.oxfordjournals.org/cgi/pmidlookup?view=long&pmid=10181019. [Cross Ref]
14. Strecher VJ. Computer-tailored smoking cessation materials: a review and discussion. Patient Educ Couns. 1999 Feb;36(2):107–17. [PubMed]
15. Te Poel F, Bolman C, Reubsaet A, de Vries H. Efficacy of a single computer-tailored e-mail for smoking cessation: results after 6 months. Health Educ Res. 2009 Dec;24(6):930–40. doi: 10.1093/her/cyp036. http://her.oxfordjournals.org/cgi/pmidlookup?view=long&pmid=19574405. [PubMed] [Cross Ref]
16. Rabius V, Pike KJ, Wiatrek D, McAlister AL. Comparing internet assistance for smoking cessation: 13-month follow-up of a six-arm randomized controlled trial. J Med Internet Res. 2008;10(5):e45. doi: 10.2196/jmir.1008. http://www.jmir.org/2008/5/e45/ [PMC free article] [PubMed] [Cross Ref]
17. de Vries H, Brug J. Computer-tailored interventions motivating people to adopt health promoting behaviours: introduction to a new approach. Patient Educ Couns. 1999 Feb;36(2):99–105. [PubMed]
18. van Keulen HM, Mesters I, Brug J, Ausems M, Campbell M, Resnicow K, Zwietering PJ, van Breukelen G, van Mechelen W, Severens JL, de Vries H. Vitalum study design: RCT evaluating the efficacy of tailored print communication and telephone motivational interviewing on multiple health behaviors. BMC Public Health. 2008;8:216. doi: 10.1186/1471-2458-8-216. http://www.biomedcentral.com/1471-2458/8/216. [PMC free article] [PubMed] [Cross Ref]
19. van Stralen MM, Kok G, de Vries H, Mudde AN, Bolman C, Lechner L. The Active plus protocol: systematic development of two theory- and evidence-based tailored physical activity interventions for the over-fifties. BMC Public Health. 2008;8:399. doi: 10.1186/1471-2458-8-399. http://www.biomedcentral.com/1471-2458/8/399. [PMC free article] [PubMed] [Cross Ref]
20. Kreuter M. Tailoring Health Messages: Customizing Communication with Computer Technology. Mahwah, NJ: Lawrence Erlbaum Associates; 2000. The Active plus protocol: systematic development of two theory- and evidence-based tailored physical activity interventions for the over-fifties. [PMC free article] [PubMed]
21. Dijkstra A. Working mechanisms of computer-tailored health education: evidence from smoking cessation. Health Educ Res. 2005 Oct;20(5):527–39. doi: 10.1093/her/cyh014. http://her.oxfordjournals.org/cgi/pmidlookup?view=long&pmid=15701665. [PubMed] [Cross Ref]
22. Dijkstra A, De Vries H, Roijackers J, van Breukelen G. Tailoring information to enhance quitting in smokers with low motivation to quit: three basic efficacy questions. Health Psychol. 1998 Nov;17(6):513–9. [PubMed]
23. Lustria ML, Cortese J, Noar SM, Glueckauf RL. Computer-tailored health interventions delivered over the Web: review and analysis of key components. Patient Educ Couns. 2009 Feb;74(2):156–73. doi: 10.1016/j.pec.2008.08.023. [PubMed] [Cross Ref]
24. Shahab L, McEwen A. Online support for smoking cessation: a systematic review of the literature. Addiction. 2009 Nov;104(11):1792–804. doi: 10.1111/j.1360-0443.2009.02710.x. [PubMed] [Cross Ref]
25. Smit ES, de Vries H, Hoving C. The PAS study: a randomized controlled trial evaluating the effectiveness of a web-based multiple tailored smoking cessation programme and tailored counselling by practice nurses. Contemp Clin Trials. 2010 May;31(3):251–8. doi: 10.1016/j.cct.2010.03.001. [PubMed] [Cross Ref]
26. Annemans L, Nackaerts K, Bartsch P, Prignot J, Marbaix S. Cost effectiveness of varenicline in Belgium, compared with bupropion, nicotine replacement therapy, brief counselling and unaided smoking cessation: a BENESCO Markov cost-effectiveness analysis. Clin Drug Investig. 2009;29(10):655–65. doi: 10.2165/11317730-000000000-00000. [PubMed] [Cross Ref]
27. Hoogendoorn M, Feenstra TL, Hoogenveen RT, Rutten-van Mölken MP. Long-term effectiveness and cost-effectiveness of smoking cessation interventions in patients with COPD. Thorax. 2010 Aug;65(8):711–8. doi: 10.1136/thx.2009.131631. [PubMed] [Cross Ref]
28. Vemer P, Rutten-van Mölken MP, Kaper J, Hoogenveen RT, van Schayck CP, Feenstra TL. If you try to stop smoking, should we pay for it? The cost-utility of reimbursing smoking cessation support in the Netherlands. Addiction. 2010 Jun;105(6):1088–97. doi: 10.1111/j.1360-0443.2010.02901.x. [PubMed] [Cross Ref]
29. Smith MY, Cromwell J, DePue J, Spring B, Redd W, Unrod M. Determining the cost-effectiveness of a computer-based smoking cessation intervention in primary care. Manag Care. 2007 Jul;16(7):48–55. [PubMed]
30. Ahern DK, Kreslake JM, Phalen JM. What is eHealth (6): perspectives on the evolution of eHealth research. J Med Internet Res. 2006;8(1):e4. doi: 10.2196/jmir.8.1.e4. http://www.jmir.org/2006/1/e4/ [PMC free article] [PubMed] [Cross Ref]
31. Smit F, Lokkerbol J, Riper H, Majo MC, Boon B, Blankers M. Modeling the cost-effectiveness of health care systems for alcohol use disorders: how implementation of eHealth interventions improves cost-effectiveness. J Med Internet Res. 2011;13(3):e56. doi: 10.2196/jmir.1694. http://www.jmir.org/2011/3/e56/ [PMC free article] [PubMed] [Cross Ref]
32. Warmerdam L, Smit F, van Straten A, Riper H, Cuijpers P. Cost-utility and cost-effectiveness of internet-based treatment for adults with depressive symptoms: randomized trial. J Med Internet Res. 2010;12(5):e53. doi: 10.2196/jmir.1436. http://www.jmir.org/2010/5/e53/ [PMC free article] [PubMed] [Cross Ref]
33. Gerhards SA, de Graaf LE, Jacobs LE, Severens JL, Huibers MJ, Arntz A, Riper H, Widdershoven G, Metsemakers JF, Evers SM. Economic evaluation of online computerised cognitive-behavioural therapy without support for depression in primary care: randomised trial. Br J Psychiatry. 2010 Apr;196(4):310–8. doi: 10.1192/bjp.bp.109.065748. http://bjp.rcpsych.org/cgi/pmidlookup?view=long&pmid=20357309. [PubMed] [Cross Ref]
34. Drummond MF, Sculpher MJ, Torrance G, O'Brien B, Stoddart G. Methods for the economic evaluation of health care programmes. Oxford: Oxford University Press; 2005.
35. Evers SM, Wolf C, van Heugten C. Neuropsychologische Behandeling. Amsterdam: Boom Uitgeverij; 2010. Economische evaluatie van neuropsychologische behandeling; pp. 115–140.
36. de Vries H, Mudde AN, Leijs I, Charlton A, Vartiainen E, Buijs G, Clemente MP, Storm H, González Navarro A, Nebot M, Prins T, Kremers S. The European Smoking Prevention Framework Approach (EFSA): an example of integral prevention. Health Educ Res. 2003 Oct;18(5):611–26. http://her.oxfordjournals.org/cgi/pmidlookup?view=long&pmid=14572020. [PubMed]
37. de Vries HD, Mudde AN. Predicting stage transitions for smoking cessation applying the attitude-social influence-efficacy model. Psychology & Health. 1998 Apr 1998;13(2):369–385. doi: 10.1080/08870449808406757. [Cross Ref]
38. Prochaska JO, Redding CA, Evers KE. The transtheoretical model and stages of change. In: Glanz K, editor. Health Behavior and Health Education: Theory, Research, and Practice. San Francisco: Jossey-Bass; 1997. pp. 99–120.
39. Ajzen I. Action-Control: From Cognition to Behavior. Heidelberg: Springer; 1985. From intentions to actions: a theory of Planned Behaviour; pp. 11–39.
40. Bandura A. Social Foundations of Thought and Action: A Social Cognitive Theory. Englewood Cliffs, NJ: Prentice-Hall; 1986.
41. Janz NK, Champion VL, Strecher VJ. The health belief model. In: Glanz K, editor. Health Behavior and Health Education: Theory, Research, and Practice. San Francisco: Jossey-Bass; 2002. pp. 45–66.
42. Armitage CJ, Conner M. Efficacy of the Theory of Planned Behaviour: a meta-analytic review. Br J Soc Psychol. 2001 Dec;40(Pt 4):471–99. [PubMed]
43. Chavannes NH, Kaper J, Frijling BD, Van der Laan JR, Jansen PWM, Guerrouj S, Drenthen AJM, Bax W, Wind LA. NHG-standaarden voor de huisarts. Houten: Bohn Stafleu van Loghum; 2009. NHG-Standaard Stoppen met roken; pp. 1434–1449.
44. Partnership Stop met Roken . Richtlijn Behandeling van Tabaksverslaving herziening 2009. Den Haag: Partnership Stop met Roken; 2009. [2013-02-26]. webcite http://www.partnershipstopmetroken.nl/sup/RichtlijnTabaksverslavingActualisatie2009definitief.pdf.
45. Hakkaart-van Roijen L, Tan SS, Bouwmans CAM. Handleiding voor kostenonderzoek: Methoden en standaard kostprijzen voor economische evaluaties in de gezondheidszorg. Geactualiseerde versie 2010. 2010. [2012-12-20]. webcite http://www.cvz.nl/binaries/live/cvzinternet/hst_content/nl/documenten/losse-publicaties/handleiding-kostenonderzoek-2010.pdf.
46. Commissie Farmacotherapautische Hulp CVZ . Farmacotherapeutisch Kompas: medisch farmaceutische voorlichting. Amstelveen: College voor zorgverzekeringen; 2011.
47. Centraal Bureau voor de Statistiek Consumentenprijzen; prijsindex 2006. 2011. [2012-06-06]. webcite http://www.statline.nl.
48. Heatherton TF, Kozlowski LT, Frecker RC, Fagerström KO. The Fagerström Test for Nicotine Dependence: a revision of the Fagerström Tolerance Questionnaire. Br J Addict. 1991 Sep;86(9):1119–27. [PubMed]
49. Craig Medical Distribution Inc. [2012-09-14]. webcite NicAlert saliva nicotine test http://www.craigmedical.com/Nicotine_Saliva_tests.htm.
50. The EuroQol Group EuroQol--a new facility for the measurement of health-related quality of life. Health Policy. 1990 Dec;16(3):199–208. [PubMed]
51. Sculpher M. NICE's 2008 Methods Guide: sensible consolidation or opportunities missed? Pharmacoeconomics. 2008;26(9):721–4. [PubMed]
52. Hoogendoorn M, Welsing P, Rutten-van Mölken MP. Cost-effectiveness of varenicline compared with bupropion, NRT, and nortriptyline for smoking cessation in the Netherlands. Curr Med Res Opin. 2008 Jan;24(1):51–61. doi: 10.1185/030079908X242917. [PubMed] [Cross Ref]
53. Hoogwegt MT, Hoeks SE, Pedersen SS, Scholte op Reimer WJ, van Gestel YR, Verhagen HJ, Poldermans D. Smoking cessation has no influence on quality of life in patients with peripheral arterial disease 5 years post-vascular surgery. Eur J Vasc Endovasc Surg. 2010 Sep;40(3):355–62. doi: 10.1016/j.ejvs.2010.05.013. [PubMed] [Cross Ref]
54. Stinnett AA, Mullahy J. Net health benefits: a new framework for the analysis of uncertainty in cost-effectiveness analysis. Med Decis Making. 1998;18(2 Suppl):S68–80. [PubMed]
55. Casparie AF, van Hout BA, Simoons ML. [Guidelines and costs] Ned Tijdschr Geneeskd. 1998 Sep 19;142(38):2075–7. [PubMed]
56. Glick HA, Doshi JA, Sonnad SS, Polsky D. Economic Evaluation in Clinical Trials. Oxford: Oxford University Press; 2007.
57. Raad voor de Volksgezondheid en Zorg Zoetermeer. 2006. [2012-12-20]. webcite Zinnige en duurzame zorg http://www.rvz.net/uploads/docs/Advies_-_Zinnige_en_duurzame_zorg.pdf.
58. Pieterse ME, Seydel ER, DeVries H, Mudde AN, Kok GJ. Effectiveness of a minimal contact smoking cessation program for Dutch general practitioners: a randomized controlled trial. Prev Med. 2001 Feb;32(2):182–90. doi: 10.1006/pmed.2000.0791. [PubMed] [Cross Ref]
59. Salize HJ, Merkel S, Reinhard I, Twardella D, Mann K, Brenner H. Cost-effective primary care-based strategies to improve smoking cessation: more value for money. Arch Intern Med. 2009 Feb 9;169(3):230–5; discussion 235. doi: 10.1001/archinternmed.2008.556. [PubMed] [Cross Ref]
60. Kaper J, Wagena EJ, van Schayck CP, Severens JL. Encouraging smokers to quit: the cost effectiveness of reimbursing the costs of smoking cessation treatment. Pharmacoeconomics. 2006;24(5):453–64. [PubMed]
61. Shiffman S, Patten C, Gwaltney C, Paty J, Gnys M, Kassel J, Hickcox M, Waters A, Balabanis M. Natural history of nicotine withdrawal. Addiction. 2006 Dec;101(12):1822–32. doi: 10.1111/j.1360-0443.2006.01635.x. [PubMed] [Cross Ref]
62. Hays JT, Croghan IT, Baker CL, Cappelleri JC, Bushmakin AG. Changes in health-related quality of life with smoking cessation treatment. Eur J Public Health. 2012 Apr;22(2):224–9. doi: 10.1093/eurpub/ckq137. [PubMed] [Cross Ref]
63. Sales MP, Oliveira MI, Mattos IM, Viana CM, Pereira ED. The impact of smoking cessation on patient quality of life. J Bras Pneumol. 2009 May;35(5):436–41. http://www.scielo.br/scielo.php?script=sci_arttext&pid=S1806-37132009000500008&lng=en&nrm=iso&tlng=en. [PubMed]
64. Shum C, Humphreys A, Wheeler D, Cochrane MA, Skoda S, Clement S. Nurse management of patients with minor illnesses in general practice: multicentre, randomised controlled trial. BMJ. 2000 Apr 15;320(7241):1038–43. http://europepmc.org/abstract/MED/10764365. [PMC free article] [PubMed]
65. McKay HG, Danaher BG, Seeley JR, Lichtenstein E, Gau JM. Comparing two web-based smoking cessation programs: randomized controlled trial. J Med Internet Res. 2008;10(5):e40. doi: 10.2196/jmir.993. http://www.jmir.org/2008/5/e40/ [PMC free article] [PubMed] [Cross Ref]
66. Wangberg SC, Bergmo TS, Johnsen JA. Adherence in Internet-based interventions. Patient Prefer Adherence. 2008;2:57–65. http://www.dovepress.com/articles.php?article_id=2062. [PMC free article] [PubMed]
67. Wangberg SC, Nilsen O, Antypas K, Gram IT. Effect of tailoring in an internet-based intervention for smoking cessation: randomized controlled trial. J Med Internet Res. 2011;13(4):e121. doi: 10.2196/jmir.1605. http://www.jmir.org/2011/4/e121/ [PMC free article] [PubMed] [Cross Ref]
68. Eysenbach G, CONSORT-EHEALTH Group CONSORT-EHEALTH: improving and standardizing evaluation reports of Web-based and mobile health interventions. J Med Internet Res. 2011;13(4):e126. doi: 10.2196/jmir.1923. http://www.jmir.org/2011/4/e126/ [PMC free article] [PubMed] [Cross Ref]

Articles from Journal of Medical Internet Research are provided here courtesy of Gunther Eysenbach
PubReader format: click here to try

Formats:

Related citations in PubMed

See reviews...See all...

Cited by other articles in PMC

See all...

Links

  • PubMed
    PubMed
    PubMed citations for these articles

Recent Activity

    Your browsing activity is empty.

    Activity recording is turned off.

    Turn recording back on

    See more...