PubMed Health. A service of the National Library of Medicine, National Institutes of Health.

New Oral Anticoagulants for the Prevention of Thromboembolic Events in Patients with Atrial Fibrillation [Internet]. Ottawa (ON): Canadian Agency for Drugs and Technologies in Health; 2012.

7.6Risk of Bias Assessment Tool

George Wells, PhD, Doug Coyle, PhD, Chris Cameron, MSc PhD (candidate), Sabine Steiner, MD MSc, Kathryn Coyle, BScPharm MSc, Shannon Kelly, MSc (candidate), Anthony Tang, MD FRCPC, Jeff Healey, MD FRCPC, Shu-Ching Hsieh, PhD, and Judith van Berkom, BA.

Author Information

George Wells, PhD,1 Doug Coyle, PhD,2 Chris Cameron, MSc PhD (candidate),1 Sabine Steiner, MD MSc,3 Kathryn Coyle, BScPharm MSc,4 Shannon Kelly, MSc (candidate),1 Anthony Tang, MD FRCPC,5 Jeff Healey, MD FRCPC,6 Shu-Ching Hsieh, PhD,1 and Judith van Berkom, BA7.

1 University of Ottawa Heart Institute, Ottawa, Ontario
2 University of Ottawa, Ottawa, Ontario
3 Medical University of Vienna, Vienna, Austria
4 Coyle Consultancy, Ottawa, Ontario
5 University of British Columbia, Victoria, British Columbia
6 McMaster University, Hamilton, Ontario
7 Independent contractor: Information Specialist, Ottawa, Ontario

Table 40ROB Assessment Tool

ItemReviewer
Assessment
Description
Adequate sequence generation?
Allocation concealment?
Blinding of objective outcomes assessment?
Blinding of subjective outcomes assessment?
Incomplete outcomes data addressed? Efficacy outcomes
Incomplete outcomes data addressed? Safety outcomes
Free of selective reporting?
Free of other bias?

7.6.1. Risk of Bias Assessment for Included Studies (n=5)

RE-LY: Connolly 20099
ItemReviewer
Assessment
Description
Adequate sequence generation?Unclear“…were randomly assigned to receive…”
Method for sequence generation not provided.
Allocation concealment?UnclearMethod for allocation concealment not provided.
Blinding of objectives outcomes assessment?LowDabigatran was administered in a blinded fashion …Warfarin was administered in an unblinded fashion…” “Each primary and secondary outcome event was adjudicated by two independent investigators who were unaware of the treatment assignments. All transient ischemic attacks were reviewed to ensure that strokes had not been missed. To detect possible unreported events, symptom questionnaires were regularly administered to patients, and adverse event and hospitalization reports were scrutinized for unreported primary or secondary outcomes.”
A blinding approach was not provided, and blinding for the warfarin group was infeasible, judged a low risk of bias given that the objective outcomes should be based on participants’ signs and symptoms, radiographic and/or laboratory evidence, and objective criteria, which were unlikely to be influenced by incomplete or ineffective blinding.
Blinding of subjective outcomes assessment?High“Dabigatran was administered, in a blinded fashion.… Warfarin was administered, in an unblinded fashion…” “To detect possible unreported events, symptom questionnaires were regularly administered to patients…”
A blinding approach was not provided, and blinding for the warfarin group was infeasible, judged a high risk of bias given that the subjective outcomes might probably be influenced by the participants' or assessors' knowledge of the allocated interventions following assignment if the blinding was incomplete or ineffective.
Incomplete outcomes data addressed? Efficacy outcomesUnclear“All analyses were based on the intention-to-treat principle.” “Two protocol changes were made by the operations committee during the enrollment period, without knowledge of emerging treatment effects.” “The median duration of the follow-up period was 2.0 years” “The rates of discontinuation for 110 mg of dabigatran, 150 mg of dabigatran, and warfarin were 14.5%, 15.5%, and 10.2%, respectively, at one year, and 20.7%, 21.2%, and 16.6% at two years.”
All randomized and included participants were included in the analysis. However, they were judged an uncertain risk of bias given that the overall completion rate at two years (the median duration of the follow-up period) was 82% (14,839/18,113), with comparatively more patients in dabigatran groups (19% and 20%) than in the warfarin group (15%) discontinuing the study. As well, the approach to handling missing data was not provided.
Incomplete outcomes data addressed? Safety outcomesUnclear“All analyses were based on the intention-to-treat principle.” “Two protocol changes were made by the operations committee during the enrollment period, without knowledge of emerging treatment effects.” “The median duration of the follow-up period was 2.0 years” “The rates of discontinuation for 110 mg of dabigatran, 150 mg of dabigatran, and warfarin were 14.5%, 15.5%, and 10.2%, respectively, at one year and 20.7%, 21.2%, and 16.6% at two years.”
However, they were judged an uncertain risk of bias given that the overall completion rate at two years (the median duration of the follow-up period) was 82% (14,839/18,113), with comparatively more patients in dabigatran groups (19% and 20%) than in the warfarin group (15%) discontinuing the study. As well, the approach to handling missing data was not provided.
Free of selective reporting?LowJudged a low risk of bias given that outcomes of interest were pre-specified in the protocol (NCT00262600 in ClinicalTrials​.gov) and reported in the published paper.
Free of other bias?High“Supported by a grant from Boehringer Ingelheim.” Three among 21 authors were employed by “Boehringer Ingelheim Pharmaceuticals, Ridgefield, CT (P.A.R., J.V., S.W.).” Some authors received grant support and fees by providing consultations and lectures.
PETRO: Ezekowits 200710
ItemReviewer
Assessment
Description
Adequate sequence generation?Unclear“Randomization was stratified in the ratio 6:9:9:4.”
Method for sequence generation not provided.
Allocation concealment?UnclearMethod for allocation concealment not provided.
Blinding of objective outcomes assessment?Low“The trial was double-blind with respect to dabigatran dose but open-label for concomitant aspirin treatment, and for randomization between dabigatran and warfarin groups.” “Two identical matching capsules containing 50 mg or 150 mg dabigatran or placebo were taken twice daily for 12 weeks.”
Blinding of subjective outcomes assessment?low“The trial was double-blind with respect to dabigatran dose, but open-label for concomitant aspirin treatment, and for randomization between dabigatran and warfarin groups.” “Two identical matching capsules containing 50 mg or 15 mg dabigatran or placebo were taken twice daily for 12 weeks.”
Incomplete outcomes data addressed? Efficacy outcomesUnclearAll randomized participants were included in the analysis, in which 92.4% (464/502) completed the 12-week study, with 91.7% (396/432) in the dabigatran groups and 97.1% (68/70) in the warfarin group. They were judged a low risk of bias given that the completion rates in each group were quite high, and the reported numbers and reasons of discontinuations in each group seemed unlikely to bias the outcome estimate.
Incomplete outcomes data addressed? Safety outcomesUnclearAll randomized participants were included in the analysis, in which 92.4% (464/502) completed the 12-week study, with 91.7% (396/432) in the dabigatran groups and 97.1% (2/70) in the warfarin group. They were judged a low risk of bias given that the completion rates in each group were quite high and the reported numbers and reasons of discontinuations in each group seemed unlikely to bias the outcome estimate.
Free of selective reporting?LowThey were judged a low risk of bias given that outcomes of interest were pre-specified in the protocol (NCT01227629 in ClinicalTrials​.gov) and reported in the published paper.
Free of other bias?High“Boehringer Ingelheim Pharmaceuticals, Biberach, Germany, is the sponsor of this study and has provided a research grant.” Two among ten authors were employed by “Departments of Medical Data Services and Drug Metabolism and Pharmacokinetics, Boehringer Ingelheim Pharmaceuticals, Biberach, Germany.”
ARISTOTLE: Granger 201111
ItemReviewer
Assessment
Description
Adequate sequence generation?Unclear“… we randomly assigned patients to treatment with apixaban or dose-adjusted warfarin.”
Method for sequence generation not provided.
Allocation concealment?UnclearMethod for allocation concealment not provided.
Blinding of objectives outcomes assessment?Low“…a double-blind, double-dummy design…”
“To maintain blinding, study medications are packaged using a double-dummy design. The two sets of tablets each subject receives are distinguishable by colour and size, but active apixaban tablets match placebo apixaban tablets and active warfarin tablets match placebo warfarin tablets to ensure blinding of the patient and investigator. After randomization, patients receive either apixaban and warfarin placebo or apixaban placebo and warfarin.” “Subjects, investigators, members of the steering and adjudication committees, and the sponsor's staff conducting the study do not have access to individual subject treatment assignments.”
Blinding of subjective outcomes assessment?Low“…a double-blind, double-dummy design…”
“To maintain blinding, study medications are packaged using a double-dummy design. The two sets of tablets each subject receives are distinguishable by colour and size, but active apixaban tablets match placebo apixaban tablets and active warfarin tablets match placebo warfarin tablets to ensure blinding of the patient and investigator. After randomization, patients receive either apixaban and warfarin placebo or apixaban placebo and warfarin.” “Subjects, investigators, members of the steering and adjudication committees, and the sponsor's staff conducting the study do not have access to individual subject treatment assignments.”
Incomplete outcomes data addressed? Efficacy outcomesHigh“The primary and secondary efficacy analyses included all patients who underwent randomization (intention-to-treat population) and included all events from the time of randomization until the cut-off date for efficacy outcomes (predefined as January 30, 2011).”
“Fewer patients in the apixaban group than in the warfarin group discontinued a study drug before the end of the study: 25.3% of the patients in the apixaban group, with 3.6% of the discontinuations due to death, versus 27.5% of patients in the warfarin group, with 3.8% due to death (P = 0.001).”
All randomized participants were included in the efficacy analysis, judged a high risk of bias given that the overall completion rate was less than 80%, and the approach to handling missing data was not provided.
Incomplete outcomes data addressed? Safety outcomesHigh“The analyses of bleeding events included all patients who received at least one dose of a study drug and included all events from the time the first dose of a study drug was received until two days after the last dose was received.”
“Fewer patients in the apixaban group than in the warfarin group discontinued a study drug before the end of the study: 25.3% of the patients in the apixaban group, with 3.6% of the discontinuations due to death, versus 27.5% of patients in the warfarin group, with 3.8% due to death (P = 0.001).”
99.7% (18,140/18,201) of the randomized participants were included in the bleeding analysis (primary safety outcome) and other safety outcome analysis, with 32 in the apixaban group and 29 in the warfarin group not taking the medication; they were judged a high risk of bias given that the overall completion rate was less than 80%, and the approach to handling missing data was not provided.
Free of selective reporting?LowThey were judged a low risk of bias given that outcomes of interest were pre-specified in the protocol (NCT00412984 in ClinicalTrials​.gov) and reported in the published paper.
Free of other bias?High“Supported by Bristol-Myers Squibb and Pfizer.” Three among 32 authors were employed by “Bristol-Myers Squibb, Princeton, New Jersey (M.H., M.G., P.M.)”. Most of the authors claimed relationship with pharmaceutical companies in the form of receiving grants and providing education and consultancy.
ARISTOTLE-J: Ogawa 201112
ItemReviewer
Assessment
Description
Adequate sequence generation?Unclear“…patients were randomized in a 1:1:1 fashion to receive…”
Method for sequence generation not provided.
Allocation concealment?UnclearMethod for allocation concealment not provided.
Blinding of objective outcomes assessment?Low“…two double-blinded doses of apixaban with open-label warfarin…” Blinding approach not provided; judged a low risk of bias given that the objective outcomes were based on radiographic and laboratory evidences (central laboratories) and objective criteria, which were unlikely to be influenced by ineffective or incomplete blinding.
Blinding of subjective outcomes assessment?High“…two double-blinded doses of apixaban with open-label warfarin…” Blinding approach not provided; judged an uncertain risk of bias given that the subjective or minor outcomes might probably be influenced by the participants' or assessors' knowledge of the allocated interventions following assignment if the blinding was ineffective or incomplete.
Incomplete outcomes data addressed? Efficacy outcomesLow“Efficacy determinations were based on the intent-to-treat population, which included all randomized patients….Analyses of efficacy end points were based on the “intended treatment period,” defined as starting on the day of randomization and ending either two days after the last dose of study drug or at the week 12 visit, whichever came last.” “In our smaller and shorter duration phase II study, we observed no imbalance in discontinuations and no MI [myocardial infarction].” All randomized participants were included in the analysis, including four not taking medications and two being treated in error (n = 222). 91%, 93%, and 88% of the patients in the apixaban-2.5, apixaban-5, and warfarin groups completed the 12-week study, respectively. They were judged a low risk of bias given that the completion rates in each group were quite high, and the reported numbers and reasons of discontinuations in each group seemed unlikely to bias the outcome estimate.
Incomplete outcome data addressed? Safety outcomesLow“The safety population comprised all randomized patients who received at least one dose of the study drug.
All randomized participants were included in the analysis, except for four not taking medications (n = 218). 91%, 93%, and 88%, of the patients in the apixaban-2.5, apixaban-5, and warfarin groups completed the 12-week study, respectively. They were judged a low risk of bias given that the completion rates in each group were quite high, and the reported numbers and reasons of discontinuations in each group seemed unlikely to bias the outcome sestimate.
Free of selective reporting?LowJudged a low risk of bias given that outcomes of interest were pre-specified in the protocol (NCT00787150 in ClinicalTrials​.gov) and reported in the published paper.
Free of other bias?High“This study was funded by Pfizer Inc. and Bristol-Myers Squibb.” One of three authors was employed by “Cardiovascular and Metabolism Therapeutics, Pfizer Japan Inc., Tokyo (K.K.), Japan.” The other two authors also related to the funders by providing consultancy.
ROCKET-AF: Patel 201113
ItemReviewer
Assessment
Description
Adequate sequence generation?Low“Randomization was performed with the use of a central 24-hour, computerized, automated voice-response system.”
Allocation concealment?UncertainMethod for allocation concealment not provided.
Blinding of objective outcomes assessment?Low“Patients in each group also received a placebo tablet in order to maintain blinding.” “…sham values (for patients in the rivaroxaban group receiving placebo warfarin) during the course of the trial.”
Blinding of subjective outcomes assessment?Low“Patients in each group also received a placebo tablet in order to maintain blinding.” “…sham values (for patients in the rivaroxaban group receiving placebo warfarin) during the course of the trial.”
Incomplete outcomes data addressed? Efficacy outcomesHigh“The primary analysis was pre-specified to be performed in the per-protocol population, which included all patients who received at least one dose of a study drug, did not have a major protocol violation, and were followed for events while receiving a study drug or within two days after discontinuation….If non-inferiority was achieved in the primary analysis, a closed testing procedure was to be conducted for superiority in the safety population during treatment, which included patients who received at least one dose of a study drug and were followed for events, regardless of adherence to the protocol, while they were receiving the assigned study drug or within two days after discontinuation…. Testing for non-inferiority and superiority was also performed in the intention-to-treat population, which included all patients who underwent randomization and were followed for events during treatment or after premature discontinuation.”
97.6%, 99.0%, and 99.3% of randomized populations in the rivaroxaban group and 98.2%, 99.3%, and 99.4% of those in the warfarin group were included in the analysis of per-protocol, safety on treatment and the intention-to-treat population, respectively. 76.0 % (5,422/7,131) and 77.6% (5,535/7,133) of patients in the rivaroxaban and warfarin groups completed the study, respectively. They were judged a high risk of bias given that the completion rates were less than 80% and the approach to handling missing data was not provided.
Incomplete outcomes data addressed? Safety outcomesHigh“The primary analysis was prespecified to be performed in the per-protocol population, which included all patients who received at least one dose of a study drug, did not have a major protocol violation, and were followed for events while receiving a study drug or within two days after discontinuation….If non-inferiority was achieved in the primary analysis, a closed testing procedure was to be conducted for superiority in the safety population during treatment, which included patients who received at least one dose of a study drug and were followed for events, regardless of adherence to the protocol, while they were receiving the assigned study drug or within two days after discontinuation…. Testing for noninferiority and superiority was also performed in the intention-to-treat population, which included all patients who underwent randomization and were followed for events during treatment or after premature discontinuation.”
97.6%, 99.0%, and 99.3% of the randomized population in the rivaroxaban group and 98.2%, 99.3%, and 99.4% of those in the warfarin group were included in the analysis of per-protocol, safety on treatment and the intention-to-treat population, respectively. 76.0 % (5,422/7,131) and 77.6% (5,535/7,133) of patients in the rivaroxaban and warfarin groups completed the study, respectively. They were judged a high risk of bias given that the completion rates were less than 80% and the approach to handling missing data was not provided.
Free of selective reporting?LowThey were judged a low risk of bias given that outcomes of interest were pre-specified in the protocol (NCT00403767 in ClinicalTrials​.gov) and reported in the published paper.
Free of other bias?High“Supported by Johnson & Johnson Pharmaceutical Research and Development and Bayer HealthCare.” Two among 16 authors were employed by “Bayer HealthCare Pharmaceuticals, Montville (J.F.P., S.D.B.).”

Authorship

George Wells wrote the protocol for this review and acted as the primary lead in the conception, design and implementation of the project, as well as all aspects of the development and writing of the report and responses to internal and external reviewers.

Doug Coyle planned and led the economic section, selected articles, and extracted data for the economic review; performed the primary economic analyses; wrote the economic sections; and contributed to revisions and responses to internal and external reviewers.

Chris Cameron extracted and checked data; performed Bayesian mixed treatment comparisons metaanalyses; wrote sections and prepared tables and figures for the clinical review; and contributed to revisions and responses to internal and external reviewers.

Sabine Steiner extracted and checked data; performed frequentist mixed treatment comparisons metaanalyses, wrote sections of the clinical review, and contributed to revisions and responses to internal and external reviewers.

Kathryn Coyle assisted with the development of the methods for the economic analysis; selected economic studies; extracted, tabulated, and analyzed data; and contributed to writing the economic sections of the report.

Shannon Kelly contributed to writing the protocol; led and participated in article selection, study quality assessment, data extraction and tabulation, subgroup appraisal and preparation of tables for the clinical review; wrote and edited sections of the clinical review; contributed to revisions and responses to internal and external reviewers; and edited the final draft.

Anthony Tang provided clinical guidance and feedback for this review; co-wrote the Introduction and Background section in the protocol and this review; assisted with the writing of the Discussion section; and contributed to revisions and responses to external reviewers.

Jeff Healey provided clinical guidance and feedback for this review; co-wrote the Introduction and Background section in the protocol and this review; assisted with the writing of the Discussion section; and contributed to revisions and responses to external reviewers.

Shu-Ching Hseih participated in article selection, study quality assessment, data extraction and checking, subgroup appraisal and preparation of tables for the clinical review; assisted with referencing of studies in the final draft; and contributed to revisions.

Judith van Berkom developed the literature search strategies and performed all searches, managed report referencing, and composed the search methodology sections.

Copyright © 2012 CADTH.
PubReader format: click here to try
Cover of New Oral Anticoagulants for the Prevention of Thromboembolic Events in Patients with Atrial Fibrillation
New Oral Anticoagulants for the Prevention of Thromboembolic Events in Patients with Atrial Fibrillation [Internet].

PubMed Health Blog...

read all...

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...