• We are sorry, but NCBI web applications do not support your browser and may not function properly. More information
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptNIH Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
J Off Stat. Author manuscript; available in PMC May 18, 2011.
Published in final edited form as:
J Off Stat. 2008; 24(2): 255–275.
PMCID: PMC3096944
NIHMSID: NIHMS269709

Risk of Disclosure, Perceptions of Risk, and Concerns about Privacy and Confidentiality as Factors in Survey Participation

Abstract

This article reports on a web-based vignette experiment investigating how likely subjects would be to participate in surveys varying in topic sensitivity and risk of disclosure. A total of 3,672 participants each responded to a series of eight vignettes, along with a variety of background questions, concerns about confidentiality, trust in various institutions, and the like.

Vignettes were randomly assigned to respondents, such that each respondent was exposed to four levels of disclosure risk for each level of topic sensitivity (high versus low). Half the sample was assigned to receive a confidentiality statement for all eight vignettes, while the other half received no mention of confidentiality in the vignettes. The order of presentation of vignettes was randomized for each respondent.

Respondents were also asked for their subjective perceptions of risk, harm, and social as well as personal benefits for one of the eight vignettes. Adding these questions permits us to examine how objective risk information presented by the researcher relates to the subjective perception of risk by the participant, and to assess the importance of both for their willingness to participate in the surveys described.

Under conditions resembling those of real surveys, objective risk information does not affect willingness to participate. On the other hand, topic sensitivity does have such effects, as do general attitudes toward privacy and survey organizations as well as subjective perceptions of risk, harm, and benefits. We discuss the limitations and implications of these findings.

Keywords: Disclosure, informed consent, survey participation, privacy, confidentiality

1. Introduction

Concern about protecting the confidentiality of personal information disclosed to researchers, including health information, has led to increasing interest in what has come to be called statistical disclosure limitation (e.g., Lambert 1993; Fuller 1993; Jabine 1993; Rubin 1993; Fienberg and Willenborg 1998; Doyle et al. 2001; Raghunathan, Reiter, and Rubin 2003). “Statistical disclosure” refers to the ability to deduce an individual’s identity despite the absence of personal identifiers such as name and address on the data file, through a process of matching an individual’s de-identified record against another record containing (some of) the same characteristics as the original file in addition to the person’s name and address. Although successful matches have been reported (Paass 1988; Winkler 1997; Malin and Sweeney 2000), little is known about the likelihood of such disclosures from publicly available data files.

The research reported in the present article is part of a research program aiming to estimate the risk (probability) of statistical disclosure in publicly available data sets and to develop procedures to reduce that risk. Our specific aim was to use this information about disclosure risk – which we expected to be very, very low – to craft informed consent statements that would meet ethical requirements while, at the same time, reassuring respondents about the confidentiality of their personal information and thus encouraging them to participate.

Several studies have shown that concerns about privacy and confidentiality reduce participation in surveys, specifically the U.S. decennial censuses of 1980, 1990, and 2000 (Fay, Bates, and Moore 1991; Singer, Mathiowetz, and Couper 1993; Singer, Van Hoewyk, and Neugebauer 2003; Hillygus et al. 2006). Bivariate relationships between indices of such concerns and census mail returns show sizable declines with increasing concern, ranging in 1990 from zero to 23 percentage points, depending on the number of concerns expressed (Singer, Mathiowetz, and Couper 1993).

In an experiment embedded in a monthly telephone survey, Singer (2003) showed that subjective perceptions of disclosure risk, and perceptions of harm from the disclosure of identified information, are highly correlated with expressed willingness to participate in surveys described in vignettes. Perceptions of risk and harm were each measured by four questions asked immediately after respondents had indicated their willingness to participate. However, neither objective risk nor the sensitivity of the survey topic was manipulated in that study.

In the current study, funded by the National Institutes of Health, we broaden our focus to investigate, via laboratory and web studies, the effects of variations in the objective risk of disclosure and the sensitivity of survey topics, as described in vignettes, on expressed willingness to participate in hypothetical surveys. We also examine the interrelationships among objective risk and topic sensitivity, subjective perceptions of disclosure risk and harm, and general attitudes toward privacy/confidentiality, survey organizations, and trust. Finally, we investigate the effect of all these variables on expressed willingness to participate in surveys.

2. Prior Research on Risk and Participation

Ours is by no means the first study to investigate the relationship of objective risk and willingness to participate in research, though not all of the earlier studies dealt specifically with risk of disclosure, and not all studied participation in survey research. One of the earliest of these studies was carried out by Berscheid and her colleagues (Berscheid et al. 1973). Their purpose was not to investigate risk but, rather, to explore the effect of deception on willingness to volunteer for experiments posing some risk to subjects. Using a student sample and self-administered questionnaires, Berscheid and colleagues had subjects indicate their willingness to participate in experiments varying in stressfulness (four vignettes described stressful, and two, nonstressful situations) and in how much of the experimental setup was disclosed. When only the original experimenter’s “cover story” was presented, 3 percent of subjects refused to participate regardless of whether the experiment was stressful or not; however, when full information about procedures was disclosed, 35 percent refused to participate in the stressful experiments.

Another early study was carried out by the U.S. Census Bureau in 1975 because of concerns about the possibility of a reduction in the length of the Census Bureau’s guarantee of confidentiality and the effect of such a reduction on willingness to participate in the 1980 census (National Research Council 1979). The introduction to the survey carried out as one part of this study systematically varied the length of the confidentiality assurance given to the potential respondents, so that one fifth were assured of confidentiality forever, one fifth for 75 years, one fifth for 25 years, one fifth got no mention of confidentiality, and the final group were told their answers would be made public. Refusal rates increased monotonically (and significantly at P < 0.10) with decreasing assurances of confidentiality, though the size of the increase was very small.

In a series of laboratory experiments with community volunteers recruited through newspaper advertisements, Willis and his colleagues (1998) investigated the effect of perceived risks and losses on respondents’ willingness to provide truthful answers to questions about drug use. These experiments are based on “subjective expected utility” theory, which attempts to account for how individuals weigh gains and losses in arriving at risky decisions (for a summary, see Tourangeau, Rips, and Rasinski 2000, pp. 281–284). In these experiments, the risks were potential breaches of anonymity and confidentiality in the answers given; losses were the perceived harms resulting from such breaches. Gains were not explicitly considered. Willis and colleagues used a within-subjects design; that is, each subject read several differing descriptions of survey situations and had to indicate his or her likelihood of answering truthfully to each one. They note that perceived risks and losses had a “quantifiable and predictable” but moderate relationship to the decision about whether or not to answer sensitive questions truthfully.

The question of how much perceptions of risks and benefits affect willingness to participate is an important one for clinical as well as survey research, where both risks and benefits must be disclosed to subjects as part of the informed consent process. Verheggen, Niemen, and Jonkers (1998) investigated the relation of risk–benefit perceptions to the participation decision by interviewing 198 patients who had just been asked to participate in a clinical trial. The patients were first asked whether they had agreed to participate or not; then they were asked questions to measure their perceptions of the risks and benefits involved. Verheggen and his colleagues found the expected significant relationship between these variables, but since those who refused to participate were also more likely to refuse the interview, these findings may be biased in a conservative direction.

Halpern et al. (2004) specifically investigated the hypothesis that higher incentives might make patients in a clinical trial more willing to assume greater risks, an issue that is of concern in the ethical design of experiments. Halpern had each of 126 patients with high blood pressure read nine vignettes resulting from a 3 × 3 design varying levels of risk in a clinical trial (here operationalized as receiving a placebo or experiencing side effects) and levels of payment. He found that willingness to participate in the experiment increased with increasing payment and decreased with decreasing objective risk, but found no interaction between payment and risk, thus refuting the hypothesis, at least under the conditions of this experiment. Interestingly enough, higher payment was required to induce those with higher incomes to participate.

3. Preliminary Studies

On the basis of the experiments just reviewed, it seemed reasonable to assume that objectively described risk of statistical disclosure would affect subjects’ willingness to participate. This assumption was strengthened by 20 cognitive interviews carried out at the beginning of this project, which suggested that any mention of risk was likely to reduce participation. Nevertheless, the first of two laboratory experiments we carried out failed to support this assumption (Conrad et al. 2006). The experiment measured two levels of disclosure risk, each described either quantitatively or qualitatively2; this risk measure was crossed with eight topics, four of them sensitive and four not sensitive. Mode (face-to-face and mail) and social benefit (higher, lower) were also varied. An incentive ($10) and the interview’s duration (20 minutes) were kept constant across all 128 vignettes.

Laboratory subjects (N = 65 for each experiment), recruited through flyers distributed in the community and an advertisement in a local newspaper, read and responded to eight vignettes presented in random order on a computer screen. For each, subjects indicated how likely they would be to participate in the survey and then provided a verbal report of how they had arrived at their decision.

In the first experiment there was no significant effect of objective risk on likelihood of participating: those receiving the high-risk vignettes scored 5.8 on a 0–10 scale of likelihood of participating, whereas those receiving the low-risk vignettes scored 6.1. Topic sensitivity, however, had a pronounced effect, with sensitive topics receiving an average likelihood score of 3.3 and nonsensitive topics a score of 6.9 (F = 291.9, df = 1, 63, P < 0.05).

Experiment 2 increased the differences between risk levels in an effort to increase the likelihood of finding the hypothesized effect of risk on participation. High risk was defined as virtual certainty (qualitative) or virtually a one hundred percent (100%) chance; low risk was defined as “no one will ever connect your answers with your name” (qualitative) or a “zero (0%) chance that someone will connect your answers with your name.” The other features of the vignettes remained the same as in Experiment 1.

Under the altered conditions of Experiment 2, we found the expected effects: high risk led to reliably lower expressed likelihood of participation; low risk led to greater likelihood of participation (6.5 vs 4.5; F = 29.77, df = 1, 63; P < 0.001). The effect of sensitivity remained significant, and we found the predicted interaction between sensitivity and risk, with those exposed to higher risk less likely to participate if the topic was sensitive than if it was not.

The second experiment also showed that objective risk reliably affected subjective perceptions of risk (data not shown), although the latter had no effect on likelihood of participating. This latter finding was at odds with results in Singer (2003), but the experimental design was not identical to that used in the earlier experiment; in particular, it used a within-subject rather than a between-subject design and collected verbal reports from subjects between vignettes with a single question (“How did you come up with your answer?”) rather than the battery used in the earlier study. We therefore designed the web experiment, which is the focus of the present article to see if the findings of the two experiments could be reconciled. Specifically, we formulated the following hypotheses:

  • Objective descriptions of disclosure risk will significantly affect respondents’ expressed willingness to participate in a survey, with higher risk leading to lower willingness to participate.
  • Sensitivity of the topic will significantly affect expressed willingness to participate in a survey, with greater sensitivity leading to lower willingness to participate.
  • An assurance of confidentiality will affect expressed willingness to participate for sensitive topics, but not for topics not considered sensitive.
  • Objective descriptions of disclosure risk will significantly affect subjective perceptions of risk.
  • Topic sensitivity will affect subjective perceptions of harm.
  • Subjective perceptions of risk, harm, and benefits will significantly affect expressed willingness to participate in a survey.

In addition, we examine the role of three additional variables – general privacy concerns, attitudes toward survey organizations, and trust – as factors in expressed willingness to participate in surveys.

4. Web Experiment: Methods

4.1. Sample and Administration

The web survey was administered by Market Strategies Inc. on a volunteer sample drawn from Survey Sampling International’s Internet panel. We received 3,671 completed questionnaires out of a total of 171,469 fielded, for a “response rate” of 3 percent. Of those who returned completed questionnaires, 57 percent were women and 43 percent were men; 75 percent were White and 25 percent were non-White; 19 percent were under 35, 57 percent were between ages 36 and 65, and 24 percent were over 65; 21 percent had a high school education or less, 41 percent had at least some college, and 38 percent were college graduates or more. This is not a probability sample; our focus, to use Kish’s (1987) terms, is on randomization rather than representation.

4.2. Questionnaire

Each questionnaire included a set of eight fictional survey invitations, or vignettes (described below); a question about willingness to participate in these fictional surveys, and reasons for participation or nonparticipation; debriefing questions about perceived risk of disclosure, harm from disclosure, and perceived personal and social benefits, administered to half the sample after the first vignette and to the other half after the eighth vignette; general questions about privacy and confidentiality concerns, attitudes toward surveys, and trust; preferences for having risks described in numbers or in words, and perceived equivalences between numerical and verbal descriptions; background questions and manipulation checks. The entire questionnaire took about 12 minutes to complete.

4.2.1. Vignettes

The vignettes experimentally varied three factors: the survey topic, the description of the risk of disclosure, and an assurance of confidentiality. Four levels of disclosure risk were varied: no mention of risk; no chance; one in a million; and one in ten. Two levels of topic sensitivity, each with two topics, were varied across the vignettes. The high-sensitivity topics were sexual behavior and finances; the low-sensitivity topics were leisure activities and work. Half the vignettes included a confidentiality assurance: “The information you provide is confidential,” and half did not. All eight vignettes seen by a respondent either included such an assurance or did not.

Each vignette also mentioned the study’s sponsor (the National Institutes of Health), a benefit statement (tailored to the topic), the interview length (20 minutes), and an incentive ($10); these features were kept constant across all 32 vignettes resulting from the complete crossing of Topic × Risk × Confidentiality. As noted, each subject was exposed to a subset of eight of these vignettes, with each set containing all four risk statements, one each for a sensitive and a nonsensitive topic, and either a confidentiality assurance or no assurance for all eight. The sets were randomly assigned to subjects after they agreed to participate, and the order in which the vignettes were administered was random within subjects. An example vignette is shown below:

Imagine that in about a week a professional survey interviewer visits your home and asks you to take part in a survey on sexual behavior and sexually transmitted diseases, sponsored by the National Institutes of Health. The information you provide will help shape government policy on sexually transmitted diseases.

The information you provide is confidential. Based on experience there is a one in a million chance that someone will connect your name with your answers.

The interview will take 20 minutes, and you will receive $10 as a token of the researcher’s appreciation.

4.2.2. Willingness to Participate and Reasons for Participation Decision

Willingness to participate (WTP) was measured by a single question, asked immediately after each vignette had been read:

On a scale from zero to ten, where zero means you would definitely not take part and ten means you would definitely take part, how likely is it that you would take part in this survey?

Thus, in this study, willingness to participate is measured by expressed willingness, not by participation in an actual survey. For half of the sample, the first time this question was asked, it was followed by an open-ended probe, asking for reasons why the respondent would or would not participate. For the other half of the sample, the open-ended probe followed the eighth vignette. The first and second reasons given were coded independently by two coders; disagreements between them were reconciled by Singer (kappa for the first codes was 0.79 for reasons for participating (n = 264), and 0.84 for reasons for not participating (n = 836)). Specific reasons were also grouped into major categories (e.g., altruistic or egoistic reasons for participating; privacy concerns or objections to aspects of surveys as reasons for not participating).

4.2.3. Debriefing Questions

Following the questions about participation willingness and reasons for the decision, respondents were asked about their perceptions of the risks, harms, and benefits of participating in the survey described. These questions were asked immediately after the first vignette (for one half of the sample) or after the eighth vignette (for the other half). These measures of risk, harm, and benefit, described below, are based on Singer (2003).

Perceptions of risk

Four items assessed the perception of risks. Parallel items asked about four different groups: family members; businesses that might try to sell you something; employers; and law enforcement agencies like the Internal Revenue Service, the Welfare Department, or the police department:

How likely do you think it is that each of the following people or groups would find out your name and address, along with your answers to the survey questions? Please answer using a scale from zero to ten, where zero means they will never be able to find out your answers, and ten means they are certain to find out your answers.

Answers to the four questions were summed and averaged for a general measure of perceived risk. Cronbach’s alpha for these four items is 0.809.

Perception of harm

“Perception of harm” was measured by the following question, asked about the same four groups listed above:

How much would you mind if each of the following found out your name and address, along with your answers to the survey questions? Please use the same scale from zero to ten, where zero means you would not mind at all, and ten means you would mind a great deal.

Answers were again summed and averaged (Cronbach’s alpha = 0.814).

Perception of benefits to self and society

Benefits to society were measured by the following question, asked about four different groups: the government agency sponsoring the survey; businesses planning new products; other researchers; and law enforcement agencies:

On a scale from zero to ten, where zero means not at all useful and ten means very useful, how useful do you think each of the following groups would find the information from the survey described above?

Answers to the four questions were summed and averaged (Cronbach’s alpha = 0.834).

Benefits to self were assessed by one question, asked immediately after the questions about societal benefits:

Would you, yourself, get anything good out of the survey? (Yes, No)

Perception of risks vs benefits

The risk–benefit ratio was measured by a question that asked:

Taking it all together, do you think the risks of this research outweigh the benefits, or do you think the benefits outweigh the risks? (Risks outweigh benefits, Benefits outweigh risks)

4.2.4. General Questions

Following the vignettes and the questions pertaining directly to them, respondents were asked a series of more general questions, some pertaining to general concerns about privacy and confidentiality and attitudes toward surveys. In an effort to reduce the number of variables for analysis, we subjected these questions to an exploratory factor analysis after appropriate recoding. Two clear factors emerged, one consisting of questions tapping privacy concerns, the other, of questions about attitudes toward survey organizations. Two other variables, measuring trust in people and in government, did not load highly on either factor although they correlated at 0.21 with each other; we combined these into a third index.

Privacy concerns

The Privacy index consisted of three questions: Q. 12: “In general, how worried are you about your personal privacy”; Q. 13: “Please indicate whether or not you ever feel your privacy is violated by the following: Banks and credit card companies when they ask about finances; the government when it collects tax returns; the government when it takes the census; computers that store a lot of information about you; people who ask questions on public opinion surveys”; and Q. 15: “Different private and public organizations have personal information about us. How concerned are you about whether or not they keep this information confidential?” The score on the Privacy index was the sum of the factor scores.

Attitudes toward survey organizations

This variable consisted of answers to two questions: Q. 16: “How much do you trust each of the following to keep the information they collect from you confidential: public opinion research companies, market research companies, government agencies, like the U.S. Census Bureau (not at all, not very much, some, a lot)?” and Q. 23: “Please indicate if you agree or disagree with the following statements: (b) Answering questions in polls or research surveys is in my own best interest; (c) answering questions in polls or research surveys is a waste of time; (d) the terms ‘poll’ and ‘survey’ are often used to disguise a sales pitch; (e) answering questions in polls or research surveys is an interesting experience; (f) the survey research industry serves a useful purpose.” Scores on the Survey index were the sums of the factor scores.

Trust

The index of trust consisted of two questions: Q. 17: “We’d like to know how you view other people, groups, and institutions. Generally speaking, would you say most people can be trusted, or that you can’t be too careful in dealing with people?” and Q. 18: “How much do you trust the government in Washington to do what is right – just about always, most of the time, some of the time, almost never?” Responses of “just about always” and “most of the time” were combined, as were “some of the time” and “never.” Scores on the Trust index ranged from 0 to 2.

4.2.5. Numbers Preference

The final variable used in the analysis measured the respondent’s preference for describing disclosure risk in words or numbers: “Suppose you had your choice of how the chance of somebody finding out your name and address, along with your answers to a survey, was described. Would you prefer to see it described in words (e.g., a very large chance) or in numbers (e.g., 1/10 [one in ten])?” Half the sample was asked first about words and then numbers (as in the above example), the other half about numbers and then words, and the size of the risk was randomly varied from very large (1 in 10) to no chance (0).

5. Web Experiment: Analysis and Results

We begin with an analysis of the three experimentally varied factors – confidentiality assurance, description of disclosure risk, and sensitivity of topic – whereby we first look at their effect on expressed willingness to participate (WTP) and then at their effect on perceptions of risk and harm. We next examine the relationship between subjective perceptions of risk and harm and WTP, and discuss the reasons given by respondents for their participation decision. Finally, we look at the role of general attitudes about privacy and surveys, and at the relative importance of subjective perceptions and general attitudes in explaining variation in participation decisions.

Unless otherwise noted, all analyses are based on responses to the first vignette only, to avoid being affected by exposure to multiple vignettes, and are further restricted to those who were aware of differences among the vignettes (75.9% of the sample). (Awareness was based on answers to the following question: “Did you notice any differences among the eight different survey descriptions?”) Thus, the analyses are based on a between-subjects design, using linear regression with maximum likelihood estimation; N = 1,359.

5.1. The Effect of Confidentiality, Risk, and Sensitivity on Willingness to Participate

Whether or not subjects received an assurance of confidentiality made no significant difference in their WTP, nor did confidentiality interact significantly with either sensitivity of topic or risk of disclosure. Accordingly, interaction terms have been omitted from Table 1.

Table 1
The effect of risk, sensitivity, and confidentiality assurance on willingness to participate, first vignette

Counter to our expectation, disclosure risk, as described in the vignettes, had no significant overall effect on expressed willingness to participate in the survey. The sensitivity of the topic, on the other hand, was highly significant, with leisure activities and work leading to higher levels of WTP than money or sex, the latter being the least likely to elicit agreement to participate. (Table 1 shows the effect for two levels of sensitivity; data on individual topics are not shown.) When the topic of the survey is sensitive, willingness drops by almost a full point on the 11-point scale: from 7.36 to 6.46. Adding the demographic variables of gender, age, education (high school or less, some college, college graduate, post-graduate), race (Black, White, other-mixed) and ethnicity (Hispanic/non-Hispanic) to the model produced only one significant effect: women were significantly less likely to express willingness to participate than men (data not shown).

We repeated these basic analyses across all eight vignettes within person, using IVEware (Raghunathan et al. 2001, 2005) to account for the repeated measures within person. The model in Table 2 includes controls for the order in which the vignettes were presented (with later vignettes producing lower levels of WTP, on average), and whether the debriefing questions were asked after the first or eighth vignette (lower levels of WTP when debriefed early). Neither of these variables interacted with the key manipulations of interest. While the results generally mirror those in Table 1, the risk of disclosure has a statistically significant effect on WTP (F = 3.71, df = 3, 2,774; P < 0.05). These results parallel the earlier laboratory results: when subjects see more than one vignette, and when they are debriefed (as half the sample was) after the first vignette, risk reaches conventional levels of significance. Examining specific contrasts, the one in ten risk is also significantly different from one in a million (t = 10.5, P < 0.001) and from no chance (t = 16.8, P < 0.001), and one in a million is significantly different from no chance (t = 2.86, P < 0.05).

Table 2
The effect of risk, sensitivity, and confidentiality assurance on willingness to participate, all eight vignettes

We also repeated key analyses restricting the sample first to those who were debriefed after the first vignette and noticed differences in topic among the vignettes (N = 1,155), and then to those who noticed differences in the chance of disclosure (N = 930). The first restriction produced no changes from the basic analyses, but the second resulted in risk reaching marginal significance (P = 0.077, df = 3, 925) in predicting WTP, thus duplicating the results of the within-subject analysis. Those who are told there is a one in ten risk of disclosure are less willing to participate than those to whom disclosure is not mentioned (P = 0.013).

5.2. The Effect of Risk, Sensitivity, and Confidentiality, on Perceptions of Risk and Harm

The overall effect of risk of disclosure on subjects’ perceptions of risk is significant (P = 0.014) (Table 3; F = 3.56, df = 3, 1,354). When the risk of disclosure is described as one in ten, perceived risk is 3.93 on an 11-point scale; when it is described as one in a million, perceived risk is 3.32. Neither confidentiality assurance nor sensitivity of the topic has a significant effect on perceived risk. Black respondents perceive higher levels of risk for a given level of disclosure risk than white respondents or respondents of another race, and Hispanic respondents perceive higher levels of risk than non-Hispanic respondents do (data not shown). We also estimated the effect of two alternative definitions of the risk variable, one restricting the analysis to the contrast between one in 10 and one in a million, and the other contrasting one in 10 with all other risk definitions combined. Although these definitions increased the size of the estimated effect of risk on perceived risk, they did not change any of the conclusions. Nor did these alternative definitions produce a significant effect of risk on WTP.

Table 3
The effect of risk, sensitivity, and confidentiality on perceived risk, first vignette

The effects of risk, sensitivity, and confidentiality on perceptions of harm are shown in Table 4. Sensitivity of the topic has a significant effect on perceptions of harm: when the topic is not sensitive, perceived harm is 5.81 on an 11-point scale; when it is sensitive, perceived harm is 6.36 (P = 0.0004, F = 12.84, df = 1). Neither of the other experimental variables is significant, nor are any of the interactions. However, a number of the demographic variables increase perceptions of harm, controlling for sensitivity: women (P = 0.08), younger respondents, college graduates (P = 0.06), those with some postgraduate education, and Black respondents all perceive higher levels of potential harm than their counterparts (data not shown).

Table 4
The effect of risk, sensitivity, and confidentiality on perceived harm, first vignette

5.3. Perceptions of Risk and Harm and Willingness to Participate

In earlier research, Singer (2003) found that subjective perceptions of risk and harm were highly correlated with WTP. However, neither objective risk nor topic sensitivity was systematically varied in that study. We therefore examined the relationship between perceptions of risk and harm and WTP in the present study, controlling for objective risk, topic sensitivity, and confidentiality assurance. As in the earlier study, both variables were significantly correlated with WTP, in the expected direction: the lower the perceptions of risk and harm, the higher the WTP (see Table 5). Neither risk nor confidentiality correlates significantly with WTP, but topic sensitivity remains significant.

Table 5
The association of risk, sensitivity, confidentiality, and perceived risk and harm with willingness to participate, first vignette

5.4. General Attitudes Toward Privacy, Confidentiality, and Surveys and Willingness to Participate

As noted, the questionnaire contained measures of general attitudes toward privacy and confidentiality, surveys, and trust, in addition to measuring perceptions of risk and harm associated with each vignette. Do these general attitudes predict WTP, and what is the relationship between them and perceptions of risk and harm, which are meant to pertain specifically to the immediately preceding vignette?

Table 6 shows the effect3 of general attitudes toward privacy, surveys, and trust on WTP, controlling for objective risk, sensitivity, and confidentiality. The coefficients for privacy concerns and attitudes toward surveys are significant, while that for trust is marginally significant (P = 0.06). To assess the relative significance of these general attitudes, we examined the increase in explained variance in WTP that can be attributed to them, relative to that explained by other variables.

Table 6
The association of risk, sensitivity, confidentiality, privacy concerns, attitudes toward surveys, and trust with willingness to participate, first vignette

The increase in variance explained by the three general attitudes over a model containing the experimental variables only is similar to that produced by adding subjective perceptions of risk and harm to the basic model: from an adjusted R-square of 0.0156 for the basic model to an adjusted R-square of 0.0714 when attitudes toward privacy, surveys, and trust are added to the equation, compared with an increase in R-square to 0.0649 for a model adding perceived risk and perceived harm to the basic model.

Coefficients for perceptions of risk and harm as well as general attitudes toward privacy, surveys, and trust are reduced when all five variables are entered simultaneously into a model predicting WTP, along with the experimentally manipulated variables. However, adding each set of predictors – general attitudes about privacy, trust, and surveys, or perceptions of risk and harm – to a model containing only the other set plus the experimental variables produces similar significant, though modest, increases in the variance explained. R-square in WTP increases from 0.0714 to 0.0909 when perceptions of risk and harm are added to general attitudes about privacy, surveys, and trust, plus the experimental variables (F = 14.23; df = 2, 1,320; P < 0.001), and it increases from 0.0649 to 0.0909 when general attitudes are added to perceptions of risk and harm (F = 12.63; df = 3, 1,320; P < 0.001). Thus, both sets of predictors make a significant, but small, contribution to explaining the total variation in such willingness to participate.

5.5. The Role of Benefits in the Participation Decision

Though not directly relevant to our attempt to understand whether, and by what process, descriptions of survey risk and harm influence participation decisions, statements about personal benefits (a $10 incentive) and social benefits (NIH’s use of the research results to formulate policy in various areas) were also included in the vignettes but not experimentally varied. Thus, three additional variables – perceived personal benefits, perceived social benefits, and respondents’ estimated risk–benefit ratio – were available for analysis. (These variables are described in the Methods section.)

As in earlier research (Singer 2003), these three variables are powerful correlates of willingness to participate, with personal benefit and the risk/benefit ratio having significant and opposite effects on the participation decision (see Table 7).

Table 7
The association of risk, sensitivity, confidentiality, privacy concerns, attitudes toward surveys, trust, perceived risk, perceived harm, personal benefit, social benefit, and risk–benefit ratio with willingness to participate, first vignette

In the model shown in Table 7, sensitivity and perceived risk remain significant, but perceived harm and general attitudes toward privacy and surveys do not. However, all three of the coefficients for variables involving benefits are highly significant, and the explained variance in WTP increases from 0.0909 for the model without these three coefficients to 0.3709 for the model including them (F = 194.72, df = 3, 1,317; P < 0.0001).

5.6. Reasons for the Participation Decision

The first debriefing question answered by respondents after indicating how likely they were to participate in the survey that had just been described to them was an open-ended question probing the reasons for their decision. As noted earlier, half the sample was asked this question after the first vignette, and half after the last (eighth) vignette. Some 13.1 percent of respondents did not answer this question when it was asked after the first vignette, and a slightly higher number – 15.5 percent – failed to answer it when it was asked after the eighth vignette. The discussion here is based on answers to the first vignette only; the distribution of codes to answers based on the last vignette did not differ significantly from those based on the first. The set of codes was adapted from those developed for the vignettes used in Singer (2003).

Responses were divided into reasons for participation (scores of 6–10 on the 11-point scale) and reasons for nonparticipation (scores of 0–5); nonresponse was somewhat higher among those who were reluctant to participate than among those who were more favorably disposed (16 percent versus 11.8 percent).

5.6.1. Reasons for Participating

Some 67 percent of the 1,552 respondents who answered the question about reasons for participating had scores of 6–10 on the 11-point scale. The reasons they gave for being likely to participate in the study described were coded into three large groupings: altruistic reasons (30.3 percent of those responding to the question), egoistic reasons (33.6 percent), and reasons related to survey characteristics (26.6 percent); 5.2 percent of the responses were coded simply as “no objection” (see Table 8).

Table 8
Reasons for participation decision, first vignette

Among the reasons coded as altruistic were a belief that the research is worth while or important; that the person wants to be helpful (e.g., to society, researchers, government) or wants to promote change in policy; or general expressions of altruism (“It’s necessary to do these from time to time”). By far the most frequent altruistic response, given by 27 percent of the sample, was a desire to be helpful or to effect change in policy. Egoistic responses were those implying the respondent would derive some personal benefit from participating (“I enjoy surveys”; “I’d learn something”; “the money”); the most frequent egoistic responses were expressions of enjoyment (14.3 percent) and references to money (14.6 percent). Together, references to personal or social benefits of the research made up almost two thirds of the reasons given for responding to the survey described in the vignette. The most frequent favorable responses to survey characteristics were positive comments about the topic (“it’s interesting”; “the issues are important”); other comments referred favorably to the organization (University of Michigan or NIH); to privacy and confidentiality (“It’s private”; “noninvasive questions”; “not intrusive information”; “I trust confidentiality guarantees”); to convenience (“can do it from home”; “easy to do”); and to length/burden (“doesn’t take much time”; “short survey”)4. Some four percent gave reasons that were uncodable (e.g., “yes”) or inconsistent with their response on the 11-point scale (e.g., someone who answered 10 on the scale but said “I would not be willing to participate”).

5.6.2. Reasons for Nonparticipation

Although they were less numerous, reasons given for nonparticipation were equally specific. Only 3.3 percent of respondents gave what we called “general” reasons for not wanting to take part: (e.g., “Not interested”, with no indication that they found the topic uninteresting). It should be kept in mind, however, that these were respondents to a web survey panel, accustomed to doing many online surveys (79% reported completing 11 or more surveys in the past twelve months).

The most frequent category of objections was that related to privacy concerns: (e.g., “don’t like intrusions”; “too much personal information”; “don’t want strangers in home”; “want/don’t trust confidentiality assurance”). But while 47.4 percent mentioned some aspect of privacy or confidentiality as a reason for not participating, only 3.3 percent specifically referred to the risk of disclosure or of someone’s finding out their answers or name, or to a lack of security or chance of exposure.

Somewhat over a quarter of the sample objected to some aspect of the survey other than privacy (e.g., “takes too long”, “need to know more about questions”); the largest subcategory of these objections referred to the face-to-face mode, with or without a preference for doing surveys online. Given the fact that respondents were members of a web panel, these objections are perhaps not surprising. An additional 13.4 percent made some negative reference to the survey topic (“not interesting enough”, “survey doesn’t apply”)5, and almost four percent specifically said they would want more money to do the survey described. The remainder (5.6 percent) gave scattered other or uncodable responses.

5.7. Preferences for Risk Described in Numbers or Words

Following the eight vignettes, we asked respondents whether they would prefer to see the risk information presented in numbers or words, on a 5-point scale ranging from strongly prefer words to strongly prefer numbers. Fully 44.3 percent of the sample reported a preference for numbers, with 19.5 percent stating a preference for words and 34.6 percent saying it did not matter (1.6% did not answer the question). In a regression model predicting the preference for numbers (not shown), age was negatively associated with a preference for numbers, as was education. Also, women were less likely to prefer numbers, as were racial and ethnic minorities. However, the preference for risk described in words or numbers did not have a significant effect on WTP, either as a main effect or in interaction with the risk manipulation.

6. Discussion

The experiments reported in this article were premised on the assumption that the risk of personal information being disclosed was a significant deterrent to survey participation, especially if the information was sensitive, and that variations in the size of that risk would be reflected in varying levels of expressed willingness to participate in research. This prediction was not supported, except in the within-subject analysis of all responses to eight vignettes, which differed systematically from one another in the size of disclosure risks as well as the sensitivity of survey topics, and when we restricted the analyses based on the first vignette to those respondents who were aware that the vignettes differed with respect to disclosure risk. Under those circumstances, the overall effect of risk was significant in both the laboratory and the web experiments. On the web, extremely small or zero risk (one in a million or no chance) was significantly associated with increased expressed willingness to participate both over the condition in which probability of disclosure was not mentioned at all and over that in which the probability was described as very large. Not even in the within-subject analyses, however, did we find a significant interaction between sensitivity of topic and probability of risk, although this interaction was significant, in the predicted direction, in the laboratory.

Nor did the confidentiality assurance significantly affect willingness to participate; in fact, the direction of the coefficient was consistently negative, suggesting that in the circumstances of the present experiment, an assurance of confidentiality may actually have reduced willingness to participate. Of the three experimentally manipulated variables, only sensitivity of the topic had a large and significant effect on participation willingness, in the predicted direction.

Although this finding may appear at odds with earlier results, in fact there may be no inconsistency. In the present experiment, as in earlier research, it is respondents’ perceptions – their concerns about confidentiality or privacy, their estimations of how likely it is that others will gain access to information about them, and their perceptions of how harmful that would be – that are significantly correlated with willingness to participate in the research. This is true both in earlier research in which respondents’ expressed concerns about privacy and confidentiality predict actual returns of their census questionnaire, and in the current web experiment, in which respondents were simply asked about their willingness to participate in a hypothetical survey. Objective disclosure risk appears to have an effect on participation willingness when respondents are exposed to multiple vignettes, as they were in the laboratory experiment and in the analyses in the present article that are based on all eight vignettes. But when no comparisons with other conditions are possible (for example, when analyses are based on the first vignette), only perceptions of risk show the expected relationship to WTP. Responses to the open-ended question asking for reasons for the nonparticipation decision seem to bear this out. Although reasons related to privacy concerns motivated almost half of the professed nonrespondents to the hypothetical vignettes, only 3.3 percent of the sample specifically referred to the risk of disclosure or a closely related reason.

In the present study, unlike earlier experiments, measures of general attitudes toward privacy, surveys, and trust were included in addition to measures of concern related to specific vignettes. Like subjective perceptions of risk and harm, these attitudes significantly predicted willingness to participate in the hypothetical survey described, and in fact the two measures are interrelated: objective risk, sensitivity, attitudes toward privacy, and attitudes toward surveys are all significant predictors of perceptions of risk; and sensitivity and attitudes toward privacy, surveys, and trust all significantly predict perceptions of harm. Both general attitudes and subjective perceptions significantly increase the variance explained in willingness to participate by the three experimental variables, by roughly the same amount; but each set produces a small but significant increase when added to the model containing the other set. Thus, although there is overlap among these variables, they are not simply proxies for each other.

The primary objective of the study reported here was to understand the role of disclosure risk in respondents’ willingness to participate in surveys; our aim was not to understand the phenomenon of survey participation in general. Nevertheless, each vignette included statements of the social and personal benefits associated with the hypothetical survey, though these were not varied across vignettes, and the debriefing questions inquired about perceived social and personal benefits as well as the perceived risk–benefit ratio. These three variables – especially personal benefit and the risk–benefit ratio – turned out to have large and significant relationships with willingness to participate in the hypothetical survey. Indeed, in their presence, the coefficients associated with general attitudes toward surveys and privacy become insignificant, and that associated with trust is significant at only the 0.10 level, while perceived risk and sensitivity retain a significant effect.

This finding, as well as the responses people gave to the open-ended question about why they were willing (or unwilling) to participate, leads us to hypothesize that while concerns about privacy and confidentiality can reduce such willingness, reassurances about confidentiality, even when they are successful, will not motivate participation. Again, analysis of the open-ended responses supports this conclusion. Almost two thirds of the reasons given for willingness to participate in the hypothetical survey were coded as either altruistic or egoistic – that is, they referred to either the social or personal benefits of the survey described. On the other hand, only 1.8 percent of those willing to participate cited the privacy or confidentiality of the survey as a reason, whereas 47.4 percent cited privacy concerns as a reason for nonparticipation. It is noteworthy that none of the social science experiments cited in our review of the literature – but both of the clinical studies –explicitly varied the benefits involved or asked for respondents’ perceptions of benefits. These findings have clear implications both for future experiments in this area and for what survey organizations say to respondents in their introduction to the survey.

7. Conclusion

The study has a number of limitations. In particular, this is a study asking about hypothetical situations among a group of volunteer subjects who, by definition, are already participating in a survey. However, the online respondents did not express a great deal of enthusiasm for responding to a hypothetical face-to-face survey; the mean WTP was 6.66 for the first vignette and 30.8 percent expressed unwillingness to participate (scores of 0–5 on the scale). Many of the variables behaved as expected in predicting WTP, leading us to believe that the relative judgments of willingness to participate are plausible. In any case, the focus of this study was on examining the effects of the experimental manipulations, rather than on estimating the actual level of participation in a sample of the general population, given a request of the type we examined.

Despite these limitations, we have learned a number of things from our experiments so far. These can be summarized as follows:

  • Except under conditions that make disclosure risk salient, either by providing a frame of reference or by restricting the sample to those most sensitive to risk information, a description of the objective risk of disclosure does not appear to reduce expressed willingness to participate. When participants are exposed to one vignette only, objective risk is not significant; when they are exposed to eight vignettes varying the risk of disclosure, objective risk has a significant effect both in the laboratory and on the web. Descriptions of disclosure risk also significantly affect subjective perceptions of risk.
  • Topic sensitivity has a consistent significant negative effect on willingness to participate as well as on perceptions of harm.
  • Perceptions of risk and harm have a consistent significant negative association with willingness to participate. In a study that measured attitudes before actual participation (Singer, Van Hoewyk, and Neugebauer 2003), perceptions of potential harm – i.e., beliefs that answers to the census could be used against the respondent, or by law enforcement agencies – significantly predicted actual returns of the census form.
  • General attitudes toward privacy, surveys, and trust also influence willingness to participate as well as actual participation.

We speculate that what reduces willingness to participate in a survey is not the actual risk of disclosure, but the perceived risk (probability) of anticipated harm from disclosure. However, based on the results of this experiment, we predict that none of the risk or harm manipulations will explain a great deal of the variation in responses to survey requests, which seem to depend more on perceptions of benefits than on perceptions of risk or harm.

Acknowledgments

We thank NICHD (Grant #P01 HD045753-01) for support. We also thank John Van Hoewyk for indispensable help with analyses and Rachel Orlowski and Catherine Militello for coding the open-ended responses. We also thank the reviewers for their helpful suggestions.

Footnotes

2Qualitative low: Extremely unlikely anyone will ever connect your answers with your name; high: Unlikely anyone will ever connect your answers with your name. Quantitative low: One in a million (0.0001%) chance that anyone will ever connect your answers with your name; high: One in a hundred (1%) chances that anyone will ever connect your answers with your name.

3Although these questions were asked after respondents were asked to indicate their willingness to participate in a survey, we argue that attitudes toward privacy and surveys are independent of the experimental manipulations, and precede them. Separate analyses indicate that the levels of risk, sensitivity, and confidentiality of the last vignette seen by respondents before answering the general attitude questions had no effect on their answers to the general questions; P > 0.05 for all contrasts.

4Based on a content analysis of open-ended responses of 140 participants in a 5-wave panel study, Porst and von Briel (1995) identified three pure types of participants: those who respond for altruistic reasons (e.g., the survey is useful for some purpose important to the respondent, or the respondent is fulfilling a social obligation – 31% of respondents); those who respond for survey-related reasons (e.g., they are interested in the survey topic, or find the interviewer appealing – 38%); and those who cite what the authors call “personal” reasons (e.g., they promised to do it – 30%). The first two categories are clearly similar to what we call altruistic and survey-related reasons, though the second category may include some of what we call personal benefits as well.

5However, specific references to not wanting to talk about sex, or objecting to sex as a topic, were coded as privacy-related reasons; only 3.7 percent of such reasons fell in the sex subcategory.

References

  • Berscheid E, Baron R, Dermer M, Libman M. Anticipating Informed Consent: An Empirical Approach. American Psychologist. 1973;28:913–925. [PubMed]
  • Conrad F, Park H, Singer E, Couper MP, Hubbard F, Groves RM. Impact of Disclosure Risk on Survey Participation Decisions. Presented at the Annual Conference of the American Association for Public Opinion Research; Montreal, QB. May, 18–21.2006.
  • Doyle P, Lane J, Theeuwes JJM, Zayatz LV, editors. Confidentiality, Disclosure, and Data Access. Amsterdam: Elsevier-North Holland; 2001.
  • Fay RL, Bates N, Moore J. Lower Mail Response in the 1990 Census: A Preliminary Interpretation. Proceedings of the Annual Research Conference; Washington, DC: U.S. Census Bureau; 1991.
  • Fienberg SE, Willenborg LCRJ, editors. Journal of Official Statistics. Vol. 14. 1998. Disclosure Limitation Methods for Protecting the Confidentiality of Statistical Data; pp. 337–345. (Special Issue)
  • Fuller WA. Masking Procedures for Microdata Disclosure Limitation. Journal of Official Statistics. 1993;9:383–406.
  • Halpern SD, Karlawish JHT, Casarett D, Berlin JA, Asch DA. Empirical Assessment of Whether Moderate Payments Are Undue or Unjust Inducements for Participation in Clinical Trials. Archives of Internal Medicine. 2004;164:801–803. [PubMed]
  • Hillygus DS, Nie NH, Prewitt K, Pals H. The Hard Count. New York: Russell Sage; 2006.
  • Jabine TB. Statistical Disclosure Limitation Practices of United States Statistical Agencies. Journal of Official Statistics. 1993;9:427–454.
  • Kish L. Statistical Design for Research. New York: John Wiley; 1987.
  • Lambert D. Measures of Disclosure Risk and Harm. Journal of Official Statistics. 1993;9:313–331.
  • Malin B, Sweeney L. Determining the Identifiability of DNA Database Entries. Proceedings, Journal of the American Medical Informatics Association; Washington, DC: Hanley and Belfus, Inc; 2000. Nov, pp. 537–541. [PMC free article] [PubMed]
  • National Research Council. Privacy and Confidentiality as Factors in Survey Response. Washington, DC: National Academy Press; 1979.
  • Paass G. Disclosure Risk and Disclosure Avoidance for Microdata. Journal of Business and Economic Statistics. 1988;6:487–500.
  • Porst R, von Briel C. ZUMA-Arbeitsbericht, Nr. 95/04. Mannheim; Germany: 1995. Wären Sie vielleicht bereit, sich gegebenenfalls noch einmal befragen zu lassen? Oder: Gründe für die Teilnahme an Panelbefragungen. [In German]
  • Raghunathan TE, Lepkowski JM, Van Hoewyk J, Solenberger P. A Multivariate Technique for Multiply Imputing Missing Values Using a Sequence of Regression Models. Survey Methodology. 2001;27:85–95.
  • Raghunathan TE, Lepkowski JM, Van Hoewyk J, Solenberger P. IVEware, A Software for the Analysis of Complex Survey Data with or without Multiple Imputations. University of Michigan, Institute for Social Research; 2005. www.isr.umich.edu/src/smp/ive.
  • Raghunathan TE, Reiter JP, Rubin DB. Multiple Imputation for Statistical Disclosure Limitation. Journal of Official Statistics. 2003;19:1–16.
  • Rubin DJ. Discussion: Statistical Disclosure Limitation. Journal of Official Statistics. 1993;9:461–468.
  • Singer E. Exploring the Meaning of Consent: Participation in Research and Beliefs about Risks and Benefits. Journal of Official Statistics. 2003;19:273–285.
  • Singer E, Mathiowetz N, Couper MP. The Impact of Privacy and Confidentiality Concerns on Census Participation. Public Opinion Quarterly. 1993;57:465–482.
  • Singer E, Van Hoewyk J, Neugebauer R. Attitudes and Behavior: The Impact of Privacy and Confidentiality Concerns on Participation in the 2000 Census. Public Opinion Quarterly. 2003;65:368–384.
  • Tourangeau R, Rips LJ, Rasinski K. The Psychology of Survey Response. Cambridge: Cambridge University Press; 2000.
  • Verheggen FWSM, Nieman F, Jonkers R. Determinants of Patient Participation in Clinical Studies Requiring Informed Consent: Why Patients Enter a Clinical Trial. Patient Education and Counseling. 1998;35:111–125. [PubMed]
  • Willis GB, Rasinski KA, Baldwin AB, Yeh W. Cognitive Research and Responses to Sensitive Survey Questions. Washington, DC: U.S. Department of Health, Centers for Disease Control and Prevention, National Center for Health Statistics, Cognitive Methods Staff Working Paper 24; 1998.
  • Winkler WE. US Census Bureau, Research Report No. RR97/01. 1997. Views on the Production and Use of Confidential Microdata.
PubReader format: click here to try

Formats:

Related citations in PubMed

See reviews...See all...

Cited by other articles in PMC

See all...

Links

  • PubMed
    PubMed
    PubMed citations for these articles

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...