Chapter  13:  Determinants of Disability in Patients With Chronic Renal Failure: Evidence Report/Technology Assessment Number 13

Methodology

Methodology

The USRDS collects patient and facility data from every Medicare-approved dialysis facility in the United States. This registry is sponsored by the National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) and the Health Care Financing Administration (HCFA), and from 1995 to 1999 was coordinated by the University of Michigan. We were primarily interested in a subset of these data, a special study known as the Dialysis Morbidity and Mortality Study (DMMS) Wave 2. This was a prospective, longitudinal study that tracked 4,026 incident dialysis patients in 1996 and 1997, measuring physiological, functional, and quality-of-life variables. This patient subset was advantageous for our purposes because its time frame approximated the time frame of interest for SSA (1 year from onset of ESRD). Therefore, it seemed possible that these data could be used in a de novo regression analysis to determine the best predictors of inability to work, as measured indirectly by the variables "self-reported ability to work full time" and "work status." The purpose of the regression analysis would be to identify a set of diagnostic criteria that would maximize the accurate inclusion of truly disabled patients with ESRD while minimizing the inappropriate inclusion of patients who are able to work.

In order to determine if such an analysis was warranted, we conducted numerous validity and reliability tests on the data. It was important to determine whether the patients in the database were similar enough to the general population of dialysis patients for results to be generalizable ("external validity"); that the variables measured were internally consistent, such that similar measures produced similar results ("construct validity"); and that any analysis of the data would produce consistent results that could be reproduced by other researchers.

To assess this database in these ways, numerous different statistical calculations were employed, including, but not limited to, the following: bivariate correlations of more than 300 variables, Kolmogorov-Smirnov nonparametric comparisons of more than 100 nominal categorical variables, ANOVAs of more than 50 continuous quantitative variables, and a sample regression analysis. Overall, to assess the validity and reliability of this database, more than 500 de novo analyses were conducted. The results of these analyses were reviewed by three physicians in the fields of nephrology and pathology, as well as by several biostatisticians.

We also performed an analysis to determine whether a regression analysis done on one random half of the database would find the same results as a regression analysis done on the other random half of the database. This was an important consideration because, although the database initially followed 4,026 patients, this was reduced to 546 eligible patients. Many were excluded because they were over age 65, lost to followup, or did not fully complete the 1-year followup questionnaire. The final group of patients eligible for our main analysis comprised only 546 individuals, with fragmented data availability, thus reducing the statistical power of any complex multivariate analyses.

Findings

• The portion of the DMMS Wave 2 database of the USRDS relevant to this evidence report demonstrated acceptable external and construct validity.

• Results of multivariate regression analyses were not reliable. Results for one random half of the database did not approximate the results of the other half.

• Because the main analysis proposed for this report could not be conducted, we could not reach conclusions about SSA's current Listings for patients with CRF.

• An illustrative regression and receiver operating characteristic (ROC) analysis are included that provide some inconclusive information about the usefulness of the outcome measures and whether laboratory and physiological measures alone can be used to predict inability to work, in isolation from sociodemographic variables.

• Summary statistics indicated that while approximately 42 percent of ESRD patients were employed full time before beginning dialysis, only 21 percent were employed when they began dialysis, and only 13 percent were employed a year later. Those patients who continued working while on dialysis were most likely to be professional/white collar workers (49 percent).

Future Research

Although the DMMS Wave 2 was a well-designed epidemiological study, its purpose was not to assess disability. Furthermore, only 42 percent of patients completed the entire followup patient questionnaire. Studying disability using data such as the DMMS Wave 2 requires one to "follow" a large number of patients who were working at the time they began dialysis until the time they completed the followup questionnaire. However, both the number of patients initially working and the number of initially working patients who completed the followup questionnaire were relatively small.

Limitations in existing research that should be addressed by future researchers include the use of univariate statistics, retrospective study design, and a focus on demographic and social variables. It would be most useful, for the purposes of answering the questions addressed here, for a study to collect multiple variables longitudinally such that, as did DMMS Wave 2, the independent variables are measured at time 1 and the dependent/outcome variable at time 2. An analysis that includes laboratory and physiologic measurements is essential. Information about each individual's Social Security disability status is also essential in order to evaluate the Listings.

Kidney Function

The kidneys serve several purposes in the waste management and proper functioning of the human body (National Institute of Diabetes and Digestive and Kidney Diseases, 1998):

• They remove wastes from blood through excretion of 1.5 to 2.5 liters of urine every day; in this process, they regulate levels of sodium, phosphorus, potassium, and other essential nutrients.

• They secrete the hormone erythropoietin (EPO), which stimulates the bone marrow to make red blood cells.

• They secrete the hormone renin, which regulates blood pressure.

• They release vitamin D, which helps maintain calcium for bones and normal chemical balance in the body.

Renal failure can be either "chronic," resulting from systemic disease, or "acute," occurring suddenly as a result of incidents such as abdominal trauma, bacterial food poisoning, or drug overdose. Common causes of CRF include diabetes, hypertension, lupus, urologic diseases, and various kidney-specific diseases. When CRF becomes so severe that the patient cannot survive with current kidney function and dialysis or kidney transplant is required, a diagnosis of ESRD applies. This report focuses on ESRD.

The gradual deterioration of the function of the kidneys as they become less able to maintain steady volume and concentration of bodily fluids can, in turn, lead to high blood pressure and edema (fluid in the tissues) as the first symptoms of renal failure. As deterioration progresses, additional symptoms may include fatigue, headaches, nausea, vomiting, decreased appetite, itching, and increased tendency to bleed (Faber and Wilde, 1993). Physiological laboratory indicators of CRF include (National Institute of Diabetes and Digestive and Kidney Diseases, 1998):

• Elevated serum creatinine levels (i.e., waste product of muscle activity, normally filtered out by the kidneys). The definition of abnormal levels depends on many factors, such as disease, sex, age, and muscle mass. Normal levels are generally 0.6 to 1.2 mg/dL. Patients with Type 2 diabetes may have acceptable levels as high as 6 mg/dL.

• Reduced creatinine clearance. Indicates how fast kidneys remove creatinine from blood. Levels below ~85 mL/min are abnormal. This is the most commonly used indicator of glomerular filtration rate (GFR).

• Elevated blood urea nitrogen (BUN). Impaired kidneys cannot filter urea, a protein waste product. Levels >20 mg/dL are abnormal.

• Proteinuria (i.e., elevated protein in urine). Kidneys fail to separate protein from waste. Levels >150 mg over 24 hours indicate proteinuria.

Causes of ESRD

There are several systemic diseases that can lead to impaired kidney function, and ultimately, to kidney failure. The most common among these is diabetes, accounting for 33 percent of prevalent cases, followed by hypertension (24 percent), glomerulonephritis (17 percent), and cystic kidney disease (5 percent) (United States Renal Data System, 1999a). Among incident cases in 1997, the top three causes were diabetes (42 percent), hypertension (25 percent), and glomerulonephritis (9 percent). Most of these diabetes cases are adult-onset (Type 2) diabetes. Other causes of ESRD include interstitial nephritis, neoplasms, and AIDS-related nephropathy (United States Renal Data System, 1999a).

However, the rates at which these diseases cause ESRD differ among age groups. Diabetes is rare (1.6 percent) among ESRD patients 20 years and younger; this age group is most often afflicted with glomerulonephritis (30.1 percent) and cystic/congenital kidney disease (26.2 percent). Among the oldest patients (over 65 years), the most common cause is diabetes (37.3 percent), followed closely by hypertension (35.4 percent) (United States Renal Data System, 1999a).

Treatments for CRF

For those patients with progressive CRF who are not yet in need of dialysis, there are no treatments to prevent the onset of ESRD, only to delay it. The approach will differ depending on the causal disease. The primary treatment approach for all patients is one of dietary therapy to ease the stress on the malfunctioning kidneys. The traditional approach is a protein-, potassium-, and phosphorus-restricted diet. Because the kidneys cannot process protein correctly, intake of normal levels of protein-containing foods (which are also usually high in potassium and phosphorus) can result in accumulation of nitrogen-containing waste products, metabolic acidosis, hyperphosphatemia, and hyperkalemia (Mitch and Maroni, 1998). Recommendations for protein limits range from 0.8 to 1.0 g/Kg of body weight per day (Mitch and Maroni, 1998; Morrison, 1997). The major risk of such a low-protein diet is malnutrition, which can be avoided by taking nitrogen-free amino acid supplements. Maintenance of normal serum phosphorus levels can prevent renal osteodystrophy and progression of renal failure, so careful dietary control is recommended to maintain serum phosphorus levels between 3.5 and 4.0 mg/dL. Foods rich in phosphorus, such as eggs, dairy products, and red meat, are generally avoided (Morrison, 1997). In addition, nephrologists frequently prescribe medications that act as phosphate binders to reduce serum phosphorus (Witten, 1999).

Diabetic patients can also slow progression of disease by monitoring blood glucose carefully. Patients with hypertension need to monitor blood pressure and avoid foods, such as salt, that may increase blood pressure. Physicians frequently prescribe ACE (angiotension converting enzyme) inhibitors for diabetics-even those with normal blood pressure readings-to slow the progression of diabetic kidney disease and reduce proteinuria (Witten, 1999).

For patients with ESRD, it is necessary to provide a replacement for the nonfunctional kidneys in order to filter waste material out of the blood. There are currently three primary options for therapy. The first, and most established, is hemodialysis (HD). This provides cleansing of the blood using an external device that includes a dialyzer to filter waste products out of the blood; dialysate, a liquid that collects the waste products; and a dialyzer for exposure to dialysate across a semipermeable membrane that facilitates molecular diffusion. The use of this external device requires the creation of an arteriovenous (AV) fistula (internal joining of artery and vein) or arteriovenous graft (artificial blood vessel used to join artery and vein), usually in the patient's forearm, or-for temporary access or when other options for access have been exhausted-a cuffed or noncuffed double-lumen percutaneously inserted central venous catheter may be inserted into the jugular vein, subclavian vein, or femoral vein. These methods allow blood to flow out of the arm, through the dialyzer, and-having had toxins removed-back into the patient. The average treatment time is 3 to 4 hours every 2 to 3 days (Kidney Dialysis Foundation, 1999; Witten, 1999; National Institute of Diabetes and Digestive and Kidney Diseases, 1994). Treatment can be provided at a dialysis center ("in-center" HD) or at home.

Peritoneal dialysis (PD), a newer approach, is the internal cleansing of blood using the peritoneal membrane of the abdominopelvic wall as the dialyzer. This approach requires a permanent catheter inserted into the abdomen. The dialysate is passed through the catheter into the patient's abdomen, where it collects waste products from capillaries in the peritoneal membrane. High glucose levels in the dialysate cause water to be drawn from the blood in the capillaries by osmosis, and waste products are removed by convection and diffusion (Kidney Dialysis Foundation, 1999). The dialysate remains in the abdomen for several hours, then is drained, to be replaced with fresh dialysate (National Institute of Diabetes and Digestive and Kidney Diseases, 1994). This can be done using three methods:

• 1

  Continuous ambulatory peritoneal dialysis (CAPD). the blood is always being cleaned. The patient drains solution from a bag through a small permanent peritoneal catheter into the peritoneal cavity several times every day. After the solution is drained, the patient disconnects from the now empty bag, wraps the catheter tubing, and covers it with a dressing to prevent infection (Witten, 1999). Each cycle takes 4 to 6 hours. The continuous nature of this method leads to better clearance of poorly dialyzable compounds, especially phosphate (Morrison, 1997).

• 2

  Continuous cycling peritoneal dialysis (CCPD). A machine automatically fills and drains dialysis solution into the peritoneal cavity through the permanent peritoneal catheter at night while the patient sleeps. Total cycle time is 10 to 12 hours nightly.

• 3

  Intermittent peritoneal dialysis (IPD). This process is similar to CCPD, but is usually done in a hospital, and it takes longer than CCPD (up to 24 hours), but is done less often (3 to 4 times a week). This treatment option is rarely used in the United States today.

Current statistics from the USRDS indicate that in-center hemodialysis continues to be the most often used modality of dialysis treatment in the United States, as it has been for the past 10 years. It has consistently been the treatment given to 80 to 85 percent of patients undergoing dialysis. Most recent figures estimate that approximately 85 percent use in-center HD, 9 percent use CAPD, 4 percent use CCPD, and 1 percent use home hemodialysis. The use of CCPD has been steadily rising for the past 5 years, replacing CAPD for many patients, but not affecting the usage of in-center hemodialysis (United States Renal Data System, 1999a). These figures are in sharp contrast to some other countries, such as Canada, Australia, and Denmark, where at-home peritoneal dialysis is the favored method (United States Renal Data System, 1999a). The choice of treatment option is currently being studied as part of a multiyear international research project titled "Dialysis Outcomes and Practice Patterns Study."

The third option, and the only alternative to dialysis, is kidney transplant, either from a live or decedent donor. Decedent donors are more common, comprising approximately three-quarters of all transplants in the United States (United Network for Organ Sharing , 1997). Live donors are most often blood relatives of the patients, in order to achieve better HLA (human leukocyte antigen) matching, and thus reduce the risk of kidney rejection. The major risks of kidney transplant are rejection of the kidney and graft failure. Patients must take immunosuppressants (such as corticosteroids, azathioprine, and cyclosporine) for the rest of their lives to prevent rejection (Morrison, 1997). The necessity of immunosuppression generally limits the patients who are eligible for transplant to those who are young and/or in reasonably good health, although improvements in the procedure are expanding the potential patient pool. Conditions and situations that limit the patients who are eligible for transplant also include poor cardiac or vascular status, recent (within 1 year) malignancy, unstable psychological status, and current misuse of alcohol or drugs. In addition, kidney transplantation is limited by the availability of viable organs for transplantation.

Transplantation of the kidney can be done either alone or in conjunction with other organs, most commonly the pancreas. Prevalence estimates from USRDS indicate that approximately 28 percent of all ESRD patients have a functioning transplant (United States Renal Data System, 1999a). Recent survival estimates from the United Network of Organ Sharing (UNOS) suggest a 1-year graft survival rate of 85.6 percent and a patient survival rate of 94.9 percent. Approximately 38 percent of patients experience rejection episodes during the first 6 months after transplant, and 12 percent undergo multiple transplants due to rejection or graft failure (United Network for Organ Sharing, 1997).

Epidemiology

The most recent calculations by the USRDS indicate that 110 of every 100,000 people have ESRD. About 29 of every 100,000 are diagnosed with ESRD each year. The rate of new diagnoses (i.e., incidence) has been growing, most recently at an average rate of 5 percent per year since 1992 (United States Renal Data System, 1999a). Men are diagnosed with ESRD more often than are women (131 per 100,000 versus 93 per 100,000), possibly a result of the higher prevalence of some of the causative diseases in men. African-Americans make up a disproportionately large percentage of the ESRD population, making up 32.1 percent of all ESRD patients (while African-Americans make up 12.6 percent of the population as a whole). This may be due to a higher incidence of diabetes and hypertension in this population (United States Renal Data System, 1999a). Sixty-one percent of ESRD patients are Caucasian, and the other 6.9 percent are minorities other than African-Americans.

The average age of all prevalent cases of ESRD is 56 years. On the other hand, the average age of new incident cases in 1997 was 61 years. The reason for this age increase is unclear. Persons aged 45 to 64 made up the largest proportion of ESRD patients in 1997, at 38.8 percent. Although patients with ESRD are generally in the older age groups, it is important to assess working ability, since patients under age 45 make up almost 28 percent of all prevalent cases, and just over 16 percent of incident cases in 1997 (United States Renal Data System, 1999a).

Determining the prevalence of CRF overall is more difficult, because many people who suffer from earlier stages of the disease do not seek treatment or are not diagnosed properly; therefore, reliable statistics are not available.

The average survival time of ESRD patients (with treatment) is between 19 and 47 percent of average survival time of the general age- and sex-matched population. First year mortality for incident ESRD patients in 1996 was 19.8 per 100 patient-years at risk (adjusted for age, sex, and primary causal disease). Although incidence of ESRD is higher in the African-American population than in the Caucasian population, first-year death rates have been consistently higher among Caucasians (23.3 percent versus 18.8 percent in 1996). However, this racial difference has been steadily decreasing, from 12.6 per 100 person-years at risk in 1986 to 4.5 per 100 person-years at risk in 1996, as the Caucasian death rate has decreased more rapidly than has that of African-Americans. Patients with diabetes demonstrate the highest first-year mortality rate of any single diagnostic group (23.2 per 100 person-years at risk). Men and women have nearly equal mortality rates (United States Renal Data System, 1999a).

History of Social Security Disability Coverage

After the establishment of the retirement insurance program under the Social Security Act of 1935 (Public Law (P.L.) 74-271), serious thought was given to whether that program should be expanded to provide wage related cash benefits to workers who become permanently and totally disabled before age 65 and to their dependents. During the period from 1940 to 1950, the Social Security Board and its 1946 successor, the Social Security Administration, recommended in their annual reports that benefits be provided to permanently and totally disabled workers and their dependents as part of the Social Security system. The 1948 Advisory Council on Social Security to the Senate Finance Committee made specific recommendations for the payment of Social Security benefits to disabled workers (S. Doc. No. 162, 80^th Cong., 1^st sess. (1948)). However, the Social Security Act Amendments of 1950 (P.L. 81-734) made no provision for disability insurance benefits. Instead, provision was made for grants in aid to the States for public assistance to permanently and totally disabled, needy individuals. The Social Security Amendments of 1952 (P.L. 82-590) included a measure providing for the establishment of a "disability freeze," but the measure was not affirmed prior to July 1953 and did not become operative.

The Social Security Amendments of 1954 (P.L. 83-761) created the first Social Security disability program with the institution of the disability freeze. As defined by the 1954 Amendments, disability meant, "inability to engage in any substantial gainful activity by reason of any medically determinable physical or mental impairment which can be expected to result in death or to be of long-continued and indefinite duration." Monthly disability insurance benefits were first established by the Social Security Amendments of 1956 (P.L. 84-880). Benefits were provided for disabled insured workers between the ages of 50 and 65 and for disabled children of retired or deceased insured workers if the child was disabled before age 18. The Social Security Amendments of 1958 (P.L. 85-840) expanded the program by including benefits for dependents of disabled workers. The Social Security Amendments of 1960 (P.L. 86-778) removed the minimum age requirement of 50 years for disability insurance beneficiaries. The Social Security Amendments of 1965 (P.L. 89-97) deleted the requirement that the impairment be of "long-continued and indefinite duration" and substituted in its place a requirement that the impairment "be expected to last for a continuous period of not less than 12 months."

The Social Security Amendments of 1967 (P.L. 90-248) added language to the definition to make it clear that a claimant may be found disabled "only if his physical or mental impairment or impairments are of such severity that he is not only unable to do his previous work but cannot, considering his age, education, and work experience, engage in any other kind of substantial gainful work which exists in the national economy, regardless of whether such work exists in the immediate area in which he lives, or whether a specific job vacancy exists for him, or whether he would be hired if he applied for work." The Secretary was also given specific statutory authority to prescribe, by regulations, criteria for determining when services performed or when earnings from services demonstrate ability to engage in substantial gainful activity (Social Security Administration, 1999a and 1999b).

The Social Security Amendments of 1972 (P.L. 92-603) created the Supplemental Security Income program for the Aged, Blind and Disabled. The definition of disability used for disability insurance benefits was carried over into the SSI program, with a modification for SSI claimants under age 18. The 1972 Amendments also extended health insurance coverage to people who had chronic renal disease and required dialysis (including peritoneal dialysis) or kidney transplantation (Solomon, 1986).

The Social Security Amendments of 1978 made significant modifications in the chronic renal disease provisions of the Medicare program. This law designated it as the End-Stage Renal Disease program and established renal disease network areas and a national ESRD medical information system and provided Medicare payment to providers of ESRD services (United States Congress, 1977). Additional amendments have been added to the Act since then, including the addition of vocational rehabilitation programs and work incentive plans.

Social Security Disability Insurance

The SSA administers two programs that provide benefits based on disability: the Social Security Disability Insurance program (title II of the Social Security Act) and the Supplemental Security Income program (title XVI of the Act). Title II provides for payment of disability benefits to individuals who are "insured" under the Act by virtue of the Social Security tax on their earnings, as well as to certain disabled dependents of insured individuals. Title XVI provides for SSI payments to individuals (including children under age 18) who are disabled and have limited income and resources.

SSA currently defines disability as "the inability to engage in any substantial gainful activity by reason of any medically determinable physical or mental impairment(s) which can be expected to result in death or which has lasted or can be expected to last for a continuous period of not less than 12 months." SSA currently defines "substantial gainful activity," in turn, as working for pay or profit and earning more than $700 a month (Social Security Administration, 1999c).

A worker who becomes disabled must wait 5 full calendar months after disability begins before receiving SSDI benefits (Social Security Administration, 1995). SSDI provides only compensation for lost employment. It does not provide health insurance until the individual has been entitled to benefits for 24 months. In general, disabled individuals requiring health insurance can receive it either through their current or former employer, a spouse's current or former employer, or they can qualify for Medicare and, in most states, persons who receive Supplemental Security Insurance benefits are automatically entitled to Medicaid without a waiting period. Persons with ESRD qualify for Medicare based on their ESRD condition. Unless the person with ESRD chooses to perform home or self-care dialysis, there is a 3-month waiting period for Medicare. Persons who are covered under an employer group health plan have a 30-month period during which Medicare is the secondary payer (Social Security Administration, 1995).

Social Security evaluates claims for disability using a five-step sequential evaluation process. The five-step process asks the following questions:

• 1

  Is the individual engaging in substantial gainful activity? If the individual is working and the work is substantial gainful activity, as defined by SSA's regulations, a determination of "not disabled" is made. Otherwise, the adjudicator proceeds to step 2 of the sequence.

• 2

  Does the individual have a medically determinable impairment or combination of impairments that is severe, as defined in SSA's regulations? If the individual does not have an impairment or combination of impairments that is severe, a determination of "not disabled" is made. If the individual has an impairment or combination of impairments that is severe, the adjudicator proceeds to step 3 of the sequence.

• 3

  Does the individual's impairment(s) meet or equal the severity of an impairment listed in appendix 1 of subpart P of part 404 of the SSA regulations? If so, and the duration requirement is met, SSA finds that he or she is disabled. If not, the adjudicator proceeds to step 4 of the sequence.

• 4

  Does the individual's impairment(s) prevent him or her from doing his or her past relevant work, considering his or her residual functional capacity? If not, a determination of "not disabled" is made. If so, the adjudicator proceeds to step 5 of the sequence.

• 5

  Does the individual's impairment(s) prevent him or her from adjusting to other work that exists in the national economy, considering his or her residual functional capacity together with the "vocational factors" of age, education, and work experience? If so, and if the duration requirement is met, SSA finds that the individual is disabled. If not, the individual is determined not disabled (Proposed rules, 1999).

Different provisions are made if the individual has never performed "skilled" labor. The determination for these individuals is made as follows:

"If you have only a marginal education and work experience of 35 years or more during which you did arduous unskilled physical labor, and you are not working and are no longer able to do this kind of work because of a severe impairment(s), we will consider you unable to do lighter work, and therefore, disabled. However, if you are working or have worked despite your impairment(s) (except where the work is sporadic or is not medically advisable), we will review all the facts in your case, and we may find that you are not disabled. In addition, we will consider that you are not disabled if the evidence shows that you have training or past work experience which enables you to do substantial gainful activity in another occupation with your impairment, either on a full-time or a reasonably regular part-time basis." (CFR 20 §404.1562) (Social Security Administration, 1997a)

Many different individuals take part in this evaluation process, including the SSA's State Disability Determination Service agencies. A claimant's physician is asked to provide medical information and at times consultative examinations may be ordered. When a claimant appeals a decision denying his or her claim for disability benefits, the court may evaluate the claim using the same sequential evaluation process.

Listing of Impairments

Since 1954, there has been an established list of medical impairments which, in and of themselves, are considered sufficient to preclude any gainful activity, absent evidence to the contrary. The Listing of Impairments was published in the regulations of August 1968. Prior to that date, the regulations contained a brief list of examples of impairments that would ordinarily be considered disabling. That list in the regulations was supplemented by a listing that appeared in the Disability Insurance State Manual (DISM).

Title XVI benefits for children under age 18 began in 1974. When this new category of recipients came into being, it was recognized that the Listings then in effect (now designated Part A) would not be appropriate in all cases for evaluating disability in children. Part B of the Listings, which contain additional medical criteria that apply only to the evaluation of impairments in children under 18, was published in the regulations in March 1977 (Social Security Administration , 1999b).

Revisions to all of the impairment listings in Part A of the Listings were published in the regulations in March 1979. Additional revisions to selected impairment listings have been published between 1979 and 1999; however, the impairment listings in Part A related to CRF have not been revised since March 1979. The Listingsprovide guidelines for determining disability for a variety of impairments, both physical and mental. The Listing for adult patients with CRF, developed by a consensus panel of physicians, from section 6.02 of the Listing of Impairments reads as follows (Social Security Administration, 1998):
6.02 Impairment of Renal Function, due to any chronic renal disease expected to last 12 months (e.g., hypertensive vascular disease, chronic nephritis, nephrolithiasis, polycystic disease, bilateral hydronephrosis, etc.). With:

• 1

  Chronic hemodialysis or peritoneal dialysis necessitated by irreversible renal failure,

• 2

  Kidney transplant, consider under a disability for 12 months following surgery; thereafter, evaluate the residual impairment (see Section 6.00C), or

• 3

  Persistent elevation of serum creatinine to 4 mg/dL (100 mL.) or greater or reduction of creatinine clearance to 20 mL/min (29 L/24 hours) or less, over at least 3 months, with one of the following:

• 1

  Renal osteodystrophy manifested by severe bone pain and appropriate radiographic abnormalities (e.g., osteitis fibrosa, marked osteoporosis, pathologic fractures); or

• 2

  A clinical episode of pericarditis; or

• 3

  Persistent motor or sensory neuropathy; or

• 4

  Intractable pruritus; or

• 5

  Persistent fluid overload syndrome resulting in diastolic hypertension (110 mm. or above) or signs of vascular congestion; or

• 6

  Persistent anorexia with recent weight loss and current weight meeting the values in 5.08, Table III or IV; or

• 7

  Persistent hematocrits of 30 percent or less.

Listing 106.02 of the Listing of Impairments(Social Security Administration, 1998) regarding children with CRF reads as follows:

• 106.02 Chronic Renal Disease. With:

• Persistent elevation of serum creatinine to 3 mg. per deciliter (100 ml.) or greater, over at least 3 months; or Reduction of creatinine clearance to 30 ml. per minute (43 liters/ 24 hours) per 1.73m 2 of body surface area over at least 3 months, or Chronic renal dialysis program for irreversible renal failure; or Renal transplant. Consider under a disability for 12 months following surgery; thereafter evaluate the residual impairment (see 106.00B).

At the time these Listings were finalized, patients receiving dialysis were treated as "end-stage" patients (i.e., patients who were severely infirm and likely to die). There were few treatments available to allow these patients to live a normal, active life. Thus, all patients on dialysis were considered disabled. It was recognized, as well, that some patients not yet on dialysis might be infirm as well, if theirs was a progressive chronic kidney disease, and so the listings in sections 6.02C and 106.02A and B (above) were included.

However, because of changes in treatments, it is unclear whether those guidelines are still appropriate today, in the 1990s, and beyond. It is not the purpose of this report, however, to evaluate advances in treatment. The purpose, instead, is to determine whether the Listings are supported by evidence in the clinical literature, and if not, what combination of laboratory, clinical, functional, and demographic variables most accurately predict inability to work in adult patients with CRF.

Work Incentives

Work incentives are provided by SSA to allow disability beneficiaries to test their ability to work without losing benefits. In general, a person has at least 3 years to test the ability to work, including full disability payments during the first 9 months and a period in which disability benefits can be started again without a new application. The patient continues to receive Medicare coverage during this time (Social Security Administration, 1995). These work incentives are active for a 9-month trial period, after which SSA assesses whether the patient's earnings are "substantial." If this is the case, the patient may lose SSDI benefits after 3 more months. If earnings are not "substantial," work incentives can continue for an additional 36 months, during which SSI is received for any month during which the patient's income falls below $700 per month (Social Security Administration, 1995).

In summary, the SSDI work incentives include:

• Reimbursement for impairment-related work expenses,

• Trial work period for 9 months,

• Extended period of eligibility for 36 months if earnings are less than $700 per month,

• Continuation of Medicare coverage for 39 months,

• Continued payment under a vocational rehabilitation program, and

• Additional benefits for those patients who are blind.

However, there has been criticism that information about these work incentives is difficult for disability beneficiaries to obtain. A major obstacle to this information is the lack of vocational rehabilitation counseling. While assessment of vocational rehabilitation usefulness is a mandated part of the SSDI application process (Social Security Administration, 1997b), patients with ESRD rarely see these rehabilitation counselors, as evidenced by one statistic showing that only half of dialysis patients had contact with a vocational rehabilitation counselor (Kutner and Brogan, 1989). Critics point out that this may be due in part to Public Law 97-35, which delays Social Security funding of State vocational rehabilitation agencies until it can be proven that their use leads to substantial gainful activity for at least 9 consecutive months (Life Options Rehabilitation Advisory Council, 1994). Other potential problems with the work incentives program include patient reluctance to give up disability payments and patient ignorance about the existence of work incentives programs (Life Options Rehabilitation Advisory Council, 1997; Kapron, Nix, and Smith-Wheelock, 1996; Evans, Manninen, Garrison et al., 1983; Lundin and Lundin, 1983).

Focus and Refinement of Topic

This project was divided into two phases, the first of which consisted of an examination of published literature and the second of which consisted of de novo statistical analysis of data in a large national database.

To focus, refine, and arrive at the key questions addressed by this assessment, the research team met with the initiator of the request of this evidence report (the SSA), the administrating agency (i.e., Agency for Healthcare Research and Quality [AHRQ], formerly the Agency for Health Care Policy and Research), and a panel of three experts in the field of nephrology. The technical experts participating in this meeting all had clinical backgrounds in nephrology as well as varying backgrounds in research. To ensure that the research team obtained and considered the viewpoints of individuals with very diverse expertise in the beginning stages of this project, several conference calls with other technical experts were conducted. These experts had expertise in the areas of health services research, disability, vocational rehabilitation, and other related fields.

In the course of this initial meeting (held in November 1998) and subsequent followup meetings, the scope of the project was defined in the following manner. For the subset of the Listings related to CRF, SSA wished to determine whether the evidence in the clinical literature:

• Supports the listed criteria as currently written,

• Refutes the criteria as currently written, or

• Is insufficient to either support or refute the criteria as currently written.

If the listed criteria as currently written were not specifically supported by the evidence in the clinical literature, SSA sought to identify any alternative criteria that might serve as better predictors of disability.

If a review or analysis of the clinical literature were insufficient to answer these questions, SSA wished to know what kinds of research would be required to reach satisfactory answers.

Initial discussions indicated that SSA would be interested in all CRF patients, both those on dialysis and those not, and both adult and pediatric patients. However, the parameters of the project were further refined during additional meetings with SSA and AHRQ over the course of Phase 1 of the project. Findings of Phase 1 (discussed below) further limited the topic to adults with CRF due to a paucity of data on pediatric CRF patients and indicated that most available data were about ESRD patients, with very little information available on nondialyzed CRF patients.

It was also made clear during these initial meetings that an evaluation of treatment efficacy was not appropriate. Rather, the Listings need to be based on physiologic measures and functional status; therefore the focus of the project would be on clinical, laboratory, and functional indices that could potentially predict ability to work. Whether or not these values are affected by particular treatment regimens is irrelevant for the purposes of this project.

Finalized key questions were arrived at by the end of Phase 1 (January 1999) as outlined above in the Introduction section.

Measuring Inability To Work

Published Studies on Difficulties in Measuring (In)ability To Work

Though SSA uses its own operational definitions of "disability" and "inability to work," the definitions of these terms in the clinical literature vary considerably. In this section, we explore some of these variations.

Table 1. Impairment and disability terminology

Table 1. Impairment and disability terminology

Concept	Nagi terminology*	WHO ICDH terminology for "disablement"**	IOM terminology***
Disease process	Active pathology	Disease	Pathology
Physical or mental function (molecular or cellular level to organ system level)	Impairment	Impairment	Impairment
Limitations of actions (e.g., movement of arm or leg)		Disability	Functional limitation
Limitations in basic activities (e.g., walking, running)	Functional limitation
Disability/disablement (Limitations in complex activities) (e.g., housework, gainful employment, recreation, interaction with community)	Disability	Handicap	Disability
Quality-of-life (the subjective experience of pain and pleasure)	Quality of life	Quality of life	Quality of life

* From Nagi and colleagues (Nagi, 1991; Nagi, 1979; Nagi, 1965)

** World Health Organization International Classification of Impairments, Disabilities, and Handicaps (World Health Organization, 1980)

***Institute of Medicine (Pope and Tarlov, 1991)

There are complex relationships among pathologies, physical and mental functions, and social functions (such as the ability to carry out gainful employment) (Verbrugge and Jette, 1994; Hahn, 1993;LaPlante, 1991). A broad categorization of the different levels of function has emerged (Nagi, 1991; Rettig and Levinsky, 1991;World Health Organization, 1980; Nagi, 1979; Nagi, 1965); however, exact definitions from various sources are overlapping and not in complete agreement (see Table 1). This broad hierarchy is briefly described here:

• Disease process. A past or current disease or pathological condition underlies and is the causative factor or one of the causative factors for the condition or health state.

• Physical or mental function. These are biological or psychological consequences of the condition that occur from the molecular and cellular level up through the organ system level.

• Limitations of action. As a consequence of the impairment, there is a loss of ability to carry out certain action(s), such as the movement of an arm or leg, detection by a sensory organ, or the performance of a simple mental task.

• Limitations in basic activities. Because of the action inability, certain basic activities are limited or impossible, such as walking, running, sensory awareness of the social and physical environment, or carrying out sequential mental tasks.

• Disability/disablement. Because of the limitation in basic activities, the individual may be unable to or have limited ability to engage in physical and mental tasks and complex social activities, such as housework, economically gainful work, social interaction with family and community, procreation, and recreation, that are normally part of the social environment or are required by the physical environment. Unlike the above levels of function, the existence and extent of disability/disablement is determined not only by the functional capabilities of the individual, but also by the demands of the social and physical environment (Verbrugge and Jette, 1994; Hahn, 1993). Also, in some cases, a disability can be ameliorated or even completely removed by behavioral, medical, or device-related interventions, even though none of the above levels of function or pathology has changed (Verbrugge and Jette, 1994; Hahn, 1993).

• Quality of life. Suboptimal performance in any of the above areas can diminish an individual's quality of life. Quality of life may not be directly relevant to employment disability, but it is involved in the consideration of disability in general.

This evidence report is concerned only with conditions that limit one's ability to engage in substantial gainful activity (SGA). However, determination of an individual's capacity to engage in SGA requires in-depth and subtle knowledge of the other levels of function described above and a mapping of these functional abilities to job requirements. The medical classification of pathologies is based on disease etiology or causation, not on the physical, mental, and social consequences of disease; therefore, there has been a general failure of strictly medical diagnoses to inform disability policy and to produce universal or consistent criteria for disability (Hahn, 1993; Stone, 1984).

The establishment of criteria for determining employment disability involves unique problems beyond defining the concept of disability in general. In a market economy in which economic competition is an inherent part of gainful work, there is the further complication that the mere ability to go through the motions of work does not ensure that the work will in fact be competitive and gainful. There are also a number of factors that are difficult to assess in terms of physical or medical observations. These include the amount of pain involved in otherwise performable activities, the amount of time and energy outside working hours that are expended dealing with the disease or its consequences, and whether a person with a terminal condition should be required to work as close as possible to the time of genuine inability to work or death. This concept is so deeply imbedded in social, cultural, and economic frameworks that are unique in time and place that it is only possible to approach the answer to the question of "who can work?" by instead answering by direct observation the question of "who does work, and under what circumstances?"

Answering the question of which patients with ESRD can work is made even more difficult by the fact that all such patients are currently found disabled at the third step of the SSA disability sequential evaluation process. Thus, the fact that a patient is not working may simply be because he or she would rather receive disability than be gainfully employed or that a patient anticipates (rightly or wrongly) that he or she will be unable to work in the near future. One can, however, gain some insight into the relationship between the impairment caused by CRF and the ability to engage in substantial gainful activity. This can be done by following those patients who are gainfully employed at the time they are diagnosed with ESRD in spite of the fact that they qualify for disability. The measurable medical parameters at the point that these patients stop working would be the criteria that ideally belong in the Listings. This is because these parameters indicate the point in disease at which well-motivated patients cannot (or do not) continue to work. Also, the point at which essentially no patients continue to work is important because this point is when education, skills, and unique job requirements no longer influence one's ability to work. Using measurable medical parameters at these two time points is compatible with the purpose of the SSDI screen-in process, which aims to identify conditions with which no one should be expected to work, regardless of skills and present job requirements. In the SSA disability sequential evaluation process, such individual background and job factors are considered in steps 2, 4, and 5. The Listings are intended to contain conditions that automatically result in a favorable disability determination for the most severely impaired patients, regardless of job factors.

A further difficulty is encountered because no diagnostic cut point for determining who is unable to work will be perfect. Any "test" for inability to work will invariably misclassify some individuals as being unable to work when, in fact, they can work, and/or individuals may be misclassified as not disabled when, in fact, they are unable to work. Therefore, the challenge becomes one of maximizing the ability of any "test" for disability to correctly determine whether a person is able or unable to work, while minimizing the ability of any such "test" to incorrectly determine an individual's inability to work. Issues related to determining the predictive value of a "test" for disability are further discussed in Appendix E.

In summary, the evidence report cannot directly assess inability to work. This is not due to a lack of literature on the subject or lack of data per se. Rather, it is due to the impracticality of directly measuring this concept. This problem is inherent to all studies of disability. Consequently, the evidence report must focus on outcomes that can be or have been measured.

Electronic Database Searches

Twenty-seven databases were searched for relevant information. We searched for information in each database from the date of its inception, therefore all records in these databases were considered:
ABI/Inform® (through November 12, 1998)
Abledata (NARIC) (through November 12, 1998)
The Cochrane Database of Systematic Reviews (through 1999 Issue 2)
The Cochrane Registry of Clinical Trials (through 1999 Issue 2)
The Cochrane Review Methodology Database (through 1999 Issue 2)
Combined Health Information Database (CHID) (through November 2, 1998)
CRISP (through December 3, 1998)
Current Contents® (through June, 1999)
The Database of Reviews of Effectiveness (Cochrane Library) (through 1999 Issue 2)
DIRLINE® (through November 1998)
ECRI Library Catalog (through October 29, 1998)
EMBASE® (Excerpta Medica) (1980 through November 23, 1998)
Health Care Finance Administration Database (through April 22, 1999)
Health Devices Alerts® (1977 through June 1999)
Healthcare Standards (1975 through June 1999)
Health Devices Sourcebase® (through June 1999)
HealthSTAR (Health Services, Technology, Administration, and Research) (1980 through April 13, 1999)
HSRProj (through November 4, 1998)
Hypertension, Dialysis & Clinical Nephrology© (HDCN) (through June 1999)
International Health Technology Assessment (IHTA)© (1990 through June 1999)
MEDLINE® (1980 through April 13, 1999)
Nursing and Allied Health (NAHL/CINAHL)® (1980 through October 20, 1998)
PsycINFO® (1980 through December 2, 1999)
RehabDATA (NARIC) (through November 12, 1998)
Sci Citation Index® (through October 22, 1998)
Social SciSearch® (through October 21, 1998)
TARGET^TM (through October 6, 1999)

The search strategies employed a number of free-text keywords as well as controlled vocabulary terms, including (but not limited to) the following concepts:

• Study design-Controlled trials: Randomized controlled; controlled clinical trials; meta-analysis; random allocation; single-blind method; double-blind method, evidence-based medicine (includes randomized controlled trials, outcomes research, and meta-analysis)

• Disability: Disabled; disability; disability evaluation

• Disorders: ESRD; end-stage renal disease; ESRF; end-stage renal failure; kidney failure, chronic

• Interventions: Dialysis; haemodialysis; hemodialysis; peritoneal dialysis; renal replacement therapies; kidney transplantation

• Miscellaneous: Educational status; patients; patient compliance; patient participation; predictive value of tests; quality-of-life; QOL; sex factors; social class; socioeconomic factors; time factors

• Work: Employment; employability; employment status; job re-entry; re-employment; unemployment; vocational rehabilitation; work capacity evaluation; workload; work scheduling.

In general, the searches were restricted to studies examining human subjects. Case reports were excluded.

World Wide Web Searches

Searches of the World Wide Web were also conducted using various search engines including (but not limited to) AltaVista, Hotbot, Infoseek, Magellan, and Yahoo!®. Pertinent Web sites included:

Hand Searches of Journal and Nonjournal Literature

More than 1,600 journals and supplements maintained in ECRI's collections were routinely reviewed. Nonjournal publications and conference proceedings from professional organizations, private agencies, and government agencies were also screened.

Other Mechanisms

Other mechanisms were used to retrieve additional relevant information, including review of bibliographies/reference lists from peer-reviewed and gray literature. (Gray literature includes reports, studies, etc. produced by local government agencies, private organizations, educational facilities, and corporations, etc., that do not appear in the peer-reviewed literature.)

Documents Identified

These search strategies identified 3,492 documents, books, and World Wide Web resources. The lead analyst reviewed the search results to identify relevant documents and to ensure that all pertinent information was retrieved using these search strategies. Input from technical experts and members of the internal review committee also helped revise the search strategies. Through these processes, new searches were conducted, and a total of 503 documents were ordered.

Categorization of Ordered Articles

When documents were ordered, they were catalogued as to the part of the project for which they were being retrieved. Below is a list of all the categories used, and the number of ordered articles that fell into each category. [Note: the total number of documents does not add up to 503 due to cross-referencing of categories.]

Final Count

We read the complete texts of these 503 studies, and determined that only 14 of them contained any analysis of predictors of employment. All of these 14 studies pertained solely to adult ESRD patients. There were also an additional 19 studies that may contain some relevant data but were limited by other factors. The final disposition of those studies deemed irrelevant is listed below.

Data on Pediatric ESRD Patients

We were unable to identify appropriate data in the published, peer-reviewed literature regarding factors that may predict disability in pediatric patients with ESRD. Not only are there few studies on this topic in the published literature, but there is also uncertainty as to what constitutes disability in children. A handful of studies have looked at the school attendance of such children, but none has identified predictor variables for school attendance. Other studies have examined the physiologic and cognitive development of such children, but have not related it to any disability measurement.

Pediatric data in the USRDS are also limited. The most extensive data are contained in the Pediatric Growth and Development subset. This data set contains extensive medical and demographic information about pediatric ESRD patients. Functionality for pediatric patients in this database is measured solely through school attendance and physical development measures; this would overly limit our definition of functionality in pediatric patients for purposes of this analysis. It does not contain information about broader quality-of-life measures. Another subset of data, the DMMS Wave 2, which contains a quality-of-life questionnaire, does not include pediatric patients.

Summary of Published Evidence

There are limitations to all of the data in the published literature that preclude their use in analysis for this report:

• 1

  Most studies used univariate statistical tests (e.g., chi square or ANOVA). These tests do not control for the effects that other variables might have upon the outcomes.

• 2

  Most of the variables were demographic or psychological, and therefore, not ethically or easily incorporated into the SSA disability assessment process.

• 3

  None of these studies was longitudinally designed to allow assessment of predictive value of independent variables.

• 4

  Patients were examined at many different time points after the start of dialysis. Most patients were examined or interviewed 5 to 6 years after beginning dialysis. This does not approximate the time frame of interest to SSA (1 year).

Because of these limitations, we concluded that there are currently no published data available to either support or refute the current Listings for CRF. We therefore evaluated the possibility of using individual patient data from the USRDS database to answer the key questions of this project. There were several potential advantages to the use of these data:

• 1

  The DMMS Wave 2 data included medical, demographic, functional, and quality-of-life data from more than 4,000 adult dialysis patients in the United States. This was substantially larger, and therefore has the potential to be more statistically reliable, than data from any published study.

• 2

  Because the data are at the individual patient level, the analysis can be tailored to the key questions.

• 3

  The DMMS Wave 2 data were collected longitudinally, in a two-stage interview process: once at the initialization of dialysis and once 9 to 12 months later. Medical, demographic, and functional measures were taken at both interviews. This allowed us to identify predictors at time 1 for outcomes at time 2.

• 4

  The time frame of this study-9 to 12 months-was almost identical to the time frame considered in evaluation of disability, unlike most published studies.

• 5

  The data collection was prospectively planned. This reduced the potential for patient selection bias.

On the other hand, this database did have the drawback that it was not designed to answer questions about disability status. As discussed in the rest of this report, it has limitations that preclude definitively addressing SSA's key question.

Acquisition of Data

The CD-ROM that included DMMS Wave 2 data was obtained through application to USRDS and NIDDK. We explained the purpose of this study, and received permission to obtain the CD-ROM in April 1999. The CD was obtained from the USRDS Coordinating Center, then at the University of Michigan.

Validity Analysis

Before using the USRDS database in de novo analyses, it was important to ensure that the results of such analyses would be relevant to the entire U.S. population of persons with ESRD that might apply for SSDI or SSI disability benefits. Obviously, any results derived from these data would be of limited utility if they could not be extended beyond the specific study sites and patients (Cook and Campbell, 1979). It was also important to determine whether a variable measures what it is supposed to measure and that no major coding errors were present.

There are several different types of validity analyses that are possible, some of which can be performed after a study has been conducted, and others of which require appropriate study design (and, hence, depend upon the originating researchers to ensure and report). Below, we discuss three general types of validity-external, internal, and construct-and our results after assessing these aspects of the USRDS data.

Validity Analyses

External Validity

Study size

In part, external validity is affected by the size of the study population. Studies with large numbers of patients are more likely to yield generalizable results than studies with small numbers of patients. In the DMMS Wave 2 study, information was recorded for 4,026 patients on dialysis at the start of the study in 1996: medical information was available for 3,985 patients on dialysis, and 2,713 completed the patient questionnaire addressing quality-of-life issues during an interview. While compliance with filling out the patient questionnaire was low, this would still be considered a large pool of individual data. However, as discussed below, the actual number of patients with followup information on employment status is much smaller than this, which causes some statistical difficulties for this analysis.

Patient selection bias

External validity also depends upon patient selection for the study. People and places selected randomly are more likely to yield data generalizable to the larger population than those nonrandomly selected. Information in the USRDS Researchers Guide indicates that, for DMMS Wave 2, 25 percent of U.S. dialysis centers were randomly selected from which to gather the patient pool. All identified peritoneal dialysis patients and 20 percent of all in-center hemodialysis patients at these centers were asked to participate. (These steps were taken due to the small proportion of PD patients in the U.S. dialysis population, and a desire to create a data set with equal numbers of patients receiving each type of treatment) (United States Renal Data System, 1999b). Hemodialysis patients were selected based on the last digit of their Social Security number. The fact that both centers and patients were selected without regard to individual characteristics represents a strength of the DMMS Wave 2 data.

However, the overrepresentation of patients on PD and the exclusion of patients on home HD is not a particular threat to external validity unless one analyzes the database in toto. It is possible to at least partly combat this threat by conducting separate statistical analyses on hemodialysis and peritoneal dialysis patients, so as not to misrepresent the makeup of the entire U.S. dialysis population. It is also possible, during statistical analyses, to "weight" the cases to reproportion the database.

Self-selection bias is a problem inherent to any voluntary-response study such as this, since certain types of people are apt to agree to participate, while others are not. The number of patients that were initially asked to take part in DMMS Wave 2 was not reported, and therefore it is not possible to calculate the study's response rate. This represents an aspect of the external validity we cannot address because it relies upon reporting by the original investigators. We can determine, however, that 67.4 percent of patients represented in the database completed the initial patient questionnaire and 42.0 percent completed the followup questionnaire, which is not an unusually low participation rate for such epidemiological studies. It does provide evidence, however, that a patient self-selection bias could be present in this database.

Comparison of DMMS Wave 2 to USRDS

Table 2. Comparison of DMMS Wave 2 patients to all (more...)

Table 2. Comparison of DMMS Wave 2 patients to all patients in the USRDS database

	DMMS Wave 2 (%)		USRDS database (%) a
Characteristic	HD	PD	HD	PD
1-year survival	85.8%		82.04% b
Average age	61.0	55.8	61.0 b
% Female	47.5	46.5	47.5	48.1
% Caucasian	57.3	68.9	53.8	67.7
% African-American	34.2	22.1	39.2	25.5
% other races	8.5	9.0	7.0	6.8
Primary cause: diabetes	43.3	43.8	38.5	35.2
Primary cause: hypertension	29.2	22.4	28.8	22.0
Primary cause: glomerulonephritis	7.1	9.9	12.4	19.5
Primary cause: other	20.4	23.9	20.3	24.7

1

Data from USRDS main database are from 1997, as reported in the 1999 Annual Data Report (United States Renal Data System, 1999a)

b

These data from USRDS main database are not broken down by type of treatment; therefore, we collapse the DMMS Wave 2 data for appropriate comparison.

HD = Hemodialysis

PD = Peritoneal dialysis

One way to determine the extent of the effects of self-selection bias involves an in-depth comparison of raw data from the DMMS Wave 2 patients to the entire USRDS patient population. Such an analysis, however, was beyond the purview of the present project. Because such comparisons were also not reported by the researchers, we approximated such a comparison by comparing the DMMS data to summary statistics provided in the USRDS Annual Data Report (United States Renal Data System, 1999a), and these results are shown in Table 2. Table 2 depicts group averages that are not the result of an original statistical analysis.

Table 2 presents results, when available, separately for peritoneal and hemodialysis patients because of the disproportionate number of each type of dialysis patient in this database compared to the entire USRDS patient population. From these data, it appears that patients with diabetes are overrepresented in the DMMS Wave 2 data, while patients with glomerulonephritis are underrepresented, thus bringing the generalizability of this database into question.

Loss of patients to followup

The DMMS Wave 2 data shown in Table 2 are those collected at the beginning of the study. A followup questionnaire was administered to these patients approximately 9 to 12 months later (mean = 10.0 months, 95 percent confidence interval = 4.8 months), and all patients who completed the first questionnaire were asked to participate in the followup. However, 2,329 of these patients (58 percent) did not complete the followup patient questionnaire and for 501 (12 percent) the followup medical information was not available. Followup data on the variables "Ability to work full time" and "Employment status," two important variables for this report, are available for 1,670 patients (41.5 percent). Of the 2,376 without this information, 978 (41.2 percent) were not followed due to known occurrence of death. Others were recorded as "lost to followup" (494), or (presumably) chose not to answer the second questionnaire.

Loss of some patients to followup is the rule rather than the exception in longitudinal studies and is often more severe when conducting surveys than when conducting an experimental research study. Data on employment from the second questionnaire are not available for about 58 percent of the patients in the DMMS Wave 2 study (53 percent if those dying are excluded from the dropout rate). It is unknown what factors other than death account for this. This dropout rate is only important, however, if it is determined that there are substantial differences in medical, demographic, or functional measures between the patients who were followed and those that were not. Dropout can also present difficulties if the number of data points remaining is not sufficient to conduct reliable and reproducible statistical analyses. (This is addressed below.)

Comparison of patients with and without followup data

Table 3. Final subgroup of patients for proposed analysis (more...)

Table 3. Final subgroup of patients for proposed analysis

Exclusion criterion	Number excluded (%) a	Cumulative number excluded (%)	Cumulative number remaining (%)
Non-incident patients -- those recorded as starting dialysis before 1996	137 (3.5%)	137 (3.5%)	3,889 (96.5%)
Patients age 65 and over	1,711 (42.5%)	1,765 (43.8%)	2,261 (56.2%)
Patients who were lost to followup: those for whom followup employment information (work status or ability to work full time) was not available OR for whom a death notice was not recorded	1,469 (36.5%)	2,718 (67.5%)	1,308 (32.5%)
Patients who were not employed (full or part time) at some point before initiating dialysis treatment	2,791 (69.3%)	3,480 (86.4%)	546 (13.6%)

a

Number excluded assumes that only this exclusion criterion was used, not in conjunction with any of the others.

Percentage is out of original patient population of 4,026. Patients may fall into more than one category, so the figures in the rightmost column of this table cannot be obtained by direct subtraction of figures in this column for 4,026.

In order to address whether the remaining data were suitable to our purposes, we conducted a de novo analysis of the DMMS Wave 2 data, specifically comparing those patients who would be included in the final analysis (a total of 546) to those who would be excluded (a total of 3,480) (see Table 3 below for a summary of the included subgroup of patients). A better analysis would compare this final data set to the USRDS as a whole, but, as mentioned above, such an analysis was beyond the scope of this report.

Included in the final data set are incident dialysis patients who were younger than 65, who provided followup data on employment status or self-reported ability to work, and who were working at some point before or during initiation of dialysis treatment. This subset allows us to best identify predictors of inability to work because only those who have worked at some point are faced with the decision of whether they can continue working or not. All medical, demographic, and functional measures taken during the first interview were compared for these two groups of patients. To control for age effects, we compared only those under age 65 in both the excluded and included groups and, to control for the effects of severity of disease, we excluded data from all patients who died during the study (even though these patients may be maintained in the final data set, because death is an outcome of significance in the disability assessment process).

The analysis conducted consisted of a series of three types of univariate statistics comparing those in the final data set to those excluded, using SPSS as the statistical software (SPSS 9.0, SPSS, Inc., Chicago, IL). This analysis employed the phi coefficient to compare nominal categorical variables; all data on demographics and most functional status variables were analyzed using this statistic. The second analysis was a Kolmogorov-Smirnov Z Test, a nonparametric test that compares ordinally ranked categorical variables for two groups. This type of analysis was appropriate for several items on the patient questionnaires. The third analysis was a one-way ANOVA, suitable for analyzing the continuous variables in this database. Several items on the medical questionnaire (e.g., weight, blood chemistry measurements) were appropriate for this type of analysis.

Table 4. Statistically significant differences between (more...)

Table 4. Statistically significant differences between patients in the final data set and those excluded from this data set

Variable	Characteristic of included patients compared to excluded patients	Effect size	p-value
Phi statistic: results as expected
Modality of dialysis	More likely to be PD patients	0.091	0.001
Primary cause of ESRD	Less likely to have diabetes; more likely to have primary glomerulonephritis and other causes, including polycystic kidney disease	0.080	0.002
Prior diagnosis of coronary heart disease/coronary artery disease (CHD/CAD)	Less likely to have or be suspected of having this diagnosis	0.130	0.001
Diagnosis of angina	Less likely to have angina	0.095	0.001
Myocardial infarction (MI)	Less likely to have MI	0.090	0.001
Cerebrovascular accident (CVA)	Less likely to have CVA	0.084	0.001
Peripheral vascular disease (PVD)	Less likely to have PVD	0.101	0.001
Congestive heart failure (CHF)	Less likely to have CHF	0.148	0.001
Pulmonary edema	Less likelyto have edema	0.072	0.004
Prior diagnosis of diabetes	Less likely to have had diabetes	0.071	0.003
History of lung disease	Less likely to have had lung disease	0.072	0.004
Hemodialysis: type of access	More likely to have AV fistula; less likely to have PTFE graft or permanent catheter	0.116	0.004
Eating independently	More likely to be able	0.058	0.001
Transferring independently	More likely to be able	0.098	0.001
Ambulating independently	More likely to be able	0.103	0.001
Marital status	More likely to be single	0.168	0.001
Limited in kind of work	Less likely to be limited	0.078	0.003
Difficulty performing work	Less likely to be limited	0.076	0.003
Sleep/nap during day	Less likely to do so	0.074	0.004
Able to work part time or full time at start of study	More likely to say "yes"	PT: 0.206 FT: 0.278	0.001
Evaluated for transplant	More likely to have been evaluated	0.199	0.001
On waiting list for transplant	More likely to be on waiting list	0.156	0.001
Assistance given to complete form	Less likely to have received assistance	0.191	0.001
Phi statistic: unexpected results
None
Kolmogorov-Smirnov analysis: results as expected
Education	Higher education	***	0.001
General health	Better general health	***	0.001
Moderate activities: lifting, climbing one or several flights of stairs, bending, walking one or several blocks, bathing/dressing self	Less limited	***	0.001
Feelings of pep	More peppy	***	0.001
Feelings of energy	More energy	***	0.001
Interference with social life	Less interference	***	0.001
Kolmogorov-Smirnov analysis: unexpected results
None
ANOVA: results as expected
Age	Lower age	0.0881	0.001
Blood urea nitrogen (BUN) postdialysis at study start date	Higher BUN	0.1020	0.001
Weight predialysis and postdialysis	Higher weight	Pre: 0.0770 Post: 0.0945	0.002 0.001
Predialysis and postdialysis diastolic blood pressure (DBP)	Higher DBP	Pre: 0.1398 Post: 0.1426	0.001 0.001
Predialysis creatinine	Higher creatinine	0.1214	0.002
ANOVA: unexpected results
None

*** Effect size cannot be computed from this statistic.

These analyses indicated that there were statistically significant differences (as indicated by the p-value) between those included in the final data set and those excluded. While these may seem important, the effect sizes are very low (see Table 4). Effect sizes are expressed similarly to correlations, where a more extreme negative or positive number conveys a stronger effect. None of these effect sizes was above 0.3, a low to moderate effect size, suggesting that these findings may not be clinically significant. Some of these statistically "significant" relationships between variables may also be spurious due to collinearity of other variables not accounted for in these univariate analyses.

All differences between the excluded and included patients were in the direction anticipated, with younger and healthier patients more likely to be included in the final data set.

Two points are worthy of mention regarding our analyses of these differences. First, we did not attempt to correct for the fact that we employed multiple individual statistical tests on the same data set. By not doing so, we minimize the chance that we will overlook any difference between excluded and included patients (i.e., we maximize the statistical power, thus reducing the probability of a Type II error), but increase the probability that at least some of these apparent differences are the result of chance (i.e., there is an inflated Type I error rate in our comparisons). As such, these results are a "worst case" scenario, chosen to illustrate the maximum possible differences that could exist between included and excluded patients.

Second, when examining these results, one should not rely on the p-values to determine the magnitude of these differences. P-values are heavily influenced by the size of a study, and thus are a poor measure of the magnitude of difference between two groups. There are numerous examples in the literature of studies that used large numbers of patients and found that very small differences were statistically significant. An example of this is the putative statistically significant relationship between height and IQ (Dowdney, Skuse, Morris et al., 1998; Downie, Mulligan, Stratford et al., 1997; Wilson, Hammer, Duncan, et al., 1986).

The results of the statistics described in Table 4 were derived from what may be regarded as a relatively large number of patients. This explains why the p-values are relatively low. However, when one examines the effect sizes, which are a more accurate reflection of the magnitude of the differences between these two groups, a different picture emerges. In general, none of them is large. It is also important to note that the 34 variables listed in Table 4 are the significant results of 466 variables on which analyses were performed. Thus, for 431 socioeconomic, demographic, clinical, and laboratory values, there were no differences between these two groups. It is important to remember, however, that because we did not adjust the p-values, one would expect 23 differences (5 percent) to be significant simply due to chance.

Another important between-group comparison would be that comparing previously working patients for whom followup data were available versus previously working patients who were lost to followup. However, this particular analysis would have been most appropriate if it were done after we conducted the main logistic regression analysis. As discussed below, we did not conduct this analysis, and therefore did not perform the group comparison recommended here.

Conclusions about external validity

The results presented in the sections above offer mixed evidence about the external validity of this database. One basic problem is that patients were not chosen completely at random, but rather in a way to make a 50/50 mix of PD and HD patients. This means that this database cannot be analyzed in toto and cannot be expected to represent the characteristics of ESRD patients in the United States. When patient characteristics for PD and HD were examined separately and compared to the USRDS as a whole, differences still emerged in the makeup of these groups, such that certain diseases were disproportionately represented in DMMS Wave 2.

However, we would not use the entire DMMS Wave 2 database for our analysis, but rather a subset of patients for whom followup data were available, who were under 65, and who had worked at some point in the past. This subset of 546 patients appears to be younger and healthier than the patients excluded from the final subset, as might be expected.

Construct Validity

Statistical analyses

As mentioned earlier, construct validity refers to whether a test or questionnaire score represents the concept of interest. This can be assessed by correlating different measures of the same characteristic and seeing how well they match. We conducted several types of statistical analyses to assess the construct validity of the DMMS Wave 2 data. The first was a series of bivariate correlation analyses. This method indexes the degree to which the value of any single variable varies consistently with any other single variable. Both Spearman's rank correlation (rho) and Pearson's r were used; Spearman's rhois calculated the same as Pearson's r except that the value of each variable has been transformed to a rank. Pearson's r is only appropriate for continuous variables, while Spearman's rhois more appropriate for ordinal variables. Because both types of variables were present in this database, both types of correlation analyses were used. If comparing continuous and ordinal data, Spearman's was used. We use this as a method of approximation to check the data for any gross errors, such as coding errors, or illogical correlations (such as divergence between two variables where convergence might logically be expected to occur).

We also performed analyses similar to those that were done for external validity. Thus, we used the phicoefficient to relate nominal categorical variables, the Kolmogorov-Smirnov nonparametric method to relate nominal and ordinal categorical variables, and one-way ANOVA to relate nominal variables with continuous variables. These tests were conducted for cases in which at least one of the variables was nominal categorical. For example, such analyses were used with the variable "treatment modality type," which is coded as 1 = hemodialysis and 2 = peritoneal dialysis. This variable is not easily correlated with such continuous variables as creatinine levels and body-mass index.

Bivariate correlations were calculated for more than 300 variables. These analyses resulted in many statistically significant findings, perhaps because of the large number of cases (more than 3,000 for some variables). As discussed above, when a large number of cases are involved, statistically significant results are likely, not because of a large effect, but because of the large number of patients. Similarly, "false-positive" results (i.e., Type I errors) are likely when one conducts a large number of univariate statistical tests, not taking into account covariation of other variables that may be affecting results. This is partly why the magnitude of the correlation (r or rho) is a better indicator of the magnitude of a relationship than the p-value.

Correlational trends

Table 5. Correlations between measures of employment (more...)

Table 5. Correlations between measures of employment and ability to work

Description of trends in correlations
Current employment status positively correlated with previous employment status.
Previously employed more likely to be professional or clerical workers.
Those who report a desire to work are those more likely to be employed or to say they are able to work.
Those who say they are able to work part or full time are more likely to be professional or clerical workers and to be looking for employment if not currently employed, and are less likely to say they are limited in the kind of work they can do.
Those who say they are not working because they will lose benefits are unlikely to be looking for work.

Table 5 shows a summary of important correlations among different measures of employment status and ability to work. Because of the large number of univariate analyses involved in these correlation matrices and the risk of Type I errors, we are, for the purposes of the present document, defining a "significant" correlation as any one whose p-value was <0.001² and r or rho-value above 0.2. As can be seen in the correlation matrices and summaries below, few of these correlations were large (defined arbitrarily as above 0.5), and most fell in the 0.2 to 0.4 range. As none of these questions asked exactly the same question about exactly the same point in time, it is to be expected that these variables would be correlated, but not strongly (related, but conceptually distinct measures). It must also be remembered that any statistically significant correlations may be spurious, as bivariate correlations do not account for collinearity of other variables.

Table 6.Correlations between employment measures and (more...)

Table 6.Correlations between employment measures and functional status measures

Description of trends in correlations
Those who have trouble performing a variety of daily activities, including climbing stairs, bending, walking several blocks, and lifting, are less likely to be previously or currently employed.
Different measures of activities of daily living are strongly positively correlated with one another.
Independent transferring, ambulating, and eating are strongly positively correlated with one another, but are not correlated with ability to perform daily activities (e.g., walking, bending, lifting).
Those with diminished general health are more likely to describe themselves as depressed, worn out, or unhappy, and to experience interference with their social activities.
Symptoms such as dry skin, bad breath, cramps, and muscle soreness (along with several other side effects of kidney impairment and dialysis) are not correlated with employment status, employment level, or self-reported ability to work.
Cognitive impairment correlates mildly (0.2 to 0.3) with measures of general health.
Measures of different daily activity impairment correlate mildly-moderately (0.2 to 0.3) with one another.
Conceptually related questions correlate moderately-strongly with one another (0.4 to 0.5). For example, "feeling energetic" correlates 0.697 with "feeling full of pep."
Cognitive and emotional impairment is not significantly correlated with physical limitations.
Symptoms of ESRD/dialysis treatment are only mildly correlated with physical limitations.
Feelings of stress are associated with an inability to perform daily tasks (~0.3).
Sleep quality and need to rest are only mildly (~0.2) correlated with physical limitations.
High correlations (~0.6) exist among different measures of impaired work performance.
Moderate correlations (0.3 to 0.4) exist between measures of pain and measures of impaired work performance.
Moderate correlations (0.3 to 0.4) exist among measures of emotional health.
Moderate correlations exist among measures of cognitive function and ability to accomplish tasks.
Mild correlations exist among measures of emotional health and symptoms of ESRD/dialysis treatment.
Amount of support from family and friends has a minor effect (~0.2) on feelings of depression.

Table 6 shows correlations among measures of functional status, and between functional status and employment status. Again, as predicted, these measures are significantly correlated with one another in a logical way, but many are only moderately correlated because they measure conceptually distinct, but related, constructs (such as height and weight). As mentioned above concerning employment-related variables, few of the questions on the patient questionnaire asked about exactly the same phenomenon, characteristic, or symptom. Therefore, they should not be strongly correlated. Those few questions that did ask conceptually similar questions yielded higher correlations (e.g., energy and "pep" had a correlation of about 0.7).

Table 7.Correlations between clinical and laboratory (more...)

Table 7.Correlations between clinical and laboratory measurements and measures of employment and self-reported ability to worka

Description of trends in correlations
Different aspects of cardiovascular impairments (e.g., angina, myocardial infarction, cardiac arrest, congestive heart failure) correlate moderately (0.3 to 0.5) with one another.
Different diagnoses of central vascular disease (e.g., stroke, transient ischemic attack) correlate moderately (~0.4) with one another.
Different symptoms of peripheral vascular problems (e.g., diagnosis of PVD, amputation, claudication, absent pulse in feet) correlate moderately (0.3 to 0.4) with one another.
Hemoglobin and hematocrit have a high correlation on Spearman's rank (rho = 0.838) and low correlation when using Pearson's r (r = 0.084). [discussed below]
Predialysis body-mass index positively correlated with triglyceride measurements (rho = 0.263)
Dialysate urea nitrogen, blood urea nitrogen, dialysate creatinine, serum creatinine, and serum phosporus are all positively correlated (0.2 to 0.5).
Those patients on peritoneal dialysis have a higher education and are more likely to say they can work full or part time (0.2 to 0.3).
Patients with congestive heart failure are less likely to have been employed within the past 2 years than those without (-0.212).
Higher postdialysis blood urea nitrogen among patients indicating ability to work part time (rho = -0.208). No such correlation with self-reported ability to work full time.

a Results from these analyses may not be reliable due to the high number of nominal categorical variables, not typically appropriate for correlation analyses.

Table 7 shows correlations between laboratory measurements and employment variables. As expected, pre- and postdialysis measures of the same laboratory values correlate with one another moderately to strongly (0.4 to 0.8). When the same laboratory measures are compared at initial interview to those taken at followup interview 9 to12 months later, there is still a correlation, but it is less powerful (0.2 to 0.4). There were few significant correlations between laboratory/medical values and employment measures, and those are shown in Table 7. This may indicate that no single clinical measurement is predictive of employment status or ability to work and indicates the need to simultaneously use several variables to predict employment. Logistic regression allows for such simultaneous consideration, as is discussed later in this document.

In addition to those shown in the tables below, more correlational trends are provided in Appendix B.

The low correlation of hematocrit and hemoglobin is an interesting finding, as these are related measurements of red blood cell count. When hematocrit is graphed against hemoglobin on an x-y axis, the following pattern is seen:

Figure 2. Hematocrit values (percent) versus hemoglobin values (g/dL)

It appears that there is a subgroup of patients (n = 63) for whom hemoglobin is high while hematocrit was low-an unusual and difficult-to-explain finding. This may be due to hemolysis during dialysis (resulting in an artificially low red blood cell, or hematocrit, count). Dialysis facilities were not instructed as to when, in relation to dialysis session, the laboratory measurements were to have been taken. These findings would suggest that for some patients, these readings were taken after dialysis. Alternatively, it could suggest that these were patients in crisis with extremely low hemoglobin, for whom transfusions were given during which hemolysis occurred. However, analysis suggests that these patients were no more likely to have been receiving transfusions than patients with normal blood count readings. It is also possible that this is a miscoding error for some patients in whom hematocrit was recorded as hemoglobin, and vice versa. This, however, is speculation. As a result, these findings are currently unexplainable.

Conclusions about construct validity

Construct validity appears to be adequate in this database with a few exceptions, in particular, that of an apparent coding error with hemoglobin and/or hematocrit. The knowledge of such a discrepancy gives us the option either to discard these apparently invalid data in our final analysis if we feel that the results would be unreliable, or to recode them properly if the source of the error can be determined.

Data Reliability

Analysis protocol

While the above results provide information about whether this database can be generalized to the entire dialysis population and whether individual datapoints measure what they are intended to measure, they do not indicate whether the data set is reliable enough for the particular statistical methods we intended to use. We have noted that for many patients, certain segments of data, especially quality-of-life and employment data, are missing. Large amounts of missing data can cause difficulties in conducting statistical analysis intended to identify variables associated with an inability to work.

The general goal of this reliability analysis was to compare analyses on one randomly selected half of the DMMS Wave 2 database to the results of analyses on the other half. Failure to obtain equivalent results from analyses of both halves of the database would suggest that the results of this analysis are unreliable. The protocol for this analysis was as follows:

• 1

  Randomly assign each patient in the DMMS Wave 2 database to one of two groups.

• 2

  From each resulting half of the database, select only those patients who, according to their medical records, were currently less than age 65 and who were working prior to diagnosis of ESRD. The, latter selection was made using the question "Occupation level before ESRD" of the DMMS Wave 2 Medical Questionnaire. Only patients whose primary occupations were listed as professional, clerical, tradesperson, or manual laborer were included. Patients who were not employed or whose primary occupations were student, other, or homemaker were excluded.

• 3

  Compute basic statistics (mean, median, minimum, and maximum) for the individual variables for each half of the database from the following sections of the medical and patient questionnaires: Patient and Facility Identification, Patient History within 10 Years Prior to Study Start Date, Information at Study Start Date, Laboratory Data, Patient Questionnaire, and Medical Care before Regular Dialysis. Items of the Patient Questionnaire were not examined individually, but as scored subscales as constructed by the developers of the KDQOL ^TM (see Appendix C).

• 4

  Exclude any variable from further analysis if data were available for less than 50 percent of the patients on this variable. This exclusion is required for technical reasons. Specifically, a logistic regression analysis requires that data from any given patient can be included only if all data from that patient are present; if there are any missing data from that patient, all of that patient's data must be excluded. Consequently, including an item for which there are data from only a few patients causes the entire regression analysis to be based on only a very few patients.

• 5

  Incorporate all questions for which more than 50 percent of the data were present into a logistic regression equation, as performed by SPSS 9.0. Conduct a separate multiple regression for each of the questionnaire subsections described in (3) above for each half of the database. The dependent variable in this regression was death within 1 year, a variable we created to identify all patients who died within 12 months of the study start date. The purpose of using this variable was to maximize the number of patients available for analysis, so that we could determine the maximum reliability provided by any relevant outcome measure. Death is an outcome measure of interest to SSA, and more data are available on this than on the outcome measures of employment status or self-reported ability to work at one year. Predictor variables were automatically entered into each equation (using SPSS statistical software) in a forward stepwise manner. In this method of regression, the variable with the greatest correlation with the dependent variable is entered first, that with the second greatest correlation entered second, and so on. Variables are entered into the regression until the point at which addition of another variable changes the log likelihood by less than 0.01 percent. This statistical technique "selects" only those variables that are correlated with death in 1 year. Variables that are not correlated with the dependent variables are not entered into the equation. Thus, one arrives at a set of variables that "predicts" death.
  In some cases, the relationship between a variable that was entered into the equation and the dependent variable was not statistically significant. Such variables were, for the purposes of this analysis, treated as if they were statistically significant and used in the regression described in Step (6), below.
  Only dichotomous variables were considered as independent categorical variables. Variables with three or more categories were not considered as categorical. Not classifying these latter variables as categorical has no effect on the statistical calculations.
  This step results in 6 multiple regression equations for each randomly selected half of the database.

• 6

  For each half of the database, incorporate all variables entered into preceding equations into another forward stepwise multiple logistic regression. The result is another set of variables that "predict" death in 1 year. This set, unlike the set described in the preceding step, is derived from all of the questions described in Step (3), above.

One disadvantage of this approach is that using several regressions and using forward stepwise regression increases the probability of obtaining a chance relationship between an independent variable and the dependent variable that is statistically significant. In statistical parlance, the strategy and techniques applied here maximize the probability of a Type 1 error. It is unlikely that this is a fatal flaw. It does not seem likely that the presence of chance relationships would mask truly large relationships between any given independent variable and the dependent variable.

Another disadvantage of the forward stepwise method that we employed is that it has no theoretical basis. However, as we noted above, published information about which variables one might preferentially wish to include in any regression equation is scarce.

Finally, we stress that we did not attempt to search for nonlinearity. This would not affect the results of comparisons of random halves of the database, but does mean that any results we present are only for purposes of validation.

Results

Table 8.Results of logistic regression on each DMMS (more...)

Table 8.Results of logistic regression on each DMMS Wave 2 medical questionnaire subsection

	First half of database			Second half of database
Database subsection	N selected a	N included b	Variables entered c	N selected a	N included b	Variables entered c
Patient and facility identification	688	629	Ethnicity (ns) d	659	601	First dialysis year(ns) d
Patient history	688	146	Limb amputation Absent foot pulse	659	251	Angina Cardiac arrest
Information at study start date	688	285	Independent eating	659	276	Daily dialysate volume Occupation Employment over past 2 years
Laboratory data	688	87	Serum albumin	659	88	No significant variables
Patient questionnaire	688	206	Physical functioning	659	212	Energy/fatigue
Medical care before first dialysis	688	70	Visit to nephrologist before ESRD	659	407	No significant variables

a

"N selected" refers to the number of patients whose data were randomly selected for inclusion into the analysis.

b

"N included" refers to the number of patients whose data were included in the multiple logistic regressions. These were patients who had data available for the predictor variables. The number of patients excluded from the analysis can be obtained by subtracting "N Included" from "N selected."

3

"Variables entered" denotes the significant variables included in each final regression equation.

d

"ns" denotes that this variable was entered into the regression equation even though its relationship with death at 1 year was not statistically significant.

The following table (Table 8) depicts the results of the comparison of the database halves. The important finding here is that for each random half of the database, each analysis entered different variables into the regression equation, indicating that for each random half of the database, different variables predicted death as an outcome. Such differences could result from the fact that there were a substantial number of patients from whom not all data were available (the impact of this is further discussed below).

Limitations of Univariate Analyses

Throughout this document, we have discussed the limitations inherent in univariate analyses, such that they do not account for multicollinearity among the independent/predictor variables. The following example relevant to this report illustrates the dangers in interpreting univariate statistics at face value.

One may notice that, among patients under age 65, those with a primary diagnosis of glomerulonephritis are significantly more likely to be working full time than are patients with diabetes. According to our statistics, 29.6 percent of patients with glomerulonephritis are working full time at the start of the study, versus 13.1 percent of diabetes patients. A cursory look at this statistic might lead one to prematurely conclude that patients with glomerulonephritis are not as sick as patients with diabetes. This conclusion is likely to be erroneous.

There are many ways in which glomerulonephritis patients differ from diabetes patients that may be affecting the rate of employment. One major difference is the average age of these two patient groups. Patients with glomerulonephritis have a mean age of 44 years, versus 51 years for patients with diabetes. This is a statistically significant difference (p <0.001) on a variable that can have a substantial impact on the likelihood of a person working. It may be the case that the diabetes group includes more patients over the age of 60 who are approaching retirement. Other variables that significantly differentiate these two patient groups include the following:

• Body-mass index (BMI) (glomerulonephritis higher)

• Hematocrit levels (diabetes higher)

• Presence of coronary artery disease (CAD) (more likely in diabetes)

• Presence of cerebrovascular disease (more likely in diabetes)

Of these variables, BMI, presence of CAD and cerebrovascular disease also significantly differentiate whether a person works full time.

All of these intertwined variables make it unclear whether patients with diabetes are less likely to work because of the diabetes itself, more severe symptomatology, coexisting diseases, older age, or a combination of severity of disease and age.

Another difficulty with using univariate tests to address questions about predictors of disability can arise when one analyzes the data as if persons with and without some characteristic were in two separate groups and then attempts to determine whether patients in these "groups" differ in their inability to work (however measured). This difficulty can be illustrated with the DMMS Wave 2 data. For this illustration, we used data from patients who are under 65 years of age, and who were employed or a student sometime during the 2 years prior to the start of the DMMS study. We then divided these patients into a "group" of patients with diabetes and another group of patients without diabetes. We determined whether one group was more likely to continue to work, as indexed by patients' answers on the followup questionnaire. These data are arrayed in the 2 x 2 table shown below:

Subjecting these data to a statistical analysis (here we use the odds ratio, but other statistics could also be used) yields an odds ratio of 1.75 with a 95 percent confidence interval of 1.12 to 2.65. Because this interval does not overlap 1.0, this odds ratio is statistically significant. As a result, it is tempting to conclude that patients with diabetes discontinue working at greater rates than those without diabetes and, therefore, that one can use presence of diabetes as a criterion for determining disability. This conclusion is, however, a poor one.

The flaw lies in the fact that the results of this group-based statistical test do not convey any information about how often this hypothetical criterion will lead one to a "correct" disability determination, or how often it will lead one to an "incorrect" determination. This is because this kind of comparison of two groups does not provide information about the diagnostic performance of this "test" for disability. To obtain information about performance, one needs to look at the results as if this were a diagnostic test. Thus, from the above table, one can compute that the sensitivity of diabetes for predicting disability is 39.23 percent, the specificity of this test is 72.78 percent, its positive predictive value is 70.34 percent, and its negative predictive value is 42.12 percent. Thus, the presence of diabetes is a fair indicator that a patient with ESRD will not continue to work (moderate positive predictive value), but the absence of diabetes is not a good indicator that a patient will continue working (low negative predictive value). In practical terms, using only the presence or absence of diabetes as a criterion for disability would appropriately provide disability benefits to patients with diabetes, but would also tend to inappropriately deny benefits to patients who do not have diabetes and who are also unable to work. To account for these latter patients, additional criteria (used in conjunction with diabetes) are needed.

In choosing these additional criteria, one does not want to choose any criterion that is highly correlated with the presence of diabetes. For example, imagine that there is a characteristic common to all persons with diabetes, and that this characteristic is not found in patients without diabetes (such as high glucose levels in the blood). Were we to use this characteristic as our second criterion for determining disability, the performance of the test would not change (i.e., the sensitivity, specificity, etc., of the test would be the same as if we used only diabetes to predict disability). The hypothetical second criterion does not provide any information beyond that provided by a diabetes diagnosis. Therefore, the second criterion that one chooses must have a low correlation with diabetes, but a high predictive value for inability to continue working.

Choosing a second criterion requires one to simultaneously consider its relationship with diabetes and with inability to continue working. Further, there is no guarantee that adding this second criterion will be sufficient to provide enough predictive value. One might need to use a "test" for disability that consists of three or more criteria. This means that choosing the third criterion involves simultaneously considering its relationship to the first two criteria and to the inability to continue working. These simultaneous considerations are best accomplished by using multivariate statistics.

The need for multivariate statistics is accentuated because choosing multiple criteria for a "test" for disability rapidly becomes very complex. Highlighting this complexity is that there is no characteristic of ESRD patients that is obviously correlated with an inability to continue working. (This is implied by the data shown in Appendix D.) These data also imply that one must examine a minimum of dozens of variables to arrive at a "test" for disability that appropriately awards benefits and does not inappropriately deny them.

Because of these complexities, we performed no univariate inferential statistical comparisons of the data we present below; we wish to minimize the possibility that a reader may come to erroneous conclusions about predictors of ability to work. Thus, we present the summary statistics below only for their potential use in future research.

Summary Statistics

Table 9. Employment status of patients in DMMS Wave (more...)

Table 9. Employment status of patients in DMMS Wave 2, under age 65

Criterion	Number meeting criterion	Number for whom data were available	Percent
Number employed full time 2 years to 6 months before dialysis initiation	899	2,145	41.9%
Number employed full time at start of study as recorded on medical or patient questionnaire	486	2,263	21.5%
Number employed full time at start of study as recorded on medical records	420	2,108	19.9%
Number employed full time at start of study as recorded on patient questionnaire	230	1,386	16.6%
Number working full time at followup on patient questionnaire	130	974	13.3%
Number of patients reporting able to work full time at start of study on patient questionnaire	301	1,363	22.1%
Number who are able to work full time at start of study who are working part time according to patient questionnaire	9	1,441	0.6%
Number able to work full time at followup on patient questionnaire	171	884	19.3%
Number of those able to work full time at followup who are not working full time according to patient questionnaire	31	874	3.5%
Number of those able to work full time at followup who are working part time	7	955	0.7%

Table 10. Statistics about patients who continue working (more...)

Table 10. Statistics about patients who continue working

Time of employment	Number employed full time	Total number for whom data are available	Percent
24 to 6 months before onset of dialysis	899	2,145	41.9%
Those employed 24 to 6 months (full or part time) who worked full time at onset of dialysis according to medical questionnaire	452	2,147	21.1%
Employed 24 to 6 months before (full or part time), at the onset of dialysis (full or part time), AND full time at 1-year followup	114	1,719	6.6%

Table 11. Working status by sex

Table 11. Working status by sex

		Employed throughout study
		Yes	No	Total
Male	Count	61	861	922
	% employed	6.62%	93.38%	100.00%
Female	Count	47	830	877
	% employed	5.20%	94.80%	100.00%
Total	Count	108	1,691	1,799
	% employed	5.92%	94.08%	100.00%

Table 12. Occupation type of individuals continuing (more...)

Table 12. Occupation type of individuals continuing to work full time throughout study

Occupation type	Frequency	Percent
Professional	52	48.6%
Clerical	18	16.8%
Tradesperson	15	14.0%
Manual labor	12	11.2%
Other	10	9.3%
Total	107	100%

Table 13. Comparison of education level of patients (more...)

Table 13. Comparison of education level of patients who worked throughout the duration of DMMS Wave 2 study versus those who did not

Education level		Continued working		Total
		Yes	No
Less than 12 yrs	Count	4	356	360
	% of this category in each working category	1.10%	98.90%	100.00%
High school grad	Count	26	465	491
	% of this category in each working category	5.30%	94.70%	100.00%
Some college	Count	29	229	258
	% of this category in each working category	11.20%	88.80%	100.00%
College grad	Count	45	139	184
	% of this category in each working category	24.50%	75.50%	100.00%
Total	Count	104	1189	1293
	% of this category in each working category	8.00%	92.00%	100.00%

Table 14. Self-reported ability to work full time at (more...)

Table 14. Self-reported ability to work full time at start of study versus employment status

		Employment status at start of study reported by patient
Able to work full time		Working full time	Working part time	In school	Keeping house	Retired	Unemployed, laid off, or looking for work	Disabled	None of the above	Total
Yes	Count	211	11	4	7	7	7	15	11	273
	% within each category of work status	77.30%	4.00%	1.50%	2.60%	2.60%	2.60%	5.50%	4.00%	100.00%
	% of each work status category reporting this ability to work	93.40%	17.70%	44.40%	6.30%	6.40%	20.60%	2.30%	12.40%	21.00%
No	Count	15	51	5	104	103	27	643	78	1,026
	% within each category of work status	1.50%	5.00%	0.50%	10.10%	10.00%	2.60%	62.70%	7.60%	100.00%
	% of each work status category reporting this ability to work	6.60%	82.30%	55.60%	93.70%	93.60%	79.40%	97.70%	87.60%	79.00%
Total	Count	226	62	9	111	110	34	658	89	1,299
	% within each category of work status	17.40%	4.80%	0.70%	8.50%	8.50%	2.60%	50.70%	6.90%	100.00%
	% of each work status category reporting this ability to work	100.00%	100.00%	100.00%	100.00%	100.00%	100.00%	100.00%	100.00%	100.00%

Table 15. Employment status versus self-reported ability (more...)

Table 15. Employment status versus self-reported ability to work at followup

Able to work full time at followup		Employment status at followup reported by patient								Total
		Working full time	Working part time	In school	Keeping house	Retired	Unemployed, laid off, or looking for work	Disabled	None of the above
Yes	Count	122	7	1	4	2	4	9	4	153
	% within each category of work status	79.70%	4.60%	0.70%	2.60%	1.30%	2.60%	5.90%	2.60%	100.00%
	% of each work status category reporting this ability to work	95.30%	17.10%	10.00%	6.30%	2.10%	15.40%	2.20%	6.70%	18.50%
No	Count	6	34	9	59	94	22	392	56	672
	% within each category of work status	0.90%	5.10%	1.30%	8.80%	14.00%	3.30%	58.30%	8.30%	100.00%
	% of each work status category reporting this ability to work	4.70%	82.90%	90.00%	93.70%	97.90%	84.60%	97.80%	93.30%	81.50%
Total	Count	128	41	10	63	96	26	401	60	825
	% within each category of work status	15.50%	5.00%	1.20%	7.60%	11.60%	3.20%	48.60%	7.30%	100.00%
	% of each work status category reporting this ability to work	100.00%	100.00%	100.00%	100.00%	100.00%	100.00%	100.00%	100.00%	100.00%

Table 16. Employment status at time of patient interview (more...)

Table 16. Employment status at time of patient interview about 1 month after study start date (according to medical records)

Self-reported employment status at study start date	Working status according to medical records									Total
		Working full time	Working part time	In school	Keeping house	Retired	Unemployed, laid off, or looking for work	Disabled	None of the above
Employed or student, full time	Count	187	21	7	6	1	4	36	15	277
	%	83.90%	27.30%	58.30%	5.10%	0.90%	10.50%	5.60%	15.50%	20.90%
Employed or student, part time	Count	16	34	1	6	2	2	27	7	95
	%	7.20%	44.20%	8.30%	5.10%	1.70%	5.30%	4.20%	7.20%	7.20%
Homemaker	Count	1	3		46	11	3	42	9	115
	%	0.40%	3.90%		39.00%	9.40%	7.90%	6.50%	9.30%	8.70%
Retired	Count	4	2		5	67		53	7	138
	%	1.80%	2.60%		4.20%	57.30%		8.20%	7.20%	10.40%
Never employed	Count	1			5		1	16	4	27
	%	0.40%			4.20%		2.60%	2.50%	4.10%	2.00%
Unemployed	Count	4	7	2	16	6	20	128	19	202
	%	1.80%	9.10%	16.70%	13.60%	5.10%	52.60%	19.80%	19.60%	15.20%
Disabled	Count	7	7	1	29	29	6	327	28	434
	%	3.10%	9.10%	8.30%	24.60%	24.80%	15.80%	50.60%	28.90%	32.70%
Other	Count	3	3	1	5	1	2	17	8	40
	%	1.30%	3.90%	8.30%	4.20%	0.90%	5.30%	2.60%	8.20%	3.00%
Total	Count	223	77	12	118	117	38	646	97	1328
	%	100.00%	100.00%	100.00%	100.00%	100.00%	100.00%	100.00%	100.00%	100.00%

NOTE: Numbers add to 100 percent in columns, not rows.

While Tables 9 to 16 provide interesting information about the working status of patients with ESRD, they do not provide information about which patients make up this small subset who continue to work full time. Appendix D provides additional information about these patients. These data are also limited because of their descriptive nature. Thus, it cannot be determined what causes the group differences seen in these tables.

Summary

As anticipated, the DMMS Wave 2 database indicates that only a very small number of patients continue to work full time once on dialysis. Out of almost 2,000 patients for whom data were available, only 114 worked full time continuously throughout the length of this study (more than 1 year). Almost every patient who reported being able to work full time was working full time; however, the significance of this finding is unclear.

There are many different variables in the DMMS Wave 2 database that differentiate those patients who continue to work full time from those who do not, but it is unclear how these predictor variables interact with one another and which of them accounts for the most variance (i.e., which among them is the "strongest" predictor).

DW2 M A. Patient and Facility Identification

DW2M B. Patient History Within 10 Years Prior to Study Start Date (date A7)

DW2M C: Information at Study Start Date (Date A7)

DW2M D: Laboratory Data

D2Q A. General Health

D2Q B. Medical Care Before Regular Dialysis

D2Q C. Choosing the Treatment for Your Kidney Failure

D2Q D. Transportation

D2Q E. Employment

D2Q F. Rehabilitation

Medical Update Questionnaire

Correlations Among Laboratory Values