Screening: Print Module


Introduction

Screening is the examination of asymptomatic people in order to classify them as likely or unlikely to have a particular disease. Epidemiologists study the screening of diseases for two reasons: the first is to determine the validity of a screening test, and the second is to evaluate the effectiveness of a screening test in a population. This exercise will lead you through the design and analysis of a study to determine the validity of a screening test in Epiville. At the conclusion of this exercise, you will read an article and answer questions about a study designed to evaluate the effectiveness of a screening test in a population.

Faculty Highlight: Ruby Senie

Dr. Ruby Senie is Professor of Clinical Epidemiology and Sociomedical Sciences in the Mailman School of Public Health.

Dr. Senie has devoted more than 20 years to research on breast cancer risk and prognostic factors. Dr. Senie is principal investigator of an NCI-funded project titled the "Metropolitan New York Registry of families at increased risk of breast cancer." Dr. Senie also has a special interest in racial/ethnic disparities associated with detection and prognosis of breast and other cancers, and is principal investigator of the NY component of the NCI-funded Asian American Network for Cancer Awareness Research and Training, which has supported outreach to community-based organizations to provide educational programs and opportunities for cancer screening.

Read more about Dr. Senie's work:

  1. Islam, N. Kwon, SC, Senie R, Kathuria, N. Breast and cervical cancer screening among South Asian women in New York City. Journal of Immigrant and Minority Health. July 2006 8(3): 211-2.
  2. Senie, R. Santella, R. Ahsan, H. The metropolitan New York Registry and CFRBCS. Unique resources for breast cancer research. Annals of Epidemiology. October 1 2000. 10(7):462.

Learning Objectives


Student Role

A goal of the Epiville Department of Health (EDOH) is to reduce the morbidity caused by the neurological disorder Susser Syndrome (SS) . A drug treatment has been found to be successful in alleviating the symptoms of Susser Syndrome in patients that already have clinical presentation and diagnosis of the disease. Now there is reason to believe that if the treatment is begun before clinical symptoms first appear, the treatment can prevent irreversible neurological damage and slow down the progression of the disease accompanied by the worsening of the neurological symptoms. In order to start treatment before the clinical symptoms of Susser Syndrome appear, it is necessary to implement a screening test to identify persons in the detectable pre-clinical phase. A laboratory test called SussStat has been developed that measures a biomarker for asymptomatic neurological damage. The director of the EDOH, Dr. Melissa Zapp, suggests that your next project should be to conduct a study to see how well this new test identifies persons with Susser Syndrome prior to the symptomatic stage.


Study Design

Before you can begin your study, you must write a proposal to secure funding for the project. In the proposal, you must describe how a screening program would fit into the prevention activities of the EDOH.

1. A screening program for Susser Syndrome would fall under what type of prevention activity?

  1. Primary
  2. Secondary
  3. Tertiary
Answer (a) — incorrect: The disease process would have already been initiated among the individuals identified by the screening program. Primary prevention is characterized by individual or community efforts to stop the disease process from initiating (Aschengrau & Seage p. 413-14).
Answer (b) — correct: The rationale for the screening program is that it will identify asymptomatic persons with Susser Syndrome and provide early treatment so that the expression of the disease will be less severe, which is characteristic of secondary prevention (Aschengrau & Seage p. 414-15).
Answer (c) — incorrect: A tertiary prevention program is implemented after a clinical diagnosis has been confirmed to alleviate suffering and minimize complications of the disease. The screening program you hope to design for Susser Syndrome will identify persons who are asymptomatic and have not yet been diagnosed.

As luck would have it, your study is funded by a generous grant from DoseEmAll Pharmaceuticals. You plan to recruit 1000 subjects with no symptoms of Susser Syndrome. Since you want to ensure that enough individuals in your study will develop Susser Syndrome, you decide to recruit workers from Glop Industries, where an agent suspected to cause Susser Syndrome is manufactured. The EDOH has conducted a study which demonstrated a relationship between exposure to chemicals involved in the production of Superclean and a higher risk of Susser Syndrome.

At enrollment, you collect a blood sample from each participating subject to perform the new SussStat test. One year later, you have each subject return to undergo thorough clinical assessment for SS, using the gold standard of diagnosis set by the EDOH.


Data Collection

The SussStat procedure tests for irregularities in the DNA, called DNA adducts, that are formed when a chemical contained in Glop Indutstry's Superclean product bonds to DNA. It is this process which researchers suspect may lead to Susser Syndrome.

You receive a spreadsheet with the DNA adduct data from the laboratory, and now must use these data to categorize each subject into the categories of 'Test Positive' and 'Test Negative' for Susser Syndrome. The manufacturer of SussStat suggests that levels of DNA adducts lie on a continuum. The manufacturer suggests dichotomizing SussStat level at 12 × 10-8. Those who test at 12 × 10-8 or higher will be considered screened "positive," and everyone else will be considered screened "negative." You use this manufacturer-suggested cutpoint to categorize your subjects into Test Positive and Test Negative for SS.

When each subject returns for the one-year clinic visit, the clinician evaluates them and uses the diagnostic criteria put forth by the EDOH to categorize each subject into Disease Positive and Disease Negative. After receiving these data from the study physician, you are now ready to compare the SussStat results to the Clinician Gold Standard! In order to prepare the data for analysis, you classify each subject into a 2×2 table:

  
Gold Standard
 
  PositiveNegativeTotal
SussStatPositive9040130
 Negative10860870
  1009001000

[ Click here for the Interactive Exercise. ]


Data Analysis

  
Gold Standard
 
  PositiveNegativeTotal
SussStatPositive9040130
 Negative10860870
  1009001000

You are now ready to compute some statistics that will tell you how well SussStat performs compared to the gold standard of clinical diagnosis. Sensitivity and specificity are commonly used measures of the validity of a screening of a test (Aschengrau & Seage, pp. 421-422). Validity is the ability of a test to correctly categorize persons into their true disease status.

The measures of positive predictive value (PPV) and negative predictive value (NPV) describe how well a positive screening test result predicts presence or absence of a disease in a particular population. The PPV and NPV are measures of a screening program's feasibility (Aschengrau & Seage, p. 423).

2. Calculate the sensitivity, specificity, PPV, and NPV.

a. Sensitivity = Answer — 0.90
b. Specificity = Answer — 0.96
c. PPV = Answer — 0.69
d. NPV = Answer — 0.988

3. Which is the best interpretation of the sensitivity of SussStat?

  1. The probability that SussStat correctly categorized an individual as not having Susser Syndrome.
  2. Of those who tested positive on the SussStat test, the percent of persons that developed Susser Syndrome.
  3. The probability that SussStat correctly categorized an individual as having Susser Syndrome.
Answer (a) — incorrect: This is an interpretation of specificity.
Answer (b) — incorrect: This is the PPV of the test, PPV = TP/(TP+FP). The denominator of sensitivity is not those who tested positive, but rather those who truly have the disease.
Answer (c) — correct: Sensitivity = TP/(TP+FN).

4. Which is the best interpretation of the specificity of SussStat?

  1. The probability of obtaining a false negative if you truly do not have Susser Syndrome.
  2. Of those who never develop Susser Syndrome, the percent that tested negative on the SussStat essay.
  3. The probability that SussStat correctly categorized an individual as having Susser Syndrome.
Answer (a) — incorrect: It is the probability that SussStat correctly categorized an individual as having Susser Syndrome, or the probability of obtaining a true negative.
Answer (b) — correct: Specificity= TN / (TN + FP).
Answer (c) — incorrect: Specificity= TN / TN + FP.

5. How could you interpret the positive predictive value (PPV)?

  1. Of those persons who developed Susser Syndrome, the percent that tested positive on the SussStat test.
  2. The probability of not developing Susser Syndrome given a negative SussStat test result.
  3. Of those who tested positive, the percent that develop Susser Syndrome.
Answer (a) — incorrect: The is the interpretation of sensitivity.
Answer (b) — incorrect: This is the NPV
Answer (c) — correct: PPV = TP/(TP+FP).

You are concerned that there are too many false positives when the manufacturer-suggested cutpoint of DNA adducts is used to define the pre-clinical SS and you want to see if you can reduce the number by changing the cutpoint, or criterion of positivity, of the SussStat test. Click here for an interactive exercise demonstrating how raising or lowering the cutpoint changes the measures you calculated for SussStat.

[ Click here for the Interactive Exercise. ]

6. What is a consequence if SussStat has a low sensitivity?

  1. You will miss the opportunity to correctly diagnose and treat people with Susser Syndrome.
  2. You will treat too many people who don't actually have the disease, which is costly and stressful to the subjects and also puts people at risk from possible side effects.
  3. Both a and b.
Answer (a) — correct: A low sensitivity means a lot of false negatives.
Answer (b) — incorrect: This is the consequence of a low specificity.
Answer (c) — incorrect: Only one of the answers listed is correct.

You are happy with the screening test's ability to identify Susser Syndrome, but you are now considering to what groups in Epiville you should target your screening program.

7. Using the sensitivity and specificity measures you calculated in Question 2 above, calculate what the PPV would be if you screen the total population of Epiville where you estimate the prevalence of SS to be only 1%. (hint: draw a new 2x2 table and write the new column totals for the Clinician Gold Standard using the new "true" population prevalence of 1%, given a total N=1000 Then use the specificity and sensitivity proportions you calculated previously to fill in the rest of the 2x2 table.) NOTE: ONLY USE THE SENSITIVITY AND SPECIFICITY VALUES ROUNDED TO 2 DECIMAL PLACES THAT APPEAR IN THE ANSWER TO QUESTION 2.

Answer = Correct answer — 0.18

You make note of the fact that as the prevalence of the disease decreases, the PPV of the screening test decreases. You recommend to the EDOH that a screening program be introduced among workers of the Glop Industries since they have a higher prevalence of the disease, and therefore SussStat will be most effective in that group because it will detect a larger proportion of actual cases among individuals with positive results (Aschengrau & Seage, p. 424).

The measures you calculated above describe the validity of the SussStat test. In contrast, reliability is the ability of a test to give the same result on repeated testing, i.e., consistency. (Aschengrau & Seage, p. 419) Reliability can also be computed to describe the extent to which two tests agree with each other. A common measure of reliability is the kappa statistic.

Intellectually curious?

Learn how to calculate Kappa.


Discussion Questions

Carefully consider the following questions. Write down your answers (1 - 2 paragraphs) for question # 1 within a word document and submit your answers to your seminar leader. Be prepared to discuss all questions during the seminar section.

  1. Think of hypothetical screening programs for diseases of interest to you; find one example of a disease where high sensitivity would be more important, and one example where high specificity would be more important.
  2. You have probably had the experience of being screened many times (e.g., cholesterol test, glucose level test, blood pressure measurement). What is more important information from a PATIENT's point of view (PPV/NPV vs. Sensitivity/Specificity), and what is more important information from a PHYSICIAN's point of view (PPV/NPV vs. Sensitivity/Specificity).
  3. What are some strategies to reduce lead time bias in studies evaluating the efficacy of screening programs?