II. Technique: Setting up grid for test efficacy or risk factor
- Examples
- Test efficacy: How well does a test detect a certain condition
- Risk Factor: How much is a particular risk associated with a given condition
- Draw 2x2 grid
- Labels
- Upper Boxes: Across the top (x-axis) place the Disease State Labels
- Left Box: Disease present (e.g. Breast Cancer)
- Right Box: Disease not present (e.g. Not Breast Cancer)
- Left Boxes: Across the left (y-axis) place the Test Result Labels
- Upper Box: Test Positive, Screened or exposed to contributing factor
- Lower Box: Test Negative, Not screened, no exposed
- Upper Boxes: Across the top (x-axis) place the Disease State Labels
III. Example: Breast Cancer Screening with Mammogram
- Given
- Risk of Breast Cancer based on age
- Age 40 years old: 1 in 69
- Age 50 years old: 1 in 42
- Age 60 years old: 1 in 29
- Mammogram efficacy
- Note: We use the upper end of the Test Sensitivity and Specificity ranges for this example
- Test Sensitivity: 77-95%
- Test Specificity: 94-97%
- Risk of Breast Cancer based on age
- Create a hypothetical grid for patients age 40 who undergo Mammograms
- Generating example data
- Of 100,000 patients, 1449 will have Breast Cancer (1 in 69)
- Of the 1449 with Breast Cancer, 1376 will be detected with Mammogram (95% Test Sensitivity)
- Of the 98,551 without Breast Cancer, 95,594 will have a normal Mammogram (97% Specificity)
- Label the grid top
- Disease Positive (or D+): Breast Cancer positive
- Disease Negative (or D-): Breast Cancer negative
- Label the grid left
- Fill in total patients first (bottom row)
- Breast Cancer positive (D+): 1449
- Every 69 in 100,000 will have Breast Cancer for those at age 40
- Breast Cancer negative (D-): 98,551
- The remainder of the 100,000 without Breast Cancer
- Breast Cancer positive (D+): 1449
- Complete the left column (D+)
- Top left: Mammogram Positive (or T+): 1376
- True positive patients represent 95% of 1449 (the Test Sensitivity)
- Bottom left: Mammogram Negative (or T-): 73
- False Negative patients represents 1449 - 1376
- Top left: Mammogram Positive (or T+): 1376
- Complete the right column (D-)
- Bottom right: Mammogram Negative (or T-): 95,594
- True negative patients represents 97% of 98,551 (the Test Specificity)
- Top right: Mammogram Positive (or T+): 2957
- False Positive patients represents 98,551 - 95,594
- Bottom right: Mammogram Negative (or T-): 95,594
- Generating example data
- Summary of grid
- D+ T+: 1376 (true positives)
- D- T+: 2957 (False Positives)
- D+ T- : 73 (False Negatives)
- D- T- : 95,594 (true negatives)
- Calculations
- Test Sensitivity (Test Recall)
- Sensitivity: True positives / (true positives + False Negatives)
- Sensitivity: 1376 / (1376 + 73) = 95%
- Test Specificity
- Specificity: True negatives / (true negatives + False Positives)
- Specificity: 95,594 / (95,594 + 2957) = 97%
- Positive Predictive Value (PPV, Test Precision)
- PPV: True positive / (true positives + False Positives)
- PPV: 1376 / (1376 + 2957) = 32%
- Negative Predictive Value (NPV)
- NPV: True negative / (true negative + False Negatives)
- NPV: 95,594 / (95,594 + 73) = 99%
- False Positive Rate (type I error or a)
- a: (1 - Test Specificity)
- a: (1 - 0.97) = 3%
- False Negative Rate (type II error or b)
- b: (1 - Test Sensitivity)
- b: (1 - 0.95) = 5%
- Likelihood Ratio positive (LR+)
- LR+: Sensitivity / (1-Specificity)
- LR+: 0.95 / (1 - 0.97) = 32 (high likelihood of disease if >10)
- Likelihood Ratio negative (LR-)
- LR-: (1 - Sensitivity) / (Specificity)
- LR-: (1 - 0.95) / 0.97 = 0.05 (low likelihood of disease if <0.1)
- F1 Score
- F1 Score is the harmonic mean of Test Precision (PPV) and Test Recall (Test Sensitivity)
- F1 is least accurate at 0, and most accurate at 1
- F1 = 2 * (Precision * Recall) / (Precision + Recall)
- F1 = 2 * (0.32 * 0.95)/(0.32 + 0.95) = 0.47
- Test Sensitivity (Test Recall)