II. Approach: Statistical Topics
-
General
- Systems Evaluation
- Treatment Effect
- Epidemiology Rates
- Risks
- Decision Analysis (Decision Tree, Chance Graph)
-
Screening Test
- Contingency Grid or Cross Tab (includes Statistics Example)
- Bayes Theorem (Bayesian Statistics)
- Fagan Nomogram
- Experimental Error (Experimental Bias)
- Lead-Time Bias
- Length Bias
- Selection Bias (Screening Bias)
- Likelihood Ratio (Positive Likelihood Ratio, Negative Likelihood Ratio)
- Number Needed to Screen (Number Needed to Treat, Absolute Risk Reduction, Relative Risk Reduction)
- Negative Predictive Value
- Positive Predictive Value
- Pre-Test Odds or Post-Test Odds
- Receiver Operating Characteristic
- Test Sensitivity (False Negative Rate)
- Test Specificity (False Positive Rate)
- U.S. Preventive Services Task Force Recommendations
III. Definitions: Statistical Significance
- Statistical Significance is the probability that study findings are due to chance
- The risk of rejecting the null hypothesis when it is actually true
- In planning a study, a level of risk is chosen (e.g. 5% risk or P-Value of 0.05)
- P-Value (a-value, level of significance)
- P-Value < 0.05: <5% that findings due to chance
- Reflects reproducibility of the study findings only
- Does not predict individual patient's effect
- Power
- Probability of detecting a difference of effect when one truly exists (true positive)
- Probability of rejecting the null hypothesis when it is truly false
- Probability of avoiding a Type 2 Error (B-error)
- Power = 1 - B
- Power = 80% (power is typically set at 80% in medical studies)
- Power is affected by mutiple factors
- Sample Size (key factor in increasing Power)
- Test Sensitivity
- Effect size of the data
- Level of significance (P-Value)
- Probability of detecting a difference of effect when one truly exists (true positive)
- Confidence Interval
- Provides a range of possible outcomes within which a true result of a statistical estimate will fall
- Example: Number Needed to Treat ranges from 20 to 100
- Confidence Intervals demonstrate the precision and accuracy of an outcome
- Confidence Intervals are typically set at 95% in medical studies
- If a study were to be repeated 100 times, the study results would fall in the given range 95% of the time
- Confidence Intervals identify factors that are most clinically relevant
- Risk ratios (RR), Odds Ratio (OR) and Hazard Ratio (HR) are positive if >1 and negative if <1
- Confidence Interval for RR, OR or HR that crosses 1.0 would be a statistically insignificant result
- Provides a range of possible outcomes within which a true result of a statistical estimate will fall
- Statistical Significance does not mean clinically useful
- See Clinical Significance below
IV. Definitions: Clinical Significance
- Reflects how much of an effect a patient sees
- Example:
- Study shows drug x significantly improves Hair Growth
- Reality: Even the patient cannot see the difference
V. Definitions: Heterogeneity
- Step 1: Clinical Heterogeneity (study similarity)
- Assess similarity of who and what was evaluated across pooled, meta-analyzed studies
- Evaluates similarity between studies (systematic review, meta-analysis)
- Combined studies in meta-analysis should have similar protocols (e.g. inclusion and exclusion criteria)
- Step 2: Statistical Heterogeneity
- Assess similarity among study results (were they consistent)
- Assign a P-Value to a combined group of studies that reflects the difference in their results and the likelihood that this is due to random chance
- Large spread of data across more than one study (i.e. greater heterogeneity) suggests the studies should not be combined in meta-analysis
- References
- Newman in Herbert (2013) EM:Rap 13(7): 7