#083 — DeLong's Test for Correlated ROC AUC Curves [083_DeLong_s_Test_for_Correlated_ROC_AUC_Curves.ipynb]
Statistical Definition
Non-parametric test comparing the Areas Under two correlated Receiver Operating Characteristic (ROC) curves evaluated on the same patient cohort.
Bioinformatics Use Case
Proving that a novel 5-gene diagnostic signature achieves significantly higher AUROC than standard clinical PSA or CA-125.
Input Variables
True binary labels (y); continuous risk scores from Model A (s1) and Model B (s2).
Application
To win diagnostic adoption, you must prove your biomarker beats the current clinical standard on the exact same patients. DeLong's test provides that definitive head-to-head proof.
#084 — Hosmer-Lemeshow Calibration Goodness-of-Fit Test
Statistical Definition
Groups predicted probabilities into deciles and applies a Chi-square test comparing observed event counts against expected predicted probabilities.
Bioinformatics Use Case
Validating clinical risk-score calibration so that a predicted 30% recurrence risk truly corresponds to 30% observed patient relapse.
Input Variables
Binary outcome labels (y); predicted probabilities (p_hat) in (0, 1).
Application
A diagnostic model with high AUC can still dangerously over- or under-predict patient risk. Hosmer-Lemeshow certifies that your model's predicted probabilities are clinically calibrated.
#085 — Cohen's Kappa Inter-Rater Agreement Test
Statistical Definition
Measures categorical agreement between two raters or classifiers beyond chance agreement and tests whether kappa > 0.
Bioinformatics Use Case
Evaluating concordance between digital pathology AI tumor grading and board-certified pathologist scoring.
Input Variables
Square k x k confusion matrix of categorical assignments from two raters.
Application
Demonstrate that your automated AI classifier matches board-certified pathologists—accounting for chance agreement—with Cohen's Kappa.
#086 — Fleiss' Kappa Multi-Rater Agreement Test
Statistical Definition
Generalizes Cohen's Kappa to assess consistency of categorical ratings across three or more independent raters or algorithms.
Bioinformatics Use Case
Evaluating consensus subtype classification across 5 independent clustering algorithms or multiple clinical centers.
Input Variables
Matrix of N subjects (rows) by k categories (columns) containing rater counts per category.
Application
When harmonizing consensus molecular subtypes across multiple consortia or algorithms, Fleiss' Kappa quantifies multi-rater concordance in a single metric.
#087 — Brier Score & Calibration Slope Permutation Test
Statistical Definition
Evaluates probabilistic prediction accuracy via mean squared probability error and tests improvement over a null baseline model.
Bioinformatics Use Case
Benchmarking probabilistic clinical risk models where both discrimination and probability accuracy are critical.
Input Variables
Binary outcomes (y); predicted probabilities (p_hat).
Application
Brier Score rewards models that are both accurate and well-calibrated—proving your risk algorithm outperforms baseline clinical heuristics.
#088 — Matthews Correlation Coefficient (MCC) Significance Test
Statistical Definition
Balanced classification correlation metric in [-1, +1] that remains reliable under extreme class imbalance, tested via Chi-square equivalence (N * MCC^2).
Bioinformatics Use Case
Evaluating rare variant callers or disease screening classifiers where positive cases represent <5% of samples.
Input Variables
2x2 confusion matrix [[TP, FN], [FP, TN]].
Application
Accuracy is meaningless when 98% of samples are negative. MCC delivers a single balanced score from -1 to +1 with an exact p-value for imbalanced diagnostic