Overview Data Pipeline Features Models Optimization SHAP

ML · Clinical AI

CRIS: Clinical Readmission
Intelligence System

End-to-end clinical ML pipeline predicting 30-day hospital readmission from the UCI Diabetes 130-US Hospitals dataset (101,766 records, 50 features). Integrates FHIR R4 clinical data ingestion, custom gradient descent from scratch, SMOTE-based class imbalance handling, Optuna hyperparameter tuning across 5 classifiers and 6 regressors, and SHAP interpretability, producing 43 output figures across all analytical stages.

PythonLightGBMXGBoost OptunaSHAPSMOTE FHIR R4Gradient Descentscikit-learn ICD-9Syntheaimbalanced-learn
101,766
Patient records
0.669
Test ROC-AUC
37
Engineered features
0.44
Recall @ thr=0.16
43
Output figures
5+6
Models compared
Background

The Clinical Problem

Unplanned 30-day hospital readmissions cost the US healthcare system an estimated $26 billion annually. The Centers for Medicare and Medicaid Services (CMS) penalizes hospitals with excess readmission rates under the Hospital Readmissions Reduction Program (HRRP). Diabetic patients are among the highest-risk populations for readmission due to comorbidity burden, medication complexity, and glycemic instability. Early identification of high-risk patients at discharge allows targeted interventions, reducing both patient harm and financial penalties.

Target Definition

Binary classification: did the patient return to any inpatient facility within 30 days of discharge? The UCI dataset encodes readmission as less than 30 days, greater than 30 days, or no readmission. We binarize: "less than 30 days" = positive class (readmitted), all others = negative. This produces an 11.16% positive rate, reflecting the real-world class imbalance inherent in population-level readmission data.

Why This Problem Is Hard

Administrative EHR features alone (age, primary diagnosis, medications, visit history) capture a limited fraction of readmission variance. Clinical severity scores (APACHE II, SOFA) and continuous lab trends, which would most strongly predict readmission, are absent. The dataset is also heavily right-skewed toward negative cases. The modeling challenge is therefore: maximize clinically useful recall (catching high-risk patients) while containing false positives to an actionable level for care teams.

Dataset Class Distribution
Raw class balance before any resampling (train split, 81,412 records)
Data

Data Sources and Schema

The backbone dataset is the UCI Diabetes 130-US Hospitals dataset covering 10 years (1999-2008) of inpatient encounters from 130 US hospitals. An optional FHIR R4 enrichment layer ingests Synthea-generated synthetic patient bundles to add Charlson Comorbidity Index, lab observations via LOINC codes, and medication flags from structured clinical resources.

UCI Diabetes Dataset

  • 101,766 encounters, 50 raw columns
  • Demographics: age bracket (10-year bins), gender, race
  • Utilization: number of inpatient, outpatient, and emergency visits in prior year
  • Clinical: time in hospital (LOS), number of diagnoses, primary ICD-9 codes (diag_1, diag_2, diag_3)
  • Medications: 23 binary medication columns encoding "Steady", "Up", "Down", "No"
  • Lab: HbA1c result, glucose serum result
  • Target: readmitted (<30 days / >30 days / NO)

FHIR R4 Enrichment Layer

  • Synthea-generated Bundle JSON files (patient-level clinical records)
  • Extracts Patient resource: age, gender, race from SNOMED codes
  • Extracts Encounter resource: LOS, admission month, day-of-week
  • Extracts Condition resource: Charlson CCI from SNOMED CT codes
  • Extracts Observation resource: 14 LOINC lab codes
  • Extracts MedicationRequest: medication count and insulin flag
  • Computes 30-day readmission target from sequential inpatient Encounter timestamps
Feature GroupRaw ColumnsAfter EngineeringKey Variables
Demographicsage, gender, raceage_numeric, race_enc10-year bracket midpoints (5-95)
Lab resultsA1Cresult, max_glu_serumabnormal_a1c, high_glucose, lab_risk_scoreOrdinal encoding + composite risk flag
Medication23 binary drug colsmeds_in_use, meds_changed, med_change_score, on_insulinCollapsed from 23 to 4 aggregate features
Diagnosis (ICD-9)diag_1, diag_2, diag_39 ICD group flags + charlson_approxCirculatory, Respiratory, Diabetes, Neurological, Musculoskeletal, Genitourinary, Digestive, Injury, Neoplasm
Utilizationnum_lab_procedures, num_procedures, num_medications, number_outpatient, number_emergency, number_inpatienttotal_prior_visits, emergency_visit_ratio, utilization_indexCaptures access and acuity patterns
InteractionsDerivedage_x_charlson, age_x_meds, visits_x_utilizationMultiplicative cross-features
Architecture

9-Step Pipeline

The pipeline is fully modular: each stage operates on the output of the previous and writes figures directly to outputs/figures/. Every stage is independently configurable via CLI flags.

Step 1
Data Ingestion
Load UCI CSV (101,766 x 50). Optional FHIR R4 parse via FHIRParser.
Step 2
EDA
Class distribution, feature distributions, correlation heatmap. 3 figures.
Step 3
Feature Engineering
7-step transform. 50 cols reduced to 37 clean features.
Step 4
Train/Test Split
80/20 stratified split. 81,412 train, 20,354 test. Shared feature matrix.
Step 5
Gradient Descent
Custom LogisticGD + LinearGD from scratch. Cosine LR schedule. Loss curves.
Step 6
Imbalance
SMOTE on train only. 81,412 → 144,652. 11% → 50% positive rate.
Step 7
Regression
6 models predict LOS. 3-fold CV on original X_tr. Optuna tunes all 6.
Step 8
Classification
5 models predict readmission. CV on X_tr (pre-SMOTE). Final fit on X_res.
Step 9
Interpretability
SHAP TreeExplainer: summary (beeswarm), waterfall, feature importance for RF, XGB, LGB.
Key design invariant: Cross-validation is always performed on the original, unbalanced training set (X_tr). SMOTE resampling is applied only once, after CV, immediately before the final .fit() call. This prevents synthetic minority samples from appearing in CV test folds, which would inflate AUC to an artifactual ~0.96.
Exploratory Analysis

EDA Findings

Three figures were generated during EDA: class distribution, feature distributions, and a correlation heatmap. Key findings informed both feature engineering and modeling strategy.

Class Imbalance

72,326 negative cases (88.84%) vs 9,086 positive cases (11.16%). The 8:1 ratio means a naive model predicting "never readmitted" achieves 88.8% accuracy, making accuracy a misleading metric. ROC-AUC and PR-AUC were chosen as primary evaluation criteria.

Age Distribution

Population clusters around the 70-80 year bracket, with the highest readmission rate in the 80-90 year group. Younger patients (under 50) show disproportionately high readmission risk relative to their overall admission count, suggesting different underlying drivers.

Medication Burden

Patients on more medications show higher readmission rates but also higher mortality proxies. The correlation between num_medications and readmission is positive but modest (r ≈ 0.12), motivating the composite med_change_score interaction feature.

Prior Utilization

number_inpatient (prior inpatient visits in last year) showed the strongest individual correlation with 30-day readmission. Patients with 2+ prior inpatient visits are 2.4x more likely to be readmitted.

LOS Distribution

Length-of-stay is right-skewed: median 4 days, long tail to 14+ days. Near-zero correlation between administrative features and LOS (confirmed in regression, R² ≈ 0) reflects that LOS is driven by clinical severity not captured in administrative coding.

Diagnosis Groups

ICD-9 analysis revealed circulatory conditions as the most prevalent comorbidity group (~38% of encounters), followed by respiratory (~22%) and diabetes-specific codes (~17%). Multi-system comorbidity was associated with 40% higher readmission odds.

Readmission Rate by Age Group
% readmitted within 30 days per 10-year age bracket
Comorbidity Group Prevalence
% of encounters with each ICD-9 diagnosis group present
Feature Engineering

From 50 Raw Columns to 37 Features

The FeatureEngineer class applies 7 sequential transformation steps, dropping 23 individual medication columns, all raw categorical identifiers, and the target column, collapsing 50 raw fields into 37 numerically encoded, interaction-enriched features.

Step 1-2

Cleaning and Demographic Encoding

Age brackets (e.g. "[70-80)") are mapped to numeric midpoints (5 to 95 in 10-year steps). A1c and glucose results are ordinal-encoded 0-3. Race is label-encoded. Gender becomes binary.

Step 3

ICD-9 Diagnosis Feature Extraction

The three raw diagnosis columns are each parsed against 9 ICD-9 code ranges to produce binary group flags. A Charlson approximation index is summed across matched comorbidity codes.

ICD-9 Groups Extracted
Circulatory (390-459, 785) Respiratory (460-519, 786) Diabetes (250.xx) Neurological (320-359) Musculoskeletal (710-739) Genitourinary (580-629, 788) Digestive (520-579, 787) Injury/Poisoning (800-999) Neoplasm (140-239)
Step 4-7

Medication Collapse, Utilization, Lab Flags, Interactions

23 individual drug columns collapsed to 4 aggregate features. Utilization index aggregates prior visit counts. Lab risk score combines A1c and glucose flags. Three multiplicative interaction features capture joint effects.

Medication Collapse (23 → 4)

meds_in_use: count where status ≠ "No"
meds_changed: count with "Up" or "Down"
med_change_score: sum of +1/"Up", -1/"Down"
on_insulin: binary insulin flag

Utilization Index

total_prior_visits = outpatient + emergency + inpatient
emergency_visit_ratio = emergency / (total + 1)
utilization_index = log1p(total) × (1 + emergency_ratio)

Interaction Features

age_x_charlson = age_numeric × charlson_approx
age_x_meds = age_numeric × meds_in_use
visits_x_utilization = total_prior_visits × utilization_index

Feature Reduction Summary
Input columns per feature group before and after engineering
From Scratch

Custom Gradient Descent Implementation

Two gradient descent models were implemented from scratch without sklearn: LogisticRegressionGD for binary classification (BCE loss) and LinearRegressionGD for LOS regression (MSE loss). Both share a base class with configurable learning rate schedules, mini-batch support, and Xavier weight initialization.

Architecture: _BaseGD

  • Weight init: Xavier uniform (scale = sqrt(6 / n_features))
  • LR schedules: constant, step decay, cosine annealing
  • Mini-batch generator with configurable batch size
  • Early stopping via absolute loss tolerance (tol=1e-6)
  • Full loss curve stored per epoch for plotting

LogisticRegressionGD

  • Loss: Binary Cross-Entropy + L2 regularization
  • Gradient: X.T @ (sigmoid(Xw) - y) / N + lambda*w
  • Cosine LR schedule: lr_t = 0.5 × lr × (1 + cos(πt / T))
  • Converged at epoch 363 (loss: 0.3388 → 0.3368)
  • Initial LR = 0.1, LR at epoch 300 = 0.0345
def _cosine_lr(self, epoch): t = epoch % self.T_max return 0.5 * self.lr * (1 + math.cos(math.pi * t / self.T_max)) def _gradient_logistic(self, X, y, w): logits = X @ w preds = self._sigmoid(logits) grad = X.T @ (preds - y) / len(y) return grad + self.lambda_ * w # L2 regularization term
Logistic GD: BCE Loss Curve
Training loss per epoch. Converged at epoch 363.
Cosine Learning Rate Schedule
LR decay: 0.1 → ~0.017 over T=500 epoch half-cycle
363
Convergence epoch (Logistic)
0.3388
Initial BCE loss
0.3368
Final BCE loss
Cosine
LR schedule used
8.97
Linear MSE @ epoch 400
Imbalance Handling

SMOTE Resampling

The 8:1 class imbalance makes raw training ineffective for detecting readmissions. SMOTE generates synthetic minority samples by interpolating between k-nearest neighbors in feature space. Applied only to the training set; test set remains untouched.

Before SMOTE

Not Readmitted72,326 (88.84%)
Readmitted9,086 (11.16%)

Total: 81,412 samples

After SMOTE

Not Readmitted72,326 (50.0%)
Readmitted72,326 (50.0%)

Total: 144,652 samples (+77.7%)

LightGBM: Precision vs Recall at Varying Thresholds
At default threshold 0.5, recall collapses to ~0.01. Lowering threshold to 0.16 recovers recall to 0.44 at cost of precision.

Strategies Evaluated

  • SMOTE: k=5 nearest neighbor interpolation in feature space. Selected for final pipeline.
  • ADASYN: Adaptive synthetic sampling, weights samples near decision boundary more heavily.
  • RandomUnderSampler: Randomly removes majority class samples. Fastest but loses information.
  • SMOTETomek: SMOTE followed by Tomek links removal to clean borderline cases.

Threshold Tuning Result (thr=0.16)

  • Precision: 0.21 (1 in 5 flagged patients is a true readmission)
  • Recall: 0.44 (catches 44% of all actual readmissions)
  • F1: 0.28
  • For a hospital discharging 200 diabetic patients weekly: flags ~47 high-risk patients, catching ~10 of the 22 who would be readmitted
Regression Task

Length-of-Stay Prediction

A secondary regression task predicts continuous Length-of-Stay (days) from the same feature set. This serves as a calibration task validating that features are correctly constructed and that zero predictive signal is a data-level finding.

ModelCV R² (3-fold)Test R²Test MAETest RMSE
Lasso-0.0001-0.00012.332.95
ElasticNet-0.0002-0.00012.332.95
Ridge-0.0005-0.00022.332.95
XGBoost-0.0065-0.00022.332.95
LightGBM-0.0057-0.00072.332.95
Random Forest-0.0485-0.00032.332.95
LOS Regression: CV R² by Model
All models produce R² ≈ 0. Administrative features cannot predict LOS; clinical severity scores are required.
Why R² ≈ 0 is expected: LOS is primarily determined by clinical severity (APACHE II score, organ dysfunction), complication events, and discharge logistics, none of which appear in administrative billing codes. The near-identical MAE/RMSE across all models (2.33 / 2.95) confirms that every model defaults to predicting the dataset mean. Optuna tuning 25 trials per model confirmed this ceiling.
Classification Task

30-Day Readmission Prediction

Five classifiers were evaluated via 3-fold stratified cross-validation on the original unbalanced training set, then re-fitted on SMOTE-balanced data for final test evaluation. Optuna TPE sampler with MedianPruner tuned each model for 25 trials with a 90-second timeout.

Model ROC-AUC: Baseline vs Optuna-Tuned vs Test
3-fold CV AUC on original training data. All values from actual pipeline run.

Baseline CV Results (3-fold, original data)

ModelCV ROC-AUCCV AUC SDCV F1CV PrecisionCV Recall
LightGBM0.66300.00040.0160.4970.008
XGBoost0.66200.00050.0220.4790.011
Decision Tree0.64140.00290.0470.3550.025
Logistic Regression0.63660.00380.0280.4830.015
Random Forest0.61820.00400.0190.4280.010

Optuna Tuning Impact

ModelBaseline AUCOptuna AUCDeltaTrials
Random Forest0.61820.6519+5.4%6
XGBoost0.66200.6685+1.0%25
LightGBM0.66300.6676+0.7%18
Decision Tree0.64140.6466+0.8%25
Logistic Regression0.63660.6372+0.1%10
Random Forest gained the most (+5.4% AUC) because its baseline used conservative defaults (n_estimators=80, no max_depth constraint). Optuna discovered max_depth=22, n_estimators=300, max_features=sqrt as the optimal configuration in just 6 trials before hitting the 90-second timeout.

Final Test-Set Results (Optuna-Tuned, Threshold = 0.5)

ModelROC-AUCPR-AUCF1PrecisionRecallMCCBrier
LightGBM0.66950.2170.0220.5310.0110.0650.095
XGBoost0.65800.2100.0270.4630.0140.0640.097
Logistic Regression0.64700.2000.2600.1730.5230.1400.231
Random Forest0.63980.1870.0310.4300.0160.0660.111
Decision Tree0.61570.1580.2220.1800.2890.1020.135
Multi-Metric Comparison (Test Set, Threshold = 0.5)
ROC-AUC, PR-AUC and Brier score across all five classifiers

Best Model: LightGBM at Threshold 0.16

ClassPrecisionRecallF1Support
Not Readmitted (0)0.920.790.8518,083
Readmitted (1)0.210.440.282,271
Accuracy0.7520,354
Macro Avg0.560.610.56
Weighted Avg0.840.750.78
Clinical interpretation: At threshold 0.16, the model flags approximately 1 in 5 discharges as high-risk. Of those flagged, 21% are true readmissions. It captures 44% of all actual 30-day readmissions. For a hospital discharging 200 diabetic patients weekly, this system would flag ~47 patients for care coordinator follow-up, catching approximately 10 of the 22 patients who would otherwise be readmitted within 30 days.
Hyperparameter Tuning

Optuna TPE Search Strategy

Optuna v4.9.0 with Tree-structured Parzen Estimator (TPE) sampler and MedianPruner was used to search hyperparameter spaces for all 11 models. Each trial ran a 3-fold CV on the original training set; trials exceeding the median score at the first fold were pruned early.

Optuna Trials Used per Model
Models hitting 90s timeout before 25 trials indicate high search cost
AUC Gain from Optuna Tuning
Absolute AUC improvement over baseline CV for each classifier

LightGBM Search Space

  • n_estimators: [100, 400]
  • learning_rate: [0.01, 0.3] log-uniform
  • num_leaves: [20, 200]
  • min_child_samples: [5, 80]
  • subsample, colsample_bytree: [0.5, 1.0]
  • reg_alpha, reg_lambda: [1e-3, 10] log-uniform

XGBoost Search Space

  • n_estimators: [100, 400]
  • learning_rate: [0.01, 0.3] log-uniform
  • max_depth: [3, 12]
  • subsample, colsample_bytree: [0.5, 1.0]
  • reg_alpha, reg_lambda: [1e-3, 10] log-uniform
  • gamma: [0, 5], min_child_weight: [1, 10]
Threading note (Windows): Running cross_validate(n_jobs=-1) with LGBMClassifier(n_jobs=-1) deadlocks on Windows due to OpenMP vs loky process pool conflict. Fix: cross_validate(n_jobs=1) with each model using internal thread parallelism via n_jobs=4 / nthread=4.
Interpretability

SHAP Explanations

SHAP TreeExplainer was applied to all three tree-based models (Random Forest, XGBoost, LightGBM), generating beeswarm summary plots, individual waterfall plots, and bar-chart feature importance figures. Nine total figures were produced.

SHAP Feature Importance (LightGBM, Top 12 Features)
Mean absolute SHAP value across test set. Larger value = stronger average impact on readmission prediction.
SHAP confirms clinical priors: The top features by mean |SHAP value| were number_inpatient (prior inpatient visits), discharge_disposition_id, age_x_charlson (age-comorbidity interaction), total_prior_visits, and on_insulin. This aligns with the clinical readmission literature, where prior utilization and comorbidity burden are consistently the strongest administrative predictors.
Top Feature: number_inpatient

Prior inpatient admissions drive the largest share of prediction separation. Patients with 2+ prior inpatient visits show strongly positive SHAP values across all three tree models. Prior admission history is the strongest known administrative readmission predictor in the clinical literature.

age_x_charlson Interaction

The multiplicative age x comorbidity interaction was among the top 5 SHAP features across all models. High values (old patients with high comorbidity burden) produced the largest positive SHAP contributions, confirming the frailty signal independently captured by this engineered feature.

on_insulin Flag

Patients on active insulin therapy show systematically higher SHAP values. Insulin use is a proxy for Type 1 diabetes or poorly controlled Type 2, both associated with higher complication rates during inpatient stays and at discharge.

med_change_score Direction

Medication decreases (negative med_change_score) were associated with positive readmission SHAP values more often than increases. Downward titration at discharge may reflect clinical instability or end-of-episode cost management.

LightGBM vs XGBoost Agreement

LightGBM and XGBoost produced nearly identical SHAP rankings (Spearman r > 0.95 across top-20 features), suggesting both models converged to the same dominant signal. The small AUC gap (0.6695 vs 0.6580) reflects minor differences in leaf-level split optimization.

discharge_disposition_id

Discharge destination (home vs skilled nursing vs home health care) showed strong SHAP signal. Patients discharged to skilled nursing facilities or with home health orders had higher readmission risk, reflecting post-acute care instability and underlying functional decline.

Clinical Standards

FHIR R4 Ingestion Layer

The FHIRParser module implements a production-style ingestion pattern for HL7 FHIR R4 Bundle JSON files. While the UCI dataset is the primary data source, the FHIR layer demonstrates real-world EHR integration capability and enables richer feature extraction from structured clinical resources.

Input
Synthea Bundles
FHIR R4 JSON Bundle files. One bundle per patient. Generated via Synthea CLI.
Parse
Resource Extraction
Patient, Encounter, Condition, Observation, MedicationRequest resources extracted.
Compute
Clinical Indices
Charlson CCI from SNOMED codes. LOINC lab values. 30-day readmission target from sequential encounters.
Output
Feature DataFrame
Structured DataFrame aligned with UCI feature schema for merge or standalone use.

SNOMED CT: Charlson Comorbidity Index

The Charlson CCI is computed from Condition resource SNOMED codes, mapping coded diagnoses to 17 Charlson domains (myocardial infarction, CHF, peripheral vascular disease, dementia, COPD, rheumatoid disease, peptic ulcer, mild liver disease, diabetes, etc.). The weighted sum provides a standardized comorbidity burden score used clinically to estimate 10-year mortality.

LOINC Observation Extraction

14 LOINC codes are extracted from Observation resources: HbA1c (4548-4), serum creatinine (2160-0), BUN (3094-0), eGFR (33914-3), sodium, potassium, glucose (2339-0), hemoglobin, WBC, platelet count, ALT, AST, albumin, and total bilirubin. These enable a richer lab feature set than the UCI dataset's single A1c and glucose columns.

Findings and Next Steps

Limitations and Future Work

Known Limitations

  • Administrative features explain a limited fraction of 30-day readmission variance. AUC 0.67 is consistent with published literature for administrative-only datasets (typical range: 0.60-0.72).
  • Length-of-Stay is unpredictable (R² ≈ 0) without clinical severity scores (APACHE II, SOFA). This is a fundamental data limitation.
  • The UCI dataset covers 1999-2008. Medication coding, discharge practices, and diagnostic coding have changed significantly.
  • FHIR enrichment requires Synthea-generated data rather than real patient records, limiting real-world validation.
  • SMOTE assumes smooth decision boundaries in feature space, which may not hold for highly non-linear clinical data.

Proposed Extensions

  • Clinical severity integration: Add APACHE II or NEWS2 scores at admission to unlock the primary LOS and readmission signal missing from administrative codes.
  • Temporal modeling: Model the admission sequence as a time series using LSTM or Transformer architecture to capture trajectory effects across multiple encounters.
  • Real FHIR deployment: Connect to a live FHIR R4 server (SMART on FHIR) for real-time inference at point of discharge.
  • Fairness audit: Evaluate model performance stratified by race and gender to detect differential performance on protected groups.
  • Calibration: Apply Platt scaling or isotonic regression to convert raw LightGBM probability outputs to calibrated clinical risk scores.
Technologies

Full Technical Stack

Data and Features

  • pandas, numpy: ETL and feature matrix
  • scikit-learn: StandardScaler, Pipeline, StratifiedKFold, cross_validate
  • scipy: Statistical utilities
  • ICD-9 / SNOMED CT / LOINC: Clinical code systems
  • HL7 FHIR R4: Clinical interoperability standard

Modeling

  • scikit-learn: LogisticRegression, DecisionTree, RandomForest, Ridge, Lasso, ElasticNet
  • xgboost: XGBClassifier, XGBRegressor
  • lightgbm: LGBMClassifier, LGBMRegressor
  • imbalanced-learn: SMOTE, ADASYN, SMOTETomek
  • optuna 4.9.0: TPE + MedianPruner

Interpretability and Visualization

  • shap: TreeExplainer, summary_plot, waterfall_plot
  • matplotlib + seaborn: 43 static publication figures
  • plotly + kaleido: Optuna history and param importance PNGs
  • Custom GD: Pure NumPy, no sklearn dependency