CRIS: Clinical Readmission
Intelligence System
End-to-end clinical ML pipeline predicting 30-day hospital readmission from the UCI Diabetes 130-US Hospitals dataset (101,766 records, 50 features). Integrates FHIR R4 clinical data ingestion, custom gradient descent from scratch, SMOTE-based class imbalance handling, Optuna hyperparameter tuning across 5 classifiers and 6 regressors, and SHAP interpretability, producing 43 output figures across all analytical stages.
The Clinical Problem
Unplanned 30-day hospital readmissions cost the US healthcare system an estimated $26 billion annually. The Centers for Medicare and Medicaid Services (CMS) penalizes hospitals with excess readmission rates under the Hospital Readmissions Reduction Program (HRRP). Diabetic patients are among the highest-risk populations for readmission due to comorbidity burden, medication complexity, and glycemic instability. Early identification of high-risk patients at discharge allows targeted interventions, reducing both patient harm and financial penalties.
Target Definition
Binary classification: did the patient return to any inpatient facility within 30 days of discharge? The UCI dataset encodes readmission as less than 30 days, greater than 30 days, or no readmission. We binarize: "less than 30 days" = positive class (readmitted), all others = negative. This produces an 11.16% positive rate, reflecting the real-world class imbalance inherent in population-level readmission data.
Why This Problem Is Hard
Administrative EHR features alone (age, primary diagnosis, medications, visit history) capture a limited fraction of readmission variance. Clinical severity scores (APACHE II, SOFA) and continuous lab trends, which would most strongly predict readmission, are absent. The dataset is also heavily right-skewed toward negative cases. The modeling challenge is therefore: maximize clinically useful recall (catching high-risk patients) while containing false positives to an actionable level for care teams.
Data Sources and Schema
The backbone dataset is the UCI Diabetes 130-US Hospitals dataset covering 10 years (1999-2008) of inpatient encounters from 130 US hospitals. An optional FHIR R4 enrichment layer ingests Synthea-generated synthetic patient bundles to add Charlson Comorbidity Index, lab observations via LOINC codes, and medication flags from structured clinical resources.
UCI Diabetes Dataset
- 101,766 encounters, 50 raw columns
- Demographics: age bracket (10-year bins), gender, race
- Utilization: number of inpatient, outpatient, and emergency visits in prior year
- Clinical: time in hospital (LOS), number of diagnoses, primary ICD-9 codes (diag_1, diag_2, diag_3)
- Medications: 23 binary medication columns encoding "Steady", "Up", "Down", "No"
- Lab: HbA1c result, glucose serum result
- Target: readmitted (<30 days / >30 days / NO)
FHIR R4 Enrichment Layer
- Synthea-generated Bundle JSON files (patient-level clinical records)
- Extracts Patient resource: age, gender, race from SNOMED codes
- Extracts Encounter resource: LOS, admission month, day-of-week
- Extracts Condition resource: Charlson CCI from SNOMED CT codes
- Extracts Observation resource: 14 LOINC lab codes
- Extracts MedicationRequest: medication count and insulin flag
- Computes 30-day readmission target from sequential inpatient Encounter timestamps
| Feature Group | Raw Columns | After Engineering | Key Variables |
|---|---|---|---|
| Demographics | age, gender, race | age_numeric, race_enc | 10-year bracket midpoints (5-95) |
| Lab results | A1Cresult, max_glu_serum | abnormal_a1c, high_glucose, lab_risk_score | Ordinal encoding + composite risk flag |
| Medication | 23 binary drug cols | meds_in_use, meds_changed, med_change_score, on_insulin | Collapsed from 23 to 4 aggregate features |
| Diagnosis (ICD-9) | diag_1, diag_2, diag_3 | 9 ICD group flags + charlson_approx | Circulatory, Respiratory, Diabetes, Neurological, Musculoskeletal, Genitourinary, Digestive, Injury, Neoplasm |
| Utilization | num_lab_procedures, num_procedures, num_medications, number_outpatient, number_emergency, number_inpatient | total_prior_visits, emergency_visit_ratio, utilization_index | Captures access and acuity patterns |
| Interactions | Derived | age_x_charlson, age_x_meds, visits_x_utilization | Multiplicative cross-features |
9-Step Pipeline
The pipeline is fully modular: each stage operates on the output of the previous and writes figures directly to outputs/figures/. Every stage is independently configurable via CLI flags.
.fit() call. This prevents synthetic minority samples from appearing in CV test folds, which would inflate AUC to an artifactual ~0.96.
EDA Findings
Three figures were generated during EDA: class distribution, feature distributions, and a correlation heatmap. Key findings informed both feature engineering and modeling strategy.
72,326 negative cases (88.84%) vs 9,086 positive cases (11.16%). The 8:1 ratio means a naive model predicting "never readmitted" achieves 88.8% accuracy, making accuracy a misleading metric. ROC-AUC and PR-AUC were chosen as primary evaluation criteria.
Population clusters around the 70-80 year bracket, with the highest readmission rate in the 80-90 year group. Younger patients (under 50) show disproportionately high readmission risk relative to their overall admission count, suggesting different underlying drivers.
Patients on more medications show higher readmission rates but also higher mortality proxies. The correlation between num_medications and readmission is positive but modest (r ≈ 0.12), motivating the composite med_change_score interaction feature.
number_inpatient (prior inpatient visits in last year) showed the strongest individual correlation with 30-day readmission. Patients with 2+ prior inpatient visits are 2.4x more likely to be readmitted.
Length-of-stay is right-skewed: median 4 days, long tail to 14+ days. Near-zero correlation between administrative features and LOS (confirmed in regression, R² ≈ 0) reflects that LOS is driven by clinical severity not captured in administrative coding.
ICD-9 analysis revealed circulatory conditions as the most prevalent comorbidity group (~38% of encounters), followed by respiratory (~22%) and diabetes-specific codes (~17%). Multi-system comorbidity was associated with 40% higher readmission odds.
From 50 Raw Columns to 37 Features
The FeatureEngineer class applies 7 sequential transformation steps, dropping 23 individual medication columns, all raw categorical identifiers, and the target column, collapsing 50 raw fields into 37 numerically encoded, interaction-enriched features.
Cleaning and Demographic Encoding
Age brackets (e.g. "[70-80)") are mapped to numeric midpoints (5 to 95 in 10-year steps). A1c and glucose results are ordinal-encoded 0-3. Race is label-encoded. Gender becomes binary.
ICD-9 Diagnosis Feature Extraction
The three raw diagnosis columns are each parsed against 9 ICD-9 code ranges to produce binary group flags. A Charlson approximation index is summed across matched comorbidity codes.
Medication Collapse, Utilization, Lab Flags, Interactions
23 individual drug columns collapsed to 4 aggregate features. Utilization index aggregates prior visit counts. Lab risk score combines A1c and glucose flags. Three multiplicative interaction features capture joint effects.
Medication Collapse (23 → 4)
meds_in_use: count where status ≠ "No"
meds_changed: count with "Up" or "Down"
med_change_score: sum of +1/"Up", -1/"Down"
on_insulin: binary insulin flag
Utilization Index
total_prior_visits = outpatient + emergency + inpatient
emergency_visit_ratio = emergency / (total + 1)
utilization_index = log1p(total) × (1 + emergency_ratio)
Interaction Features
age_x_charlson = age_numeric × charlson_approx
age_x_meds = age_numeric × meds_in_use
visits_x_utilization = total_prior_visits × utilization_index
Custom Gradient Descent Implementation
Two gradient descent models were implemented from scratch without sklearn: LogisticRegressionGD for binary classification (BCE loss) and LinearRegressionGD for LOS regression (MSE loss). Both share a base class with configurable learning rate schedules, mini-batch support, and Xavier weight initialization.
Architecture: _BaseGD
- Weight init: Xavier uniform (scale = sqrt(6 / n_features))
- LR schedules: constant, step decay, cosine annealing
- Mini-batch generator with configurable batch size
- Early stopping via absolute loss tolerance (tol=1e-6)
- Full loss curve stored per epoch for plotting
LogisticRegressionGD
- Loss: Binary Cross-Entropy + L2 regularization
- Gradient: X.T @ (sigmoid(Xw) - y) / N + lambda*w
- Cosine LR schedule: lr_t = 0.5 × lr × (1 + cos(πt / T))
- Converged at epoch 363 (loss: 0.3388 → 0.3368)
- Initial LR = 0.1, LR at epoch 300 = 0.0345
SMOTE Resampling
The 8:1 class imbalance makes raw training ineffective for detecting readmissions. SMOTE generates synthetic minority samples by interpolating between k-nearest neighbors in feature space. Applied only to the training set; test set remains untouched.
Before SMOTE
Total: 81,412 samples
After SMOTE
Total: 144,652 samples (+77.7%)
Strategies Evaluated
- SMOTE: k=5 nearest neighbor interpolation in feature space. Selected for final pipeline.
- ADASYN: Adaptive synthetic sampling, weights samples near decision boundary more heavily.
- RandomUnderSampler: Randomly removes majority class samples. Fastest but loses information.
- SMOTETomek: SMOTE followed by Tomek links removal to clean borderline cases.
Threshold Tuning Result (thr=0.16)
- Precision: 0.21 (1 in 5 flagged patients is a true readmission)
- Recall: 0.44 (catches 44% of all actual readmissions)
- F1: 0.28
- For a hospital discharging 200 diabetic patients weekly: flags ~47 high-risk patients, catching ~10 of the 22 who would be readmitted
Length-of-Stay Prediction
A secondary regression task predicts continuous Length-of-Stay (days) from the same feature set. This serves as a calibration task validating that features are correctly constructed and that zero predictive signal is a data-level finding.
| Model | CV R² (3-fold) | Test R² | Test MAE | Test RMSE |
|---|---|---|---|---|
| Lasso | -0.0001 | -0.0001 | 2.33 | 2.95 |
| ElasticNet | -0.0002 | -0.0001 | 2.33 | 2.95 |
| Ridge | -0.0005 | -0.0002 | 2.33 | 2.95 |
| XGBoost | -0.0065 | -0.0002 | 2.33 | 2.95 |
| LightGBM | -0.0057 | -0.0007 | 2.33 | 2.95 |
| Random Forest | -0.0485 | -0.0003 | 2.33 | 2.95 |
30-Day Readmission Prediction
Five classifiers were evaluated via 3-fold stratified cross-validation on the original unbalanced training set, then re-fitted on SMOTE-balanced data for final test evaluation. Optuna TPE sampler with MedianPruner tuned each model for 25 trials with a 90-second timeout.
Baseline CV Results (3-fold, original data)
| Model | CV ROC-AUC | CV AUC SD | CV F1 | CV Precision | CV Recall |
|---|---|---|---|---|---|
| LightGBM | 0.6630 | 0.0004 | 0.016 | 0.497 | 0.008 |
| XGBoost | 0.6620 | 0.0005 | 0.022 | 0.479 | 0.011 |
| Decision Tree | 0.6414 | 0.0029 | 0.047 | 0.355 | 0.025 |
| Logistic Regression | 0.6366 | 0.0038 | 0.028 | 0.483 | 0.015 |
| Random Forest | 0.6182 | 0.0040 | 0.019 | 0.428 | 0.010 |
Optuna Tuning Impact
| Model | Baseline AUC | Optuna AUC | Delta | Trials |
|---|---|---|---|---|
| Random Forest | 0.6182 | 0.6519 | +5.4% | 6 |
| XGBoost | 0.6620 | 0.6685 | +1.0% | 25 |
| LightGBM | 0.6630 | 0.6676 | +0.7% | 18 |
| Decision Tree | 0.6414 | 0.6466 | +0.8% | 25 |
| Logistic Regression | 0.6366 | 0.6372 | +0.1% | 10 |
Final Test-Set Results (Optuna-Tuned, Threshold = 0.5)
| Model | ROC-AUC | PR-AUC | F1 | Precision | Recall | MCC | Brier |
|---|---|---|---|---|---|---|---|
| LightGBM | 0.6695 | 0.217 | 0.022 | 0.531 | 0.011 | 0.065 | 0.095 |
| XGBoost | 0.6580 | 0.210 | 0.027 | 0.463 | 0.014 | 0.064 | 0.097 |
| Logistic Regression | 0.6470 | 0.200 | 0.260 | 0.173 | 0.523 | 0.140 | 0.231 |
| Random Forest | 0.6398 | 0.187 | 0.031 | 0.430 | 0.016 | 0.066 | 0.111 |
| Decision Tree | 0.6157 | 0.158 | 0.222 | 0.180 | 0.289 | 0.102 | 0.135 |
Best Model: LightGBM at Threshold 0.16
| Class | Precision | Recall | F1 | Support |
|---|---|---|---|---|
| Not Readmitted (0) | 0.92 | 0.79 | 0.85 | 18,083 |
| Readmitted (1) | 0.21 | 0.44 | 0.28 | 2,271 |
| Accuracy | 0.75 | 20,354 | ||
| Macro Avg | 0.56 | 0.61 | 0.56 | |
| Weighted Avg | 0.84 | 0.75 | 0.78 |
Optuna TPE Search Strategy
Optuna v4.9.0 with Tree-structured Parzen Estimator (TPE) sampler and MedianPruner was used to search hyperparameter spaces for all 11 models. Each trial ran a 3-fold CV on the original training set; trials exceeding the median score at the first fold were pruned early.
LightGBM Search Space
- n_estimators: [100, 400]
- learning_rate: [0.01, 0.3] log-uniform
- num_leaves: [20, 200]
- min_child_samples: [5, 80]
- subsample, colsample_bytree: [0.5, 1.0]
- reg_alpha, reg_lambda: [1e-3, 10] log-uniform
XGBoost Search Space
- n_estimators: [100, 400]
- learning_rate: [0.01, 0.3] log-uniform
- max_depth: [3, 12]
- subsample, colsample_bytree: [0.5, 1.0]
- reg_alpha, reg_lambda: [1e-3, 10] log-uniform
- gamma: [0, 5], min_child_weight: [1, 10]
cross_validate(n_jobs=-1) with LGBMClassifier(n_jobs=-1) deadlocks on Windows due to OpenMP vs loky process pool conflict. Fix: cross_validate(n_jobs=1) with each model using internal thread parallelism via n_jobs=4 / nthread=4.
SHAP Explanations
SHAP TreeExplainer was applied to all three tree-based models (Random Forest, XGBoost, LightGBM), generating beeswarm summary plots, individual waterfall plots, and bar-chart feature importance figures. Nine total figures were produced.
Prior inpatient admissions drive the largest share of prediction separation. Patients with 2+ prior inpatient visits show strongly positive SHAP values across all three tree models. Prior admission history is the strongest known administrative readmission predictor in the clinical literature.
The multiplicative age x comorbidity interaction was among the top 5 SHAP features across all models. High values (old patients with high comorbidity burden) produced the largest positive SHAP contributions, confirming the frailty signal independently captured by this engineered feature.
Patients on active insulin therapy show systematically higher SHAP values. Insulin use is a proxy for Type 1 diabetes or poorly controlled Type 2, both associated with higher complication rates during inpatient stays and at discharge.
Medication decreases (negative med_change_score) were associated with positive readmission SHAP values more often than increases. Downward titration at discharge may reflect clinical instability or end-of-episode cost management.
LightGBM and XGBoost produced nearly identical SHAP rankings (Spearman r > 0.95 across top-20 features), suggesting both models converged to the same dominant signal. The small AUC gap (0.6695 vs 0.6580) reflects minor differences in leaf-level split optimization.
Discharge destination (home vs skilled nursing vs home health care) showed strong SHAP signal. Patients discharged to skilled nursing facilities or with home health orders had higher readmission risk, reflecting post-acute care instability and underlying functional decline.
FHIR R4 Ingestion Layer
The FHIRParser module implements a production-style ingestion pattern for HL7 FHIR R4 Bundle JSON files. While the UCI dataset is the primary data source, the FHIR layer demonstrates real-world EHR integration capability and enables richer feature extraction from structured clinical resources.
SNOMED CT: Charlson Comorbidity Index
The Charlson CCI is computed from Condition resource SNOMED codes, mapping coded diagnoses to 17 Charlson domains (myocardial infarction, CHF, peripheral vascular disease, dementia, COPD, rheumatoid disease, peptic ulcer, mild liver disease, diabetes, etc.). The weighted sum provides a standardized comorbidity burden score used clinically to estimate 10-year mortality.
LOINC Observation Extraction
14 LOINC codes are extracted from Observation resources: HbA1c (4548-4), serum creatinine (2160-0), BUN (3094-0), eGFR (33914-3), sodium, potassium, glucose (2339-0), hemoglobin, WBC, platelet count, ALT, AST, albumin, and total bilirubin. These enable a richer lab feature set than the UCI dataset's single A1c and glucose columns.
Limitations and Future Work
Known Limitations
- Administrative features explain a limited fraction of 30-day readmission variance. AUC 0.67 is consistent with published literature for administrative-only datasets (typical range: 0.60-0.72).
- Length-of-Stay is unpredictable (R² ≈ 0) without clinical severity scores (APACHE II, SOFA). This is a fundamental data limitation.
- The UCI dataset covers 1999-2008. Medication coding, discharge practices, and diagnostic coding have changed significantly.
- FHIR enrichment requires Synthea-generated data rather than real patient records, limiting real-world validation.
- SMOTE assumes smooth decision boundaries in feature space, which may not hold for highly non-linear clinical data.
Proposed Extensions
- Clinical severity integration: Add APACHE II or NEWS2 scores at admission to unlock the primary LOS and readmission signal missing from administrative codes.
- Temporal modeling: Model the admission sequence as a time series using LSTM or Transformer architecture to capture trajectory effects across multiple encounters.
- Real FHIR deployment: Connect to a live FHIR R4 server (SMART on FHIR) for real-time inference at point of discharge.
- Fairness audit: Evaluate model performance stratified by race and gender to detect differential performance on protected groups.
- Calibration: Apply Platt scaling or isotonic regression to convert raw LightGBM probability outputs to calibrated clinical risk scores.
Full Technical Stack
Data and Features
- pandas, numpy: ETL and feature matrix
- scikit-learn: StandardScaler, Pipeline, StratifiedKFold, cross_validate
- scipy: Statistical utilities
- ICD-9 / SNOMED CT / LOINC: Clinical code systems
- HL7 FHIR R4: Clinical interoperability standard
Modeling
- scikit-learn: LogisticRegression, DecisionTree, RandomForest, Ridge, Lasso, ElasticNet
- xgboost: XGBClassifier, XGBRegressor
- lightgbm: LGBMClassifier, LGBMRegressor
- imbalanced-learn: SMOTE, ADASYN, SMOTETomek
- optuna 4.9.0: TPE + MedianPruner
Interpretability and Visualization
- shap: TreeExplainer, summary_plot, waterfall_plot
- matplotlib + seaborn: 43 static publication figures
- plotly + kaleido: Optuna history and param importance PNGs
- Custom GD: Pure NumPy, no sklearn dependency