Survival prediction
with the classics.

Abstract

Predicting in-hospital mortality for intensive-care admissions in MIMIC, with the model class fixed in advance: K-nearest neighbours and support vector machines, nothing else. When the estimator is given to you, every decision that remains is a preprocessing decision, and that is where the performance turns out to live.

Data

20,885 ICU stays for training, 5,221 for scoring; 85 raw columns plus an auxiliary ICD-9 diagnosis table. The target (death before discharge) is true for 2,345 stays, about 11%.

Method

Feature extraction from the diagnosis tables → per-vital aggregation → target encoding → scaling and KNN imputation → grid search over both model families, scored on ROC-AUC.

What the data gives you

Each stay arrives as demographics (age, gender, insurance, religion, marital status, ethnicity), admission metadata (type, hour of admission, first care unit, whether the patient had been in the ICU before), and vital signs summarised as minimum, maximum and mean: heart rate, systolic, diastolic and mean blood pressure, respiratory rate, temperature, glucose, oxygen saturation.

The diagnoses live in a separate table, one row per ICD-9 code per admission. Collapsing it to a count of distinct codes gives a serviceable proxy for comorbidity (how many things were wrong at once) that no vital sign captures.

The three columns per vital are largely redundant, so each triplet is folded into one:

def grouped_vitals(col_name: str, df: pd.DataFrame):
    col_mean = col_name + '_Mean'
    col_min  = col_name + '_Min'
    col_max  = col_name + '_Max'

    df[col_name] = 0.5 * df[col_mean] + 0.5 * df[[col_min, col_max]].mean(axis = 1)

Encoding without exploding

The diagnosis fields are the awkward ones: 358 ICD-9 codes appear at least ten times, and 176 distinct free-text admission diagnoses. One-hot encoding either of them buys hundreds of near-empty columns, which is fatal for distance-based methods, since every added dimension dilutes the metric that KNN and an RBF kernel both depend on. They are target-encoded instead, with the smoothing strength left as a hyper-parameter for the grid search to settle.

Everything else follows from the same concern. Skewed numerics are scaled with a robust scaler, the rest standardised, missing vitals filled by a KNN imputer rather than a column mean, and for the KNN branch a univariate filter keeps only the twenty most informative features. All of it inside one pipeline, so the encoders and the imputer are fit on training folds only and the cross-validation stays honest.

Tuning

Both families were searched on ROC-AUC, the sensible target when the positive class is one in nine and accuracy would reward predicting survival every time.

KNN settled at 750 neighbours with distance weighting, about 4% of the training set voting on each patient. That is far past where a textbook stops, and it reflects a noisy, weakly separable outcome: heavy smoothing is what the data will support. Cross-validated AUC, 0.924.

The SVM search began with all four kernels. Linear, polynomial and sigmoid were beaten by the RBF in every fold and were dropped. With balanced class weights and a soft margin, it reached 0.925, a difference from KNN that sits comfortably inside the noise.

Takeaway

Two estimators with almost nothing in common land on the same number, which suggests the ceiling belongs to the features rather than to either model. The work that moved the needle was upstream: turning a diagnosis table into a comorbidity count, keeping the dimensionality low enough that distances still mean something, and choosing a metric that a rare outcome cannot game.