Predicting cancer treatment response with Random Forest in Python

This is a brief example of how one can predict the outcome of cancer treatment using patient data. More specifically, we will focus on the prediction of the pathologically complete response after neoadjuvant chemoradiotherapy for oesophageal cancer.

The following columns (predictor variables) are included in the data:

• Patient ID - as UUID,

• Patient age - integer, based on mean and range from the corresponding paper,

• Patient sex - binary, based the corresponding paper data distribution,

• Tumor type - categorical, based the corresponding paper data distribution,

• Differentiation grade - ordinal, based the corresponding paper data distribution,

• T-stage - ordinal, based the corresponding paper data distribution,

• N-stage - ordinal, based the corresponding paper data distribution,

• M-stage - ordinal, based the corresponding paper data distribution,

• Overall stage - ordinal, based on the lookup table https://www.researchgate.net/figure/Esophageal-cancer-staging-The-TNM-tumor-node-and-metastasis-staging-system-takes_fig1_274257853,

• Survival time - float, based on the Kaplan-Meier curves for this diagnosis https://www.researchgate.net/figure/Kaplan-Meier-curves-for-overall-survival-in-patients-with-esophageal-cancer-treated_fig2_282245546.

  
  

The Outcome or response variable is the Treatment response:

• Treatment response, based on the nomogram from the corresponding paper which turned into computational predictive model in Halilaj I, Oberije C, Chatterjee A, van Wijk Y, Rad NM, Galganebanduge P, Lavrova E, Primakov S, Widaatalla Y, Wind A, et al. Open Source Repository and Online Calculator of Prediction Models for Diagnosis and Prognosis in Oncology. Biomedicines. 2022; 10(11):2679. https://doi.org/10.3390/biomedicines10112679.

First, we have to pre-process the data: this includes the removal of the variable ID which is the unique patient identifier; as well as the variable ‘Smoking’ which contains a lot of missing values:

Second, we have to drop the remaining randomly missing values in the dataset using Python’s dropna() method.

Third, we should encode all categorical variables using LabelEncoder.

And this completes the data pre-processing step. Next we have to partition the data into a training and validating datasets:

Once done with the partitioning we can train the random forest model on the training data and validate its accuracy on the validation dataset.

Done! The accuracy of the model is very good as well - about 96% predictive power. This means that the data is good and we can use it to on average predict the treatment response in patients with cancer.