From Policy to Patient: A Finite-Horizon Markov Decision Process for Optimizing Non-Small Cell Lung Cancer Treatment

Abstract

Advancements in immunotherapy have transformed treatment for advanced stages of non-small cell lung cancer (NSCLC). However, optimal sequencing of chemotherapy, immunotherapy, and combination chemoimmunotherapy still remains unresearched. Chemotherapy may prime the tumor microenvironment, enhancing immune activation and, as a result, immunotherapy’s effectiveness. To explore this timing advantage, we develop a finite-horizon Markov Decision Process (MDP) to model treatment selection over a course of ten cycles. The model incorporates four clinical variables to guide decision making: toxicity, PD-L1 expression (as a proxy for immune activation), disease progression, and overall survival. Transition probabilities and survival outcomes are derived from clinical trial data, and cost is defined as a normalized ratio of burden (toxicity and disease progression) to survival. The results indicate that chemotherapy is only optimal under extreme exaggeration of its role in immune activation or when parameters like progression are eliminated. However, there is benefit in combined regimens: chemoimmunotherapy followed by immunotherapy proves optimal in all initial states of no toxicity or disease progression. When compared to the three therapies on their own, the costs for the optimal policy is significantly lower in all cases, highlighting the benefit of an adaptive treatment plan. Such can inform future clinical trial planning for NSCLC. This work is the first of its kind to integrate immunotherapy and account for dynamic immune activation, providing a novel starting point for more complex treatment optimization.