Seminars of the series “Optimization in Machine Learning”

Talk Title: New optimization training strategies for Survival and Classification Trees
Speaker: Antonio Consolo, Università degli Studi di Milano Bicocca
When: Wednesday, 9th July, 11:00-12:00
Where: Alan Turing Seminar Room
Abstract: Decision trees are supervised learning models widely used to handle tabular data thanks to their interpretability and their capacity to capture complex relationships. They are applied in various tasks, including survival analysis and classification. Recently, there has been increasing interest in globally optimized trees, where the entire tree is trained by minimizing an error function over all its parameters. In this talk, we present new formulations and optimization methods to train globally optimized trees for both survival analysis and classification.
As to survival analysis, we propose a new soft survival tree model (SST), with a soft splitting rule at each branch node, trained via a nonlinear optimization formulation amenable to decomposition. SSTs provide for every input vector a specific survival function associated to a single leaf node, satisfying the conditional computation property. SST and the training formulation combine flexibility with interpretability: any smooth survival function estimated through maximum likelihood can be used, while the leaf nodes represent clusters of distinct survival curves corresponding to different data points. Numerical experiments on 15 well-known datasets show that SSTs, with parametric and spline-based semiparametric survival functions, trained using an adaptation of the node-based decomposition algorithm proposed by Consolo et al. (2024) for soft regression trees, outperform three benchmark survival trees in terms of four widely-used discrimination and calibration metrics. Moreover, SSTs can be easily  extended to consider group fairness.
Concerning classification, we present a multivariate deterministic tree trained using a nonlinear optimization formulation adapted from the method proposed by Blanquero et al. (2021) for soft classification trees. This alternative formulation leverages a modified activation function at the branch nodes, which allows to satisfy the conditional computation property while also offering probabilistic confidence outputs. The training procedure is based on a projected search algorithm. Early experimental results on 54 datasets indicate that our method achieves performance comparable to OCT-H (Bertsimas et al., 2017) as well as traditional ensemble methods such as random forests and XGBoost. 

Talk Title: Counterfactual optimization for fault prevention in complex wind energy systems
Speaker: Martina Fischetti, University of Seville
When: Thursady, 10th July, 15:30-16:30
Where: online on Teams, link: https://teams.microsoft.com/l/meetup-join/19%3ameeting_YWM1MWYxNTEtZGEyYy00MzU0LTg2ZDktZDQ1MmM3NDc1Y2Q3%40thread.v2/0?context=%7b%22Tid%22%3a%229df08a7c-31d7-4024-9ba6-5ed5efac1a01%22%2c%22Oid%22%3a%22d4f0c029-8a66-42b2-b484-cd0d5763162a%22%7d
Abstract: Machine Learning models are increasingly used in businesses to detect faults and anomalies in complex systems. In this work, we take this approach a step further: beyond merely detecting anomalies, we aim to identify the optimal control strategy that restores the system to a safe state with minimal disruption. We frame this challenge as a counterfactual problem: given a Machine Learning model that classifies system states as either “good” or “anomalous,” our goal is to determine the minimal adjustment to the system’s control variables (i.e., its current status) that is necessary to return it to the “good” state. To achieve this, we leverage a mathematical model that finds the optimal counterfactual solution while respecting system-specific constraints. Notably, most counterfactual analysis in the literature focuses on individual cases where a person seeks to alter their status relative to a decision made by a classifier, such as for loan approval or medical diagnosis. Our work addresses a fundamentally different challenge: optimizing counterfactuals for a complex energy system, specifically an offshore wind turbine oil-type transformer. This application not only advances counterfactual optimization in a new domain but also opens avenues for broader research in this area. Our tests on real-world data provided by our industrial partner show that our methodology easily adapts to user preferences and brings savings in the order of 3 million euros per year in a typical farm.