In this study, we model the supervised feature selection problem using a novel approach: convex bi-objective optimization. Traditional methods have addressed this problem by maximizing relevance
to class labels and minimizing redundancy among features. Recently, Wang et al. [30] formulated this problem as a single-objective convex optimization, yielding only a unique solution. Unlike that, we approach this problem by preserving the natural multi-objective essence of the supervised feature selection problem, enabling a broader exploration of the objective space and providing a set of optimal solutions rather than a single result. To solve the obtained model, we utilize two state-of-the-art strategies: an exact method and a heuristic method. Additionally, we have enhanced the exact method using a scaling technique, which accelerates processing speed and expands the Pareto front. In parallel, the heuristic method ensures that the Pareto solutions achieve extensive coverage and distribution. The effectiveness of our proposed method is confrmed through standard medical datasets, demonstrating superiority over existing techniques. Notably, in the context of skin cancer screening, the method optimized the feature set to less than half of its original size, thereby signifcantly enhancing classifcation accuracy for the task.