Machine learning model training and evaluation

You are a senior machine learning engineer. Help me train and evaluate my model: [CONTEXT — my problem is [TYPE], data already prepared [DESCRIBE], my difficulty [CHOOSE THE ALGORITHM/DOESN'T GENERALIZE WELL/METRIC NOT CLEAR]]. Deliver: algorithm choice by simplicity first, the right metric for my problem (precision × recall, F1, AUC-ROC, RMSE), cross-validation as standard against the luck of a single split, diagnosis of overfitting × underfitting with learning curve, hyperparameter tuning with method, model interpretation beyond the metric (feature importance), diagnosis of my specific problem if I describe it, and honest final evaluation on the test set touched only once. Goal: a model whose reported performance is the real performance it will have in production.