Loading
Data is divided into a training set the model learns from, a validation set used to choose settings, and a test set touched once at the end; using the test set to make choices turns it into a second validation set.
In machine learning the input data used to build a model is divided into a training set, a validation set for tuning and a test set for the final, unbiased evaluation (Wikipedia). The discipline is behavioural: the test set is looked at once, patients (not samples) are the unit of splitting so one patient's slides do not appear on both sides, and the split is decided before modelling. A held-out 80/20 split is the minimum; an external cohort from another institution is the real test.
Shares Data leakage in model evaluation, External validation, Cancer AI vocabulary (CanSim terms map) and the tag cansim-terms.
Shares Cross-validation and stratified k-fold, Cancer AI vocabulary (CanSim terms map) and the tag cansim-terms.
Shares Data leakage in model evaluation, Cancer AI vocabulary (CanSim terms map) and the tag cansim-terms.
Shares External validation, Cancer AI vocabulary (CanSim terms map) and the tag cansim-terms.
Shares Pre-registered experiment, Cancer AI vocabulary (CanSim terms map) and the tag cansim-terms.
Shares External validation, Cancer AI vocabulary (CanSim terms map) and the tag cansim-terms.
Shares External validation, Cancer AI vocabulary (CanSim terms map) and the tag cansim-terms.
Shares Data leakage in model evaluation, Cancer AI vocabulary (CanSim terms map) and the tag cansim-terms.