Fine-tuning versus frozen features, and LoRA

Fine-tuning adapts a model trained for one task to a more specific task, a form of transfer learning (Wikipedia); LoRA, introduced by Microsoft researchers in 2021, adapts a pretrained model with far fewer trainable parameters by learning low-rank matrices added to the weights (Wikipedia). With hundreds of labelled patients, full fine-tuning overfits and frozen features plus a linear or ridge head is the standard recipe; the honest comparison is against a genome-wide baseline trained from scratch, which a frozen foundation model does not always beat.