{"entity":{"id":"fine-tuning-vs-frozen","kind":"term","name":"Fine-tuning versus frozen features, and LoRA","aka":["fine-tuning","fine tuning","full fine-tuning","frozen encoder","frozen features","frozen embeddings","LoRA","low-rank adaptation","parameter-efficient fine-tuning","PEFT"],"tldr":"Fine-tuning updates a pretrained model's weights on the new task; using it frozen keeps the weights fixed and trains only a small head on its features; LoRA is a cheap middle way that trains small low-rank updates.","summary":"Fine-tuning adapts a model trained for one task to a more specific task, a form of transfer learning (Wikipedia); LoRA, introduced by Microsoft researchers in 2021, adapts a pretrained model with far fewer trainable parameters by learning low-rank matrices added to the weights (Wikipedia). With hundreds of labelled patients, full fine-tuning overfits and frozen features plus a linear or ridge head is the standard recipe; the honest comparison is against a genome-wide baseline trained from scratch, which a frozen foundation model does not always beat.","asOf":"2026-09-24","wikipedia":"https://en.wikipedia.org/wiki/Fine-tuning_(deep_learning)","links":[{"label":"Wikipedia: LoRA","url":"https://en.wikipedia.org/wiki/LoRA_(machine_learning)"},{"label":"Hu et al., LoRA: Low-Rank Adaptation of Large Language Models (arXiv 2021)","url":"https://arxiv.org/abs/2106.09685"},{"label":"Wikipedia","url":"https://en.wikipedia.org/wiki/Fine-tuning_(deep_learning)"}],"tags":["cansim-terms"],"related":["cancer-ai-vocabulary"],"cancers":[],"sections":[],"technologies":[],"targets":[],"drugs":[],"companies":[],"institutions":[],"pathways":[],"terms":["transfer-learning","linear-probe","embedding","foundation-model"],"trials":[],"people":[],"bottlenecks":[],"keyPapers":[],"journals":[],"dependsOn":[],"notes":["Listed in the CanSim terms map 1.0.0 (docs/onco/terms.json, generated 2026-09-24), CC BY 4.0, attribution: CanSim project, an open, public-data-first cancer foundation-model programme; CanSim page path /terms/fine-tuning-vs-frozen-embeddings."],"provenance":{"editedBy":"OnCo CanSim terms wave (Wikipedia summaries, standards and project pages, GDC and FDA pages, Europe PMC)","editedOn":"2026-09-24","note":"CanSim terms map 1.0.0 (docs/onco/terms.json, generated 2026-09-24), CC BY 4.0, attribution: CanSim project, an open, public-data-first cancer foundation-model programme"},"category":"Methods and models"},"route":"/terms/fine-tuning-vs-frozen/","neighbours":{"term":[{"id":"cancer-ai-vocabulary","kind":"term","name":"Cancer AI vocabulary (CanSim terms map)","route":"/terms/cancer-ai-vocabulary/"},{"id":"embedding","kind":"term","name":"Embedding (learned representation)","route":"/terms/embedding/"},{"id":"foundation-model","kind":"term","name":"Foundation model","route":"/terms/foundation-model/"},{"id":"mixed-precision-gpu","kind":"term","name":"GPU training and mixed precision","route":"/terms/mixed-precision-gpu/"},{"id":"linear-probe","kind":"term","name":"Linear probe","route":"/terms/linear-probe/"},{"id":"single-cell-foundation-models","kind":"term","name":"Single-cell and transcriptome foundation models: UCE, GeneCompass, BulkFormer, BulkRNABert","route":"/terms/single-cell-foundation-models/"},{"id":"transfer-learning","kind":"term","name":"Transfer learning and the low-label regime","route":"/terms/transfer-learning/"}]}}