# Fine-tuning versus frozen features, and LoRA

Source: https://onco.cc/terms/fine-tuning-vs-frozen/  
OnCo record `fine-tuning-vs-frozen` (Term). Data CC BY-NC 4.0, attribute "Data from OnCo (onco.cc)"; commercial use needs a licence.

## TL;DR

Fine-tuning updates a pretrained model's weights on the new task; using it frozen keeps the weights fixed and trains only a small head on its features; LoRA is a cheap middle way that trains small low-rank updates.

## Summary

Fine-tuning adapts a model trained for one task to a more specific task, a form of transfer learning (Wikipedia); LoRA, introduced by Microsoft researchers in 2021, adapts a pretrained model with far fewer trainable parameters by learning low-rank matrices added to the weights (Wikipedia). With hundreds of labelled patients, full fine-tuning overfits and frozen features plus a linear or ridge head is the standard recipe; the honest comparison is against a genome-wide baseline trained from scratch, which a frozen foundation model does not always beat.

## Fields

- Kind: Term
- Last checked: 2026-09-24
- Also known as: fine-tuning; fine tuning; full fine-tuning; frozen encoder; frozen features; frozen embeddings; LoRA; low-rank adaptation; parameter-efficient fine-tuning; PEFT
- Tags: cansim-terms

## Notes

- Listed in the CanSim terms map 1.0.0 (docs/onco/terms.json, generated 2026-09-24), CC BY 4.0, attribution: CanSim project, an open, public-data-first cancer foundation-model programme; CanSim page path /terms/fine-tuning-vs-frozen-embeddings.

## Sources

- Wikipedia: https://en.wikipedia.org/wiki/Fine-tuning_(deep_learning)
- Wikipedia: LoRA: https://en.wikipedia.org/wiki/LoRA_(machine_learning)
- Hu et al., LoRA: Low-Rank Adaptation of Large Language Models (arXiv 2021): https://arxiv.org/abs/2106.09685
- Wikipedia: https://en.wikipedia.org/wiki/Fine-tuning_(deep_learning)

## Connected records

- terms: [Cancer AI vocabulary (CanSim terms map)](https://onco.cc/terms/cancer-ai-vocabulary/), [Embedding (learned representation)](https://onco.cc/terms/embedding/), [Foundation model](https://onco.cc/terms/foundation-model/), [GPU training and mixed precision](https://onco.cc/terms/mixed-precision-gpu/), [Linear probe](https://onco.cc/terms/linear-probe/), [Single-cell and transcriptome foundation models: UCE, GeneCompass, BulkFormer, BulkRNABert](https://onco.cc/terms/single-cell-foundation-models/), [Transfer learning and the low-label regime](https://onco.cc/terms/transfer-learning/)

---
JSON: https://onco.cc/api/v1/entities/fine-tuning-vs-frozen.json