# Transfer learning enables predictions in network biology

Source: https://onco.cc/key-papers/paper-theodoris-nature/  
OnCo record `paper-theodoris-nature` (Key paper). Data CC BY-NC 4.0, attribute "Data from OnCo (onco.cc)"; commercial use needs a licence.

## TL;DR

Paper cited by one technology page, indexed on Europe PMC as PubMed record 37258680 and published in Nature; the citing page links this DOI, which is how the record was matched.

## Summary

Mapping gene networks requires large amounts of transcriptomic data to learn the connections between genes, which impedes discoveries in settings with limited data, including rare diseases and diseases affecting clinically inaccessible tissues. Recently, transfer learning has revolutionized fields such as natural language understanding 1,2 and computer vision 3 by leveraging deep learning models pretrained on large-scale general datasets that can then be fine-tuned towards a vast array of downstream tasks with limited task-specific data. Here, we developed a context-aware, attention-based deep learning model, Geneformer, pretrained on a large-scale corpus of about 30 million single-cell transcriptomes to enable context-specific predictions in settings with limited data in network biology. During pretraining, Geneformer gained a fundamental understanding of network dynamics, encoding network hierarchy in the attention weights of the model in a completely self-supervised manner. Fine-tuning towards a diverse panel of downstream tasks relevant to chromatin and network dynamics using limited task-specific data demonstrated that Geneformer consistently boosted predictive accuracy. Applied to disease modelling with limited patient data, Geneformer identified candidate therapeutic targets for cardiomyopathy. Overall, Geneformer represents a pretrained deep learning model from which fine-tuning towards a broad range of downstream applications can be pursued to accelerate discovery of key network regulators and candidate therapeutic targets.

Indexed on Europe PMC as PubMed record 37258680 (DOI 10.1038/s41586-023-06139-9). Matched by DOI alone: one technology page cites this DOI among its external links (the pages are listed under Related), and this page was written so that the citation resolves inside OnCo. No figure has been checked by an editor.

## Fields

- Kind: Key paper
- Last checked: 2026-09-22
- Tags: europepmc-ingest
- Journal: Nature
- Year: 2023
- DOI: 10.1038/s41586-023-06139-9
- Authors: Theodoris CV, Xiao L, Chopra A, et al.
- What it means: One technology page on OnCo cites this paper by its DOI; this record gives the citation a page of its own so a reader can follow it without leaving OnCo. Read the abstract above alongside the citing page listed under Related; the record was created automatically from the Europe PMC entry and its figures have not been checked by hand.
- Caveats: Matched to the citing OnCo records by DOI alone; the summary reproduces the Europe PMC abstract and no figure has been verified against the full paper.

## Sources

- Nature 2023: https://doi.org/10.1038/s41586-023-06139-9
- PubMed: https://pubmed.ncbi.nlm.nih.gov/37258680/
- Europe PMC: https://europepmc.org/article/MED/37258680

## Connected records

- technologies: [Geneformer](https://onco.cc/technologies/geneformer/)
- journals: [Nature](https://onco.cc/journals/nature/)

---
JSON: https://onco.cc/api/v1/entities/paper-theodoris-nature.json