# Geneformer

Source: https://onco.cc/technologies/geneformer/  
OnCo record `geneformer` (Technology). Data CC BY-NC 4.0, attribute "Data from OnCo (onco.cc)"; commercial use needs a licence.

## TL;DR

Geneformer is a transformer trained on about 30 million single cells that encodes each cell as a ranked list of its genes, so deleting a gene in silico shows which genes matter in a disease. It was the first single-cell foundation model in general use, though benchmarks find only modest gains over linear baselines on some tasks.

## Summary

Geneformer is a transformer trained on single-cell gene expression that encodes each cell as a ranked list of its genes and learns by masked prediction, capturing gene-gene context without labelled data. The Nature 2023 paper pretrained it on about 30M cells (later 95M) and showed that in silico perturbation, deleting a gene in the model and watching the cell representation move, identified therapeutic targets in cardiomyopathy; the approach has since been applied to tumours. It was the first widely used single-cell foundation model and is valued for transfer learning to tasks with little labelled data. Its rank encoding loses expression magnitude, and benchmarks have found only modest gains over linear baselines on some tasks, so its added value is debated. For a newcomer: Geneformer learned the grammar of genes from millions of cells and can be asked which genes matter in a disease.

## Fields

- Kind: Technology
- Status: emerging
- Last checked: 2026-09-08
- Tags: foundation-model; virtual-cell
- Principle: Transformer over rank-ordered gene expression with masked learning.
- Since: 2023
- Strengths: Transfer learning with little labelled data
- Limitations: Rank encoding loses magnitude; Modest gains over linear baselines on some tasks

## Sources

- Nature 2023: https://doi.org/10.1038/s41586-023-06139-9

## Connected records

- roadmaps: [AI in oncology roadmap: pattern readers → foundation models → agents in the workflow](https://onco.cc/roadmaps/ai-oncology-roadmap/), [Drug discovery roadmap: screening in mice → maps of dependency → designing in silico](https://onco.cc/roadmaps/drug-discovery-roadmap/), [Virtual cell roadmap: from bulk omics to a predictive model of a cancer cell](https://onco.cc/roadmaps/virtual-cell/)
- fronts: [AI & Computation](https://onco.cc/fronts/ai-computation/), [Drug Discovery Platforms](https://onco.cc/fronts/drug-discovery/)
- technologies: [Single-cell & spatial profiling](https://onco.cc/technologies/single-cell-spatial/)
- key papers: [Transfer learning enables predictions in network biology](https://onco.cc/key-papers/paper-theodoris-nature/)

---
JSON: https://onco.cc/api/v1/entities/geneformer.json