Nucleotide Transformer: building and evaluating robust foundation models for human genomics
Paper cited by one technology page, indexed on Europe PMC as PubMed record 39609566 and published in Nature methods; the citing page links this DOI, which is how the record was matched.
Overview
The prediction of molecular phenotypes from DNA sequences remains a longstanding challenge in genomics, often driven by limited annotated data and the inability to transfer learnings between tasks. Here, we present an extensive study of foundation models pre-trained on DNA sequences, named Nucleotide Transformer, ranging from 50 million up to 2.5 billion parameters and integrating information from 3,202 human genomes and 850 genomes from diverse species. These transformer models yield context-specific representations of nucleotide sequences, which allow for accurate predictions even in low-data settings. We show that the developed models can be fine-tuned at low cost to solve a variety of genomics applications. Despite no supervision, the models learned to focus attention on key genomic elements and can be used to improve the prioritization of genetic variants. The training and application of foundational models in genomics provides a widely applicable approach for accurate molecular phenotype prediction from DNA sequence.
Indexed on Europe PMC as PubMed record 39609566 (DOI 10.1038/s41592-024-02523-z). Matched by DOI alone: one technology page cites this DOI among its external links (the pages are listed under Related), and this page was written so that the citation resolves inside OnCo. No figure has been checked by an editor.
One technology page on OnCo cites this paper by its DOI; this record gives the citation a page of its own so a reader can follow it without leaving OnCo. Read the abstract above alongside the citing page listed under Related; the record was created automatically from the Europe PMC entry and its figures have not been checked by hand.
- Matched to the citing OnCo records by DOI alone; the summary reproduces the Europe PMC abstract and no figure has been verified against the full paper.