# Nucleotide Transformer (InstaDeep)

Source: https://onco.cc/technologies/nucleotide-transformer/  
OnCo record `nucleotide-transformer` (Technology). Data CC BY-NC 4.0, attribute "Data from OnCo (onco.cc)"; commercial use needs a licence.

## TL;DR

DNA language models trained on thousands of genomes for variant and regulatory prediction.

## Summary

Nucleotide Transformer from InstaDeep is a family of DNA language models trained with masked language modelling over 6-mer tokens, learning sequence patterns without labels. The Nature Methods 2024 paper describes models up to 2.5B parameters trained on 3,200 human genomes and 850 species, and fine-tunes them for regulatory element and variant effect prediction. The weights are open, which has made the models a common baseline in genomic language modelling. Their context window is short compared with newer long-context models, so they cannot see distant regulatory interactions, and their usefulness for interpreting non-coding cancer variants depends on downstream fine-tuning. For a newcomer: it is an early, freely available model that learned patterns in DNA from thousands of genomes.

## Fields

- Kind: Technology
- Status: emerging
- Last checked: 2026-09-08
- Tags: foundation-model; genome
- Principle: Nucleotide Transformer uses masked language modelling over 6-mer tokens.
- Since: 2024
- Strengths: Open weights
- Limitations: Short context

## Sources

- Nature Methods 2024: https://doi.org/10.1038/s41592-024-02523-z

## Connected records

- fronts: [AI & Computation](https://onco.cc/fronts/ai-computation/)
- key papers: [Nucleotide Transformer: building and evaluating robust foundation models for human genomics](https://onco.cc/key-papers/paper-dalla-torre-nat-methods/)
- terms: [Autoregressive (next-token) modelling](https://onco.cc/terms/autoregressive-modelling/), [Genomic and protein language models: Evo 2, Enformer, ESM](https://onco.cc/terms/genomic-and-protein-language-models/), [Variant effect prediction](https://onco.cc/terms/variant-effect-prediction/)

---
JSON: https://onco.cc/api/v1/entities/nucleotide-transformer.json