# Cell segmentation in tissue measurements

Source: https://onco.cc/terms/cell-segmentation/  
OnCo record `cell-segmentation` (Term). Data CC BY-NC 4.0, attribute "Data from OnCo (onco.cc)"; commercial use needs a licence.

## TL;DR

Image analysis estimates which pixels or detected molecules belong to each cell. Those assignments affect the measurements reported for that cell.

## Summary

The Mesmer study developed and tested whole-cell segmentation across tissue images and used the estimated boundaries to extract cellular features. A separate spatial-transcriptomics reanalysis found that misassigned molecules could change downstream cell-state and interaction analyses. These findings concern defined datasets and methods, not one universal error rate. Segmentation quality and signal assignment need evaluation before a spatial measurement is treated as a biological conclusion. Mesmer was trained on TissueNet, a dataset of more than 1 million manually labelled cells, and was reported to generalise across the tissue types and imaging platforms in that dataset at human-level performance; its authors note that segmentation inaccuracies can substantially bias the identification and counting of cells, and that cells whose nucleus lies outside the imaging plane are missed by nucleus-based methods (Greenwald and colleagues, 2022). In that evaluation, a predicted and a true cell with a one-to-one match by intersection over union count as accurate, predicted cells with no match are false positives and true cells with no match are false negatives. The Cellpose study reported a generalist method trained on more than 70,000 segmented objects that worked without retraining across the image types it evaluated (Stringer and colleagues, 2021). Baysor was developed because methods that approximate cell positions from nuclear stains make cell boundaries hard to distinguish in single-molecule spatial data (Petukhov and colleagues, 2022).

## Fields

- Kind: Term
- Last checked: 2026-11-10
- Also known as: single-cell tissue segmentation

## Notes

- Reading guide (definitions plus OnCo inference). Detection asks whether a cell was found, segmentation asks which pixels belong to it, and phenotyping classifies the cell afterwards; finding a nucleus does not give a reliable whole-cell boundary. Intersection over union is the area two masks share divided by the area either covers, from 0 to 1, and a single value can hide missed or merged cells, so check object counts and characteristic errors too. A mask assigns existing signals to estimated regions and is not a measurement of a new marker. Merging, splitting and missing cells distort counts, marker combinations and spatial relations, so overlay masks on the original channels.
- Fictional worked example (two stipulated cells). One mask covers a small T cell and an adjacent cancer cell, so one table row reports both an immune marker and keratin; that is a merged object, not a hybrid cell. A split into three objects for two cells adds objects, not cells. Correcting the mask may show two neighbouring cells, and their proximity still does not prove immune recognition.
- Figure reading (synthetic, light and dark renders agree): the marker channels A (immune) and K (keratin) stay fixed while a merged mask gives one object, a split gives three, and a shifted border assigns signals to the wrong object; the table changes and the image does not. An overlay does not prove identity or function.
- Analysis card in words (a reading guide). Input and tissue cost: existing image channels and scale information; computation consumes no additional tissue, though upstream staining or imaging may already have consumed material. Output and units: labelled masks, object counts, estimated cell area in square micrometres when calibrated, coordinates and distances in micrometres, and unitless overlap scores. Controls: original-image overlays, representative manual annotations, held-out images, checks by cell type and region, and recorded software, model and settings. Thresholds: error tolerances set for the downstream task. Failure modes: merged, split or missing cells, weak boundary stains, misaligned channels, debris, and tissue or stain differences from the development data. Cannot tell you: new molecular signals, complete cell outlines outside the section, receptor specificity, interaction or killing. Validation context: an image-analysis component validated for a defined tissue, acquisition and task.

## Sources

- Whole-cell segmentation of tissue images with human-level performance using large-scale data annotation and deep learning.: https://doi.org/10.1038/s41587-021-01094-0
- Impact and correction of segmentation errors in spatial transcriptomics.: https://doi.org/10.1038/s41588-025-02497-4
- Stringer et al., Cellpose: a generalist algorithm for cellular segmentation (Nature Methods 2021): https://doi.org/10.1038/s41592-020-01018-x
- Petukhov et al., Cell segmentation in imaging-based spatial transcriptomics (Nature Biotechnology 2022): https://doi.org/10.1038/s41587-021-01044-w

## Connected records

- technologies: [Imaging mass cytometry](https://onco.cc/technologies/imaging-mass-cytometry/), [Spatial transcriptomics](https://onco.cc/technologies/spatial-transcriptomics/)
- terms: [Immune measurements and product evidence](https://onco.cc/terms/immune-product-evidence/), [Reverse-phase protein array (RPPA)](https://onco.cc/terms/rppa/), [Tumour purity](https://onco.cc/terms/tumour-purity/)

---
JSON: https://onco.cc/api/v1/entities/cell-segmentation.json