# Array and table formats: HDF5, Zarr, OME-Zarr, Parquet

Source: https://onco.cc/terms/hdf5-zarr-parquet/  
OnCo record `hdf5-zarr-parquet` (Term). Data CC BY-NC 4.0, attribute "Data from OnCo (onco.cc)"; commercial use needs a licence.

## TL;DR

HDF5 and Zarr store large numerical arrays in chunks (Zarr is the cloud-friendly one, and OME-Zarr its bio-imaging profile for slide tiles); Parquet stores tables column by column for fast analytical queries.

## Summary

Hierarchical Data Format is a set of file formats for storing and organising large amounts of data, maintained by The HDF Group (Wikipedia); ARCHS4 distributes its million-sample expression matrix as HDF5. Zarr is an open standard for chunked multidimensional arrays designed for random access in cloud storage (Wikipedia), and OME-NGFF, or OME-Zarr, is the Open Microscopy Environment's next-generation file format for bio-imaging, including whole-slide pyramids. Apache Parquet is a column-oriented storage format for analytical queries (Wikipedia), used for clinical and metadata tables in DuckDB or Spark.

## Fields

- Kind: Term
- Last checked: 2026-09-24
- Also known as: HDF5; HDF5 file; Zarr; Zarr array; OME-Zarr; OME-NGFF; Parquet; Apache Parquet; Parquet file; columnar storage; chunked array
- Tags: cansim-terms

## Notes

- Listed in the CanSim terms map 1.0.0 (docs/onco/terms.json, generated 2026-09-24), CC BY 4.0, attribution: CanSim project, an open, public-data-first cancer foundation-model programme; CanSim page path /terms/hdf5.

## Sources

- Wikipedia: https://en.wikipedia.org/wiki/Hierarchical_Data_Format
- Wikipedia: Zarr: https://en.wikipedia.org/wiki/Zarr_(data_format)
- OME-NGFF (OME-Zarr) specification: https://ngff.openmicroscopy.org/latest/
- Wikipedia: Apache Parquet: https://en.wikipedia.org/wiki/Apache_Parquet
- Wikipedia: https://en.wikipedia.org/wiki/Hierarchical_Data_Format

## Connected records

- terms: [Analytical databases as a data catalogue (DuckDB, PostgreSQL)](https://onco.cc/terms/duckdb-catalog/), [AnnData and h5ad files](https://onco.cc/terms/anndata-h5ad/), [Cancer AI vocabulary (CanSim terms map)](https://onco.cc/terms/cancer-ai-vocabulary/), [Digital pathology and whole-slide images (WSI)](https://onco.cc/terms/digital-pathology-wsi/)

---
JSON: https://onco.cc/api/v1/entities/hdf5-zarr-parquet.json