{"entity":{"id":"hdf5-zarr-parquet","kind":"term","name":"Array and table formats: HDF5, Zarr, OME-Zarr, Parquet","aka":["HDF5","HDF5 file","Zarr","Zarr array","OME-Zarr","OME-NGFF","Parquet","Apache Parquet","Parquet file","columnar storage","chunked array"],"tldr":"HDF5 and Zarr store large numerical arrays in chunks (Zarr is the cloud-friendly one, and OME-Zarr its bio-imaging profile for slide tiles); Parquet stores tables column by column for fast analytical queries.","summary":"Hierarchical Data Format is a set of file formats for storing and organising large amounts of data, maintained by The HDF Group (Wikipedia); ARCHS4 distributes its million-sample expression matrix as HDF5. Zarr is an open standard for chunked multidimensional arrays designed for random access in cloud storage (Wikipedia), and OME-NGFF, or OME-Zarr, is the Open Microscopy Environment's next-generation file format for bio-imaging, including whole-slide pyramids. Apache Parquet is a column-oriented storage format for analytical queries (Wikipedia), used for clinical and metadata tables in DuckDB or Spark.","asOf":"2026-09-24","wikipedia":"https://en.wikipedia.org/wiki/Hierarchical_Data_Format","links":[{"label":"Wikipedia: Zarr","url":"https://en.wikipedia.org/wiki/Zarr_(data_format)"},{"label":"OME-NGFF (OME-Zarr) specification","url":"https://ngff.openmicroscopy.org/latest/"},{"label":"Wikipedia: Apache Parquet","url":"https://en.wikipedia.org/wiki/Apache_Parquet"},{"label":"Wikipedia","url":"https://en.wikipedia.org/wiki/Hierarchical_Data_Format"}],"tags":["cansim-terms"],"related":["cancer-ai-vocabulary"],"cancers":[],"sections":[],"technologies":[],"targets":[],"drugs":[],"companies":[],"institutions":[],"pathways":[],"terms":["anndata-h5ad","digital-pathology-wsi","duckdb-catalog"],"trials":[],"people":[],"bottlenecks":[],"keyPapers":[],"journals":[],"dependsOn":[],"notes":["Listed in the CanSim terms map 1.0.0 (docs/onco/terms.json, generated 2026-09-24), CC BY 4.0, attribution: CanSim project, an open, public-data-first cancer foundation-model programme; CanSim page path /terms/hdf5."],"provenance":{"editedBy":"OnCo CanSim terms wave (Wikipedia summaries, standards and project pages, GDC and FDA pages, Europe PMC)","editedOn":"2026-09-24","note":"CanSim terms map 1.0.0 (docs/onco/terms.json, generated 2026-09-24), CC BY 4.0, attribution: CanSim project, an open, public-data-first cancer foundation-model programme"},"category":"Methods and models"},"route":"/terms/hdf5-zarr-parquet/","neighbours":{"term":[{"id":"duckdb-catalog","kind":"term","name":"Analytical databases as a data catalogue (DuckDB, PostgreSQL)","route":"/terms/duckdb-catalog/"},{"id":"anndata-h5ad","kind":"term","name":"AnnData and h5ad files","route":"/terms/anndata-h5ad/"},{"id":"cancer-ai-vocabulary","kind":"term","name":"Cancer AI vocabulary (CanSim terms map)","route":"/terms/cancer-ai-vocabulary/"},{"id":"digital-pathology-wsi","kind":"term","name":"Digital pathology and whole-slide images (WSI)","route":"/terms/digital-pathology-wsi/"}]}}