{"entity":{"id":"multimodal-fusion","kind":"term","name":"Multimodal fusion (early, late, modality dropout)","aka":["multimodal fusion","multimodal model","multimodal learning","modality fusion","early fusion","late fusion","modality dropout","attention masking over modalities","missing modalities"],"tldr":"Multimodal fusion combines several kinds of data about one patient (slides, expression, mutations, clinical variables) in one model; modality dropout randomly hides modalities during training so the model still works when some are missing.","summary":"Multimodal learning integrates and processes multiple data types, or modalities, in one deep model (Wikipedia). Patient data is rarely complete, so fusion models mask absent modalities in attention and use modality dropout, an application of dropout regularisation (Wikipedia), to stay robust. PORPOISE fused histology and genomics for pan-cancer prognosis; the recurring finding is that when one modality (expression) already carries the signal, fusion adds little, and the honest test is each modality alone against the combination.","asOf":"2026-09-24","wikipedia":"https://en.wikipedia.org/wiki/Multimodal_learning","links":[{"label":"Wikipedia: dropout","url":"https://en.wikipedia.org/wiki/Dropout_(neural_networks)"},{"label":"Chen et al., PORPOISE: pan-cancer integrative histology-genomic analysis via multimodal deep learning (Cancer Cell 2022)","url":"https://doi.org/10.1016/j.ccell.2022.07.004"},{"label":"Wikipedia","url":"https://en.wikipedia.org/wiki/Multimodal_learning"}],"tags":["cansim-terms"],"related":["cancer-ai-vocabulary"],"cancers":[],"sections":[],"technologies":[],"targets":[],"drugs":[],"companies":[],"institutions":[],"pathways":[],"terms":["transformer-architecture","contrastive-learning","ablation-study","pca"],"trials":[],"people":[],"bottlenecks":[],"keyPapers":[],"journals":[],"dependsOn":[],"notes":["Listed in the CanSim terms map 1.0.0 (docs/onco/terms.json, generated 2026-09-24), CC BY 4.0, attribution: CanSim project, an open, public-data-first cancer foundation-model programme; CanSim page path /terms/multimodal-fusion."],"provenance":{"editedBy":"OnCo CanSim terms wave (Wikipedia summaries, standards and project pages, GDC and FDA pages, Europe PMC)","editedOn":"2026-09-24","note":"CanSim terms map 1.0.0 (docs/onco/terms.json, generated 2026-09-24), CC BY 4.0, attribution: CanSim project, an open, public-data-first cancer foundation-model programme"},"category":"Methods and models"},"route":"/terms/multimodal-fusion/","neighbours":{"term":[{"id":"ablation-study","kind":"term","name":"Ablation study and multi-task heads","route":"/terms/ablation-study/"},{"id":"cancer-ai-vocabulary","kind":"term","name":"Cancer AI vocabulary (CanSim terms map)","route":"/terms/cancer-ai-vocabulary/"},{"id":"contrastive-learning","kind":"term","name":"Contrastive learning (InfoNCE)","route":"/terms/contrastive-learning/"},{"id":"pca","kind":"term","name":"Principal component analysis (PCA) as a feature compressor","route":"/terms/pca/"},{"id":"transformer-architecture","kind":"term","name":"Transformer and attention","route":"/terms/transformer-architecture/"}]}}