Loading
A masked autoencoder hides a random part of the input (image patches, or half the genes in a profile) and learns to reconstruct it from the rest.
An autoencoder learns an encoding that compresses the input and a decoding that recreates it (Wikipedia); the masked variant reconstructs only the hidden portion, which forces the encoder to learn the structure that relates parts of the input. For expression data the hidden parts are genes, so the model learns co-expression structure; for multimodal data one modality's token can be reconstructed from the others. It is the pretraining objective behind many pathology and transcriptome foundation models.
Shares Gene co-expression structure, Cancer AI vocabulary (CanSim terms map) and the tag cansim-terms.
Shares Self-supervised pretraining (SSL), Cancer AI vocabulary (CanSim terms map) and the tag cansim-terms.
Shares Self-supervised pretraining (SSL), Cancer AI vocabulary (CanSim terms map) and the tag cansim-terms.
Shares Loss functions: cross-entropy and mean squared error, Cancer AI vocabulary (CanSim terms map) and the tag cansim-terms.
Shares Loss functions: cross-entropy and mean squared error, Cancer AI vocabulary (CanSim terms map) and the tag cansim-terms.
Shares Gene co-expression structure, Cancer AI vocabulary (CanSim terms map) and the tag cansim-terms.
Shares Tokenisation (genes, tiles and sequence as tokens), Cancer AI vocabulary (CanSim terms map) and the tag cansim-terms.
Shares Tokenisation (genes, tiles and sequence as tokens), Cancer AI vocabulary (CanSim terms map) and the tag cansim-terms.