Optimization Dynamics Imprint Semantic Specificity in Contrastive Embedding Norms
By Ziwei Su · Paper · stat.ML
Contrastive embedding models trained with scale-invariant losses are typically paired with distance metrics like cosine similarity, effectively ignoring embedding magnitudes. However, surprisingly, empirical studies reveal that despite this, these "discarded" norms seem to correl
Pytorch · Simclr · Moco · Barlowtwins