MSc · Supervisor · Completed 2025 · Stellenbosch University
A Contrastive Approach to Weight Space Learning
Dewald (DJ) Swanevelder
A contrastive framework that jointly embeds neural network weights, the datasets they are trained on, and their performance metrics into a shared space — using separate weight and dataset encoders with learned performance-bin embeddings. The shared space supports interpretability and conditional model sampling, approximating P(W | D, R). The work surfaces limitations of current weight-encoding strategies (notably linear PCA compression) while demonstrating the promise of aligning these heterogeneous modalities for conditional model generation.