Git Re-Basin: Merging Models modulo Permutation Symmetries by Ainsworth, S. K., Hayase, J., & Srinivasa, S. (2022)

source

(Ainsworth et al. 2022)

tags

Neural networks

Summary

This paper introduces various methods for matching and interpolating the weights of multiple neural networks of the same architecture trained from different starting points or data. These neural networks have different weight values after the training.

Comments

Bibliography

1. Samuel K. Ainsworth, Jonathan Hayase, Siddhartha Srinivasa. September 11, 2022. "Git Re-basin: Merging Models Modulo Permutation Symmetries". arXiv. DOI.