Synthetic training data

tags

Machine learning, LLM, Transfer learning, Neural network training

A way of self-training is to generate synthetic labels/data that are then used to retrain the model. This is for instance what happens in <&zhangEmbarrassinglySimpleSelfDistillation2026>.