Self-Ensembling Techniques in Semi-Supervised Learning

Self-ensembling has become a central design pattern for deep semi-supervised learning: a model is trained not only to fit scarce labeled examples, but also to agree with stable targets produced from its own perturbed predictions, historical predictions, or weight-averaged teacher. This paper presents Agreement-Gated Self-Ensembling Review and Refinement (AGSER), a compact framework for analyzing and improving such methods under label scarcity, unlabeled-set shift, and class-dependent confidence imbalance. AGSER separates three decisions that are often coupled in practice: how teacher targets are formed, which unlabeled examples are admitted into the unsupervised loss, and how class-level thresholds are adapted as learning proceeds. The method combines an exponential-moving-average teacher with a two-view agreement gate, class-aware confidence smoothing, and a small audit table that records accepted, rejected, and deferred unlabeled examples. A controlled study on CIFAR-10, SVHN, STL-10, and a shifted unlabeled-pool variant compares temporal ensembling, Mean Teacher, MixMatch, FixMatch-style confidence selection, and adaptive-threshold variants. The results indicate that agreement gates improve stability in the first third of training and reduce high-confidence errors under unlabeled-set shift, while adaptive class thresholds recover examples from weakly learned classes. The paper argues that self-ensembling should be treated as a target-selection and calibration problem, not only as an augmentation-consistency loss.