Geometry-Aware Update Control for Robust Optimization

Training-time corruption is a persistent source of instability in deep learning. Even when model architectures and data pipelines are fixed, a small number of abnormal optimization steps can accumulate drift, contaminate optimizer state, and permanently alter the training trajectory. Such failures often arise directly in the update space, where gradients become misaligned with descent, inject energy into irrelevant subspaces, or exhibit rare but extreme magnitudes. This work studies robustness from an update-centric perspective and characterizes a class of geometry-defined perturbations that disrupt optimization, including directional deviation, orthogonal energy injection, heavy-tailed noise, and sign reversal. We introduce Geometry-Aware Update Control (GAUC), an optimizer-side mechanism that intervenes between backpropagation and parameter updates. GAUC evaluates incoming updates using simple geometric consistency signals derived from recent descent history and selectively constrains their influence through projection-based correction and scale-aware, budgeted clipping. The design bounds the per-step impact of unreliable updates while preserving the geometry of benign descent directions. GAUC is agnostic to the choice of optimizer and loss function, requires only lightweight per-block state, and incurs linear-time overhead. Empirical results show that GAUC consistently stabilizes training under update-space perturbations while maintaining performance under clean conditions.