Sparse Relaxed-Lasso Steering: Automatic Sparse Autoencoder Feature Selection for Precise Image Editing

Precise, training-free image editing with text-to-image diffusion models requires balancing alignment (faithful realization of the target attribute), consistency (preserving non-target content), and quality (maintaining sharp, artifact-free textures). Sparse autoencoder (SAE) steering offers interpretable, smooth ``slider-like'' control by manipulating SAE feature activations derived from the text encoder; however, existing approaches rely on heuristic feature selection and manual steering-strength tuning, leading to suboptimal trade-offs among the three objectives. We propose Sparse Relaxed-Lasso Steering (SRLS), which casts steering-vector discovery as a convex sparse recovery problem. Exploiting the affine structure of the SAE decoder, SRLS automatically identifies sparse, generalizable support sets via a Lasso objective and then debiases the coefficients using support-restricted ridge refitting. We further replace manual strength tuning with a fixed-budget Bayesian optimization procedure. Across diverse attributes and subjects, SRLS improves the alignment--consistency--quality trade-off over competing methods.