Ground-Truth Free Meta-Learning for Deep Compressive Sampling

Improved SURE (iSURE) loss for meta-learning and adaption iSURE-based GT-free model-agnostic meta-learning (MAML) Nullspace-consistent model adaptation Unrolling CNN with bias-tuning iSURE is a noisy form of SURE to provide robust estimation in Range 𝚽 𝐇 . ℓ SURE 𝝎; 𝒚, 𝚽 ≔ 𝚽ℱ 𝝎 𝒚, 𝚽 -𝒚 2 2 + 2𝜎tr(𝚽 + (𝜕ℱ 𝝎 𝒚), 𝚽 /𝜕𝒚)) ℓ iSURE 𝝎; 𝒚, 𝚽, 𝝐 ′ ≔ 𝚽ℱ 𝝎 𝒚 + 𝝐′, 𝚽 -𝒚 2 2 + 2𝜎tr(𝚽 + (𝜕ℱ 𝝎 𝒚 + 𝝐′), 𝚽 /𝜕𝒚)) Let 𝑱 𝝎 be the Jacobian matrix w.r.t. 𝝎, i.e. 𝑱 𝝎 𝓕 𝝎 = 𝜕𝓕 𝝎 /𝝏𝝎 and 𝒚 = 𝚽𝐱 + 𝝐. Assume 𝝐, 𝝐′ ∽ 𝒩 𝟎, 𝝈 2 𝑰 are independent. Then, we have: ∇ 𝝎 𝔼 𝒚,𝝐′ ℓ iSURE 𝝎; 𝒚, 𝚽, 𝝐 ′ = 2𝔼 𝒚,𝝐′ 𝑱 𝝎 (ℱ 𝝎 𝒚 + 𝝐 ′ , 𝚽 𝚽 + (𝚽ℱ 𝝎 𝒚 + 𝝐 ′ , 𝚽 -𝒚 + 𝝐 ′ ) . Theorem 1 • Noise injection mitigates overfitting in GT-free meta-learning and model adaption, as well as allows ensemble in learning and inference. • iSURE allows efficient gradient update, without using MCMC.