GLID$^2$E: A Gradient-Free Lightweight Fine-tune Approach for Discrete Biological Sequence Design

Diffusion models have emerged as powerful tools for biological sequence design, offering flexible conditional generation for engineering functional biomolecules. While reinforcement learning (RL)-based fine-tuning enables multi-objective optimization on limited data, existing methods face a critical trade-off: gradient-free approaches suffer from training instability in discrete spaces, whereas gradientbased methods incur prohibitive computational costs. This trade-off severely limits their practical applicability in biological design tasks. We propose GLID 2 E, a light-weight gradient RL framework that achieves stable and efficient fine-tuning of discrete diffusion models. Our key insight is to constrain the exploration space through a clipped likelihood mechanism while employing reward shaping to align generation with design objectives. This combination mitigates the inherent instabilities in RL-guided diffusion while maintaining computational efficiency. We demonstrate GLID 2 E's effectiveness on DNA and protein sequence design benchmarks, where it matches or exceeds the performance of gradient-based methods while requiring significantly lower computational resources. Our approach provides a practical solution for function-driven biological sequence optimization. The code is available at: https://github.com/chq1155/GLID2E.