With the widespread deployment of Multimodal Large Language Models (MLLMs) for visual-reasoning tasks, improving their safety has become crucial. Recent research indicates that despite training-time safety alignment, these models remain vulnerable to jailbreak attacks—carefully crafted image-prompt pairs that compel the model to generate harmful content. In this work, we first highlight a critical safety gap, demonstrating that alignment achieved solely through safety training may be insufficient against jailbreak attacks. To address this vulnerability, we propose Immune, an inference-time defense framework that leverages a safe reward model during decoding to defend against jailbreak attacks. Additionally, we provide a rigorous mathematical characterization of Immune, offering provable guarantees against jailbreaks. Extensive evaluations on diverse jailbreak bench- marks using recent MLLMs reveal that Immune effectively enhances model safety while preserving the model’s original capabilities. For instance, against text-based jailbreak at- tacks on LLaVA-1.6, Immune reduces the attack success rate by 57.82% and 16.78% compared to the base MLLM and state-of-the-art defense strategy, respectively.
: Improving Safety Against Jailbreaks in Multi-modal LLMs via
Inference-Time Alignment@misc{ghosal2024immuneimprovingsafetyjailbreaks,
title={Immune: Improving Safety Against Jailbreaks in Multi-modal LLMs via Inference-Time Alignment},
author={Soumya Suvra Ghosal and Souradip Chakraborty and Vaibhav Singh and Tianrui Guan and Mengdi Wang and Ahmad Beirami and Furong Huang and Alvaro Velasquez and Dinesh Manocha and Amrit Singh Bedi},
year={2024},
url={https://arxiv.org/abs/2411.18688},
}