DELMAN: Dynamic Defense Against Large Language Model Jailbreaking with Model Editing

Yi Wang 1 Fenghua Weng 1 Sibei Yang 2 Zhan Qin 3 Minlie Huang 4 Wenjie Wang 1†

1ShanghaiTech University

ASPIRE — AI Security, Privacy and Robustness
2Sun Yat-sen University3Zhejiang University4Tsinghua University

†Corresponding author.

Paper Code arXiv ACL Anthology

Impact

Safety that can be updated after release.

Jailbreak threats continue after a model is released. DELMAN uses targeted model editing to associate harmful token representations with safe responses, without broad retraining or additional inference-time defense modules.

Across five language models, DELMAN substantially reduces jailbreak success while retaining most general-task utility. Sequential editing experiments further show that later updates can largely preserve the effects of earlier safety edits.

Before and after DELMAN

Qwen2.5-7B-Instruct: safety and general utility before and after editing.
Metric Before After
Jailbreak success · Lower is better
Average ASR (%) 63.4 1.1
General utility · Higher is better
MT-Bench 8.82 8.48
NLI 0.835 0.833
Reasoning 0.987 0.988

Table 1 · ASR averages cover three attacks across four benchmarks.

Overview

DELMAN in the Model Lifecycle

Figure 1
DELMAN introduces dynamic defense during deployment, after foundational and downstream safety alignment. A targeted model edit changes the model's harmful response into a safe refusal.

Why Do Deployed Models Need Safety Updates?

Models may undergo foundational and downstream safety alignment before reaching users, yet new jailbreak attempts can still emerge during deployment. DELMAN addresses these threats through localized weight updates, allowing further safety edits as new attacks are identified.

Methodology

How Does DELMAN Edit a Model?

Five-step DELMAN editing process: identify harmful tokens, generate varied contexts, average harmful key representations, optimize safe values with utility regularization, and update MLP down-projection weights.

From Harmful Tokens to Safe Responses

DELMAN identifies harmful tokens, computes representative key vectors across different contexts, optimizes safe target values, and updates selected MLP down-projection weights.
Optimization and Weight UpdateEqs. (5) and (8)

The target value objective combines safe refusal supervision with a KL regularizer for benign prompts:

v∗=arg⁡min⁡v[Lsafe+λLutility].v^* = \arg\min_v\left[\mathcal{L}_{\mathrm{safe}} + \lambda\mathcal{L}_{\mathrm{utility}}\right].

The MLP update encodes the new key–value mappings while using cached covariance statistics to preserve existing associations:

W^down=Wdown+RDKD⊤(C+KDKD⊤)−1.\widehat{W}_{\mathrm{down}} = W_{\mathrm{down}} + R_D K_D^\top\left(C + K_D K_D^\top\right)^{-1}.

In practice, updates are distributed across selected layers.

Results

How Does DELMAN Balance Safety and Utility?

Compare model performance

Jailbreak attacks · ASR ↓

Table 8
Llama2-7B-chat: Jailbreak attacks · ASR ↓. Table 8.
Method HarmBench AdvBench JailbreakBench MaliciousInstruct
GCG AutoDAN PAIR GCG AutoDAN PAIR GCG AutoDAN PAIR GCG AutoDAN PAIR
Original Model 42% 23% 2% 39% 19% 1% 46% 27% 4% 45% 30% 0%
LoRA 13% 1% 2% 2% 0% 0% 50% 1% 2% 32% 0% 0%
SafeDecoding 0% 0% 1% 4% 0% 4% 1% 0% 3% 1% 0% 0%
LED 2% 2% 1% 0% 1% 0% 8% 2% 4% 8% 2% 1%
DELMAN (Ours) 0% 0% 0% 0% 0% 0% 0% 0% 0% 1% 0% 0%

Attack success rate (ASR, %), lower is better. Bold values mark the lowest ASR among defense methods, including ties. Attack success is judged by the HarmBench classifier.

Can Safety Edits Accumulate over Time?

DELMAN applies four safety edits sequentially, each targeting a different harmful behavior category. In both cases, overall HarmBench ASR decreases, while previously edited categories maintain low ASR after subsequent updates.

Four successive safety edits

Figure 8
Case 1: HarmBench attack success over four successive DELMAN edits, showing the full dataset and individual edited behavior categories.
Case 2: HarmBench attack success over four successive DELMAN edits, showing the full dataset and individual edited behavior categories.

Sequential editing on Llama2-7B-chat under GCG. Each case applies four successive edits, with ASR tracked across the full HarmBench dataset and all six behavior categories.

Sequential edits: safety and utilityTable 10
Editing strategyMT-Bench ↑ HarmBench
ASR (%) ↓ 		AdvBench
ASR (%) ↓ 		JailbreakBench
ASR (%) ↓ 		MaliciousInstruct
ASR (%) ↓
DELMAN6.310%0%0%1%
Sequential · Case 16.353%0%10%0%
Sequential · Case 26.644%5%6%0%

Llama2-7B-chat under GCG. DELMAN uses the full editing set, while sequential variants apply four category specific edits.

How Does DELMAN Generalize Beyond the Editing Data?

We examine how DELMAN generalizes across harmful behavior categories and model scales, and visualize shared key representations across behaviors and datasets.

Do edits transfer across harmful behaviors?Figure 6

We edit one HarmBench behavior category at a time and evaluate all six categories. The heatmaps show that targeted edits can also reduce attack success on categories not used for editing.

Llama2-7B cross-category attack success after a single-category edit under GCG.
Llama2-7B-chat · GCG
Llama2-7B cross-category attack success after a single-category edit under AutoDAN.
Llama2-7B-chat · AutoDAN

Columns denote editing categories, and rows denote evaluated categories. Off-diagonal cells show transfer to other categories. Original ASR values appear in parentheses. ASR is shown on a 0–1 scale (lower is better).

What do harmful key representations show?Figure 7

PCA projections of harmful keys at the selected editing layer show substantial overlap across behavior categories and datasets. This suggests shared representation patterns among harmful inputs.

PCA projections of harmful token keys, colored by behavior category.
Across harmful behaviors
PCA projections of harmful token keys, colored by dataset.
Across datasets
Additional models and scalesTable 9
ModelMethodHarmBenchAdvBenchJailbreakBenchMaliciousInstruct
Qwen3-8B Original Model 63%38%53%90%
DELMAN 1% 0% 3% 5%
Llama2-13B-chat Original Model 14%16%17%18%
DELMAN 1% 0% 0% 0%
Qwen2.5-14B-Instruct Original Model 69%42%68%93%
DELMAN 3% 0% 5% 3%

GCG attack success rate (ASR, %), lower is better.

Editing Cost and Deployment

Training time ↓

0.4 h

DELMAN requires 0.4 hours of training on Vicuna-7B-v1.5, compared with 0.6 hours for SafeDecoding, 1.5 hours for LoRA, and 6.2 hours for LED.

Vicuna-7B-v1.5 · Single NVIDIA A40 GPU · Averaged over five runs.

LoRA 1.5 h
SafeDecoding 0.6 h
LED 6.2 h
DELMAN 0.4 h

Table 2 · Training time (hours), lower is better.

Citation

If you find DELMAN useful for your research, please consider citing our paper:

@inproceedings{wang2025delman,
  title={DELMAN: Dynamic defense against large language model jailbreaking with model editing},
  author={Wang, Yi and Weng, Fenghua and Yang, Sibei and Qin, Zhan and Huang, Minlie and Wang, Wenjie},
  booktitle={Findings of the Association for Computational Linguistics: ACL 2025},
  pages={11465--11481},
  year={2025},
  doi={10.18653/v1/2025.findings-acl.598},
  url={https://aclanthology.org/2025.findings-acl.598/}
}