ForceForget: Reinforcement Concept Removal for Enhancing Safety in Text-to-Image Models

Reading Notes

Personal reading notes. Original paper: arXiv:2606.14351v1.

Paper Basic Information

• Paper Title: ForceForget: Reinforcement Concept Removal for Enhancing Safety in Text-to-Image Models
• Paper ID: 2606.14351v1
• Authors: Dong Han, Yong Li
• Published Conference: ICML 2026 (Proceedings of the 43rd International Conference on Machine Learning)
• Research Area: Generative AI Safety, Text-to-Image (T2I) Models, Concept Removal, Reinforcement Learning

Research Background and Motivation

With the rapid development of generative artificial intelligence, Text-to-Image (T2I) models (such as Stable Diffusion, MidJourney, DALL·E 2) have acquired the capability to generate diverse content. However, these models may still generate unsafe content, raising concerns about content safety in both academia and industry.

The paper points out that existing methods for mitigating unsafe content generation have significant limitations:

1. Dataset Filtering: Removing harmful substances from training data may inadvertently affect downstream performance.
2. Post-processing Filtering: Filters based on predefined unsafe concepts are easily bypassed through reverse engineering.
3. Fine-tuning and Training-free Methods: Existing concept removal methods (such as ESD, SLD, SafeGen, etc.) tend to over-remove unsafe concepts while suppressing benign concepts contained in harmful prompts (e.g., removing the “nude” concept may also suppress the benign concept of “human”), thereby negatively impacting the model’s utility.
4. Insufficiency in I2I Scenarios: Existing T2I concept removal methods perform poorly in Image-to-Image (I2I) tasks and cannot effectively address privacy and security risks in I2I scenarios.

To address the above issues, the authors propose a new framework aimed at eliminating unsafe content while maintaining the model’s interpretability of safe semantic meanings.

Core Method: ForceForget

The authors propose a concept removal framework named ForceForget. This method reformulates concept removal as a reward optimization problem in Reinforcement Learning (RL).

1. Reinforcement Learning Framework and Reward Optimization

Inspired by the success of using RL to adjust diffusion models for fuzzy goals (such as image compressibility, aesthetic quality), ForceForget utilizes dynamic reward updates to remove unsafe content. The paper designs specific Safety and Alignment Rewards, guiding the model to learn through optimizing the Concept Erasing Reward (CER).

2. Safe Adapter Module

To avoid over-erasure of content, the authors introduce a Safe Adapter into the cross-attention layers of the diffusion model. This module is used to project partial text embeddings, thereby achieving efficient regulation of concepts. In this way, the model can more precisely control the generation of specific concepts without affecting other benign semantics.

3. Overall Process

The paper illustrates the overall process of ForceForget. As shown in Figure 2, given the target concept to be removed in the prompt, the model continuously generates image samples and optimizes them through the designed reward mechanism.

Experimental Results and Analysis

The authors conducted extensive experiments on multiple datasets to verify the effectiveness of ForceForget in mitigating unsafe content generation, maintaining high fidelity of benign images, and robustness.

1. Balance between Safety and Utility

Existing methods often impair the model’s ability to generate human-oriented content when removing unsafe concepts. ForceForget, through RL optimization, better preserves benign concepts while removing unsafe content.

• Celebrity Generation Retention Rate: As shown in Figure 3, ForceForget maintains a high celebrity generation retention rate after removing unsafe concepts, indicating that it does not over-suppress benign features when handling prompts involving “people”.
• Safety Content Score: Figure 4 shows the VQAScore-SC metric, indicating that this method outperforms existing state-of-the-art (SOTA) methods in maintaining safe content quality.

2. Robustness and Adversarial Attacks

The paper evaluates the robustness of the method against attacks from red-teaming tools. Experiments show that ForceForget performs better than comparison methods when facing adversarial prompts aimed at bypassing safety mechanisms.

3. Transferability in Image-to-Image (I2I) Scenarios

Addressing the issue of poor performance of existing concept removal methods in I2I tasks, the authors demonstrate the superiority of ForceForget in this scenario.

• I2I Nudity Removal: As shown in Figure 6, in the I2I setting, ForceForget can effectively “purify” unwanted input concepts, removing nude content while maintaining the overall structure of the image.

4. Extension to General Concept Removal

In addition to safety, the authors extend the method to the removal of general concepts, such as art styles and specific objects.

• Art Style Removal: As shown in Figure 5, the method can remove specific art styles (e.g., Van Gogh style) while preserving other style features.
• Function Overview: Figure 1 summarizes multiple capabilities of ForceForget, including eliminating unsafe content, protecting artwork copyright, removing specific objects, and purifying input concepts in I2I settings.

Summary and Outlook

The paper argues that ForceForget is the first attempt to eliminate pornographic content generation by fine-tuning T2I diffusion models through Reinforcement Learning. Its main contributions include:

1. Pointing out that current SOTA concept removal methods suffer from over-erasure and damage to model utility.
2. Proposing the ForceForget framework combining RL reward optimization and Safe Adapter.
3. Verifying the advantages of the method in safety, utility, robustness, and I2I scenario transferability.
4. Demonstrating the generalization capability of the method in removing general concepts (such as art styles and objects).

This research provides new ideas for safety alignment in generative AI, particularly making progress in balancing content safety and model generation capabilities. The authors also state that the content discussion in the paper may contain sexually suggestive content offensive to some readers, but all images used are synthetic or from public datasets.