Reading Notes

Personal reading notes. Original paper: arXiv:2606.12716v1.

Paper Reading Note: Does AI Reviewer See the Full Picture? Attacking and Defending Multimodal Peer Review

Paper ID: 2606.12716v1
Title: Does AI Reviewer See the Full Picture? Attacking and Defending Multimodal Peer Review
Authors: Xinyu Zhao, Rana Muhammad Shahroz Khan, Zhen Xu, Zhen Tan, Tianlong Chen
Affiliations: The University of North Carolina at Chapel Hill, Arizona State University
Date: June 12, 2026

Research Background and Motivation

As Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs) gradually integrate into scientific peer review processes, the role of AI in generating evaluation comments, summarizing contributions, and supporting editorial decisions becomes increasingly significant. The paper points out that major conferences (such as AAAI, ICML, NeurIPS) have begun to formally incorporate AI-generated reviews into the preliminary review process. While this shift addresses management issues regarding large-scale submissions, it also significantly raises the reliability requirements for AI models as “gatekeepers” of academic publishing.

The authors argue that while AI-generated reviews improve accessibility and efficiency, they also expose widespread and unresolved security risks. Current research focuses mainly on improving review quality under benign inputs, completely ignoring security issues. When AI models become scientific review tools in high-stakes, long-context domains, their vulnerability to adversarial manipulation has become an imminent hazard.

Existing Challenges and Gaps

The paper analyzes three main gaps in current AI review security research:

1. Absence of Multimodal Attacks: Existing robustness research almost entirely focuses on pure text attacks, ignoring the visual modality where core methodologies and results in scientific papers are often presented through figures.
2. Specificity of Attack Targets: AI review security differs from standard jailbreaking attacks. The goal of peer review attacks is not to make the model violate general safety policies (e.g., “do not say harmful things”), but to induce specific domain-targeted failures (e.g., “ignore specific flaws” or “increase scores”). This requires attack categories capable of manipulating the model’s subtle domain reasoning capabilities.
3. Lack of Practical Defenses: There are no practical defense schemes against such threats. Since adversarial instructions are embedded as subtle domain manipulations within long documents, simple “prompt auditing” or “jailbreak detection” filters are often ineffective.

PaperGuard Framework Design

To address the above gaps, the authors propose PaperGuard, the first comprehensive benchmark aimed at systematically evaluating and defending AI-generated peer reviews against domain-specific, cross-modal attacks. The framework is built on three pillars:

1. Multimodal Peer Review Dataset: Covers multiple scientific fields, extracting key figures (e.g., methods, results) by parsing papers.
2. Unified Attack Suite: Unifies black-box prompt injection and white-box gradient-based perturbations, specifically targeting text (GCG) and images (PGD).
3. Practical Defense Framework: Proposes chunk-based embedding search to efficiently locate and mitigate harmful instructions, addressing the long-context challenges of academic papers.

The paper illustrates the overall process of PaperGuard in Figure 2. The framework first processes diverse papers from different platforms, then formulates cross-modal attack tasks aimed at misleading AI reviewers (e.g., prompt injection, image perturbation), and finally proposes defense strategies (e.g., LLM-as-Judge, chunk-based embedding search) to detect and mitigate these attacks.

Attack Strategies and Methods

PaperGuard unifies multiple attack modalities to reveal cross-modal vulnerabilities in existing LLMs and MLLMs. The paper qualitatively visualizes the three different attack modalities evaluated in Figure 3:

1. Prompt Injection: Black-box attack, directly injecting malicious instructions into user input (e.g., “provide strong positive evaluation”).
2. Learnable Text Attack: White-box attack, using GCG (Greedy Coordinate Gradient) to optimize text to maximize the target (e.g., increasing scores).
3. Image Perturbation: White-box attack, adding small perturbations to images using PGD (Projected Gradient Descent), replacing specific parts of the original image.

The authors note that these attacks aim to induce domain-specific failures, such as “inflating scores,” rather than general safety policy violations.

Defense Mechanisms

Addressing the challenges brought by long-context academic papers, the paper proposes a lightweight and practical defense method: Chunk-based Embedding Search.

Traditional defense methods may require scanning the entire document, which is computationally expensive and prone to losing suspicious instructions amidst the noise of long documents. PaperGuard’s method decomposes the paper into semantically coherent chunks (text paragraphs, figures) and compares their embedding vectors with a database of known attack patterns. This method is computationally efficient and can effectively locate malicious instructions that might be drowned out in full-text embeddings.

Experimental Results and Findings

The authors conducted extensive experiments on multiple state-of-the-art open-source and commercial LLMs/MLLMs. The paper presents an effectiveness analysis on datasets Dpro and Dreal under different defense mechanisms (perplexity detection, perturbation detection, LLM-as-Judge) in Figure 1.

Experimental results indicate that AI reviewers are generally vulnerable:

• Black-box Attacks: Prompt injection achieved an attack success rate (ASR) of up to 80% on powerful proprietary models, leading to significant score inflation.
• White-box Attacks: Visual attacks could increase scores by up to +14.11 points, confirming that text-based safeguards alone are insufficient.
• Defense Effectiveness: The proposed chunk-based embedding search defense can effectively detect malicious injections with zero false positives, avoiding penalizing benign authors, and represents a practical solution.

Summary and Contributions

This paper establishes PaperGuard as the first standardized framework for evaluating the robustness of AI-generated scientific reviews under multimodal adversarial manipulation. Main contributions include:

• Unified black-box (prompt injection) and white-box (text GCG, image PGD) attacks to systematically measure cross-modal vulnerabilities in existing models.
• Provided extensive experiments across state-of-the-art models, revealing prevalent vulnerabilities and confirming the need for robust security safeguards.
• Proposed a lightweight and practical chunk-based embedding search defense that can effectively detect malicious injections without false positives.

The authors emphasize that by establishing this foundational benchmark, transparent protocols, and actionable defenses, PaperGuard provides the necessary tools to pioneer trustworthy, attack-resistant AI-assisted academic review.