Reading Notes

Personal reading notes. Original paper: arXiv:2606.10307v1.

arXiv Paper Reading Note: Early-Token Confidence Predicts Reasoning Quality in Multi-Agent LLM Debate

Paper Basic Information

• Paper ID: 2606.10307v1
• Title: Early-Token Confidence Predicts Reasoning Quality in Multi-Agent LLM Debate
• Authors: Ali Keramati, Justin Cheok, Jacob Horne, Mark Warschauer
• Affiliation: University of California, Irvine
• Date: June 9, 2026

Research Background and Problem Definition

With the development of Large Language Models (LLMs), Multi-Agent Systems have demonstrated significant advantages in solving complex tasks. The authors point out that by decomposing problems into role-specific subtasks, such systems improve performance, robustness, and consistency in applications involving reasoning, planning, and automated decision-making. Among these, Debate serves as an interaction paradigm that encourages the exploration of diverse reasoning paths and exposes errors that might be hidden in single-agent trajectories by stimulating supporting and opposing arguments.

However, evaluating reasoning quality in multi-agent LLM systems is challenging, particularly in open-ended tasks without reference answers. The paper argues that while rubric-based scoring provides a concrete evaluation scenario (such as Automated Essay Scoring, AES), debate adds system complexity by introducing multiple agents, multiple messages, and potential risks of subtle procedural failures.

Existing evaluation methods primarily rely on the “LLM-as-judge” paradigm, where an independent language model scores generated outputs based on predefined criteria. Although this has become a scalable alternative to human evaluation, the authors note that it only provides an external quality signal. A key open question is: to what extent do these judgments reflect the true reliability of the underlying reasoning process? Specifically, can internal signals within the generative model be identified that correlate with reasoning quality as assessed externally?

Methodology and Experimental Framework

To address the above issues, the authors propose using token-level log-probabilities generated during decoding as an operationalized metric for model confidence. The paper hypothesizes that if an agent follows a more coherent and evidence-based reasoning path, the model should assign higher probability mass to the tokens it generates, resulting in a more confident logprob trajectory.

Multi-Agent Debate Architecture

The paper constructs a framework based on multi-agent debate, combined with an LLM-as-judge meta-evaluation module for reasoning analysis. The overall framework is shown in Figure 1.

The framework consists of two main stages:

1. Debate Stage: The Advocate and Skeptic generate opposing arguments for a given essay–rubric pair, while exposing token-level log-probabilities as internal confidence signals.
2. Evaluation Stage: An independent meta-evaluator scores each argument based on rubric-based dimensions (such as instruction following, argument quality, and evidence grounding).

Agent Role Settings

The debate framework consists of three specialized agents that interact sequentially for each essay–rubric pair:

• Advocate: Initiates the debate, constructing arguments that emphasize the strengths of the essay relative to rubric traits, using only supporting evidence from the essay text, without assigning scores.
• Skeptic: Responds by identifying limitations or weaknesses of the essay relative to the same standards, generating evidence-based rebuttals to challenge the Advocate’s claims, also without assigning scores.
• Synthesizer-Judge Scorer: Reads the completed dialogue record and generates final trait-level scores. Since this agent performs a constrained decision task, its output can be directly evaluated for accuracy against true scores; therefore, this study primarily focuses on the open-ended reasoning produced by the Advocate and Skeptic.

Confidence Signal Calculation

The authors use token-level log-probabilities obtained during generation to estimate model confidence. For a generated response containing $T$ tokens, the model produces a log-probability at each decoding step: $\ell_t = \log p(t_t | t_{<t}, x)$ where $x$ is the prompt context and $t_{<t}$ represents the preceding tokens. The study aims to analyze whether these token-level probability signals correlate with the reasoning quality of agents as evaluated externally.

Core Experimental Findings

Early Token Confidence is a Strong Predictor

The paper compares confidence proxies with rubric-based judge scores on two ASAP essay datasets. The authors find that early token confidence, particularly confidence within the first few generated tokens, is the most consistent predictor of reasoning quality, outperforming full-sequence statistics.

Analysis of log-probability trajectories shows that the initial phase of generation is the most heterogeneous and therefore the most informative. This means that the model’s state of uncertainty or determinacy at the start of generation can well predict the overall quality of subsequent reasoning.

Confidence Trajectory Analysis

Figure 2 shows the token-level log-probability trajectories for Advocate and Skeptic responses on Essay Set 7 (left) and Essay Set 8.

By observing these trajectories, the authors further validate the importance of the early generation phase. The trajectory plots reveal probability distribution differences between different roles and different essay sets, supporting the conclusion that early decoding dynamics serve as a lightweight and effective signal.

Systematic Asymmetry Between Agent Roles

The paper also observes systematic asymmetry between agent roles. The authors note that the alignment between confidence and quality is stronger in supportive reasoning (Advocate) than in adversarial criticism (Skeptic). This implies that for the Advocate, high confidence is more likely to correspond to high-quality reasoning, whereas for the Skeptic, this correlation may be weaker. This finding suggests that different roles may have differences in cognitive load or generation strategies during the debate, affecting the reliability of the confidence signal.

Distribution Characteristics of Evaluation Dimensions

Regarding evaluation dimensions, Figure 3 shows the distribution of LLM-as-judge scores across different evaluation dimensions.

The authors note that the Evidence grounding dimension exhibits high saturation. This means that during the evaluation process, scores in this dimension may lack sufficient discrimination, or most samples reach a high or low level in this dimension, which could affect confidence correlation analysis based on this dimension.

Conclusion and Implications

In summary, this study explores the evaluation of reasoning quality in multi-agent LLM debate systems. The authors propose that internal confidence signals (token-level log-probabilities) can predict reasoning quality as evaluated by LLM-as-judge.

Main conclusions include:

1. Importance of Early Signals: Early token confidence is the strongest predictor of reasoning quality, outperforming full-sequence statistics.
2. Heterogeneity of Generation Phase: The initial phase of generation is the most informative.
3. Role Asymmetry: Supportive reasoning shows stronger confidence-quality alignment than adversarial criticism.

These results indicate that early decoding dynamics provide a lightweight and effective signal for estimating reasoning reliability in multi-agent LLM systems. This finding has important practical implications for optimizing the evaluation layer of multi-agent systems, reducing reliance on external evaluation, and improving system reliability.