Uncertainty-Aware Tool Recruitment: Quantifying Disagreement and Confidence in Multi-Modal Debates

TL;DR: Let’s teach agents to not just disagree, but to measure how unsure they are—and use this uncertainty to decide which tools to call, making debates smarter and more data-efficient.

Research Question: How can explicit uncertainty quantification of agent and tool outputs enhance the DART framework’s tool recruitment, leading to more reliable multi-agent multimodal reasoning?

Hypothesis: Incorporating agent and tool uncertainty metrics (e.g., predictive entropy, calibration error) into the disagreement measurement will enable more nuanced and effective tool recruitment, reducing unnecessary calls and improving answer reliability.

Experiment Plan: Augment DART so each agent and tool provides a calibrated uncertainty score with its predictions. Design a policy where only disagreements with high aggregate uncertainty trigger tool recruitment; low-uncertainty disagreements are deprioritized. Evaluate on VQA datasets: track number of tool calls, debate convergence, and final accuracy. Compare with standard DART (disagreement-driven) and with random tool invocation. Analyze whether uncertainty-aware recruitment leads to fewer, more impactful tool interventions and improves confidence in final answers.

References:

• Sivakumaran, N., et al. (2025). DART: Leveraging Multi-Agent Disagreement for Tool Recruitment in Multimodal Reasoning.
• Lu, M., Xu, R., Fang, Y., et al. (2025). Scaling Agentic Reinforcement Learning for Tool-Integrated Reasoning in VLMs.