Bridging the Static-Dynamic Divide: How LLM Uncertainty Quantification Affects Live Trading Performance

TL;DR: Imagine LLMs as traders who freeze under pressure—this study tests whether teaching them to say "I don't know" when uncertain improves real-world trading. We'll integrate DeLLMa's uncertainty reasoning into LiveTradeBench agents and measure if adaptive abstention reduces losses during market volatility spikes.

Research Question: Why do high-performing LLMs in static benchmarks (e.g., LMArena) fail in live trading, and can uncertainty-aware decision-making bridge this gap?

Hypothesis: LLMs with calibrated uncertainty quantification (e.g., conformal abstention policies) will outperform standard LLMs in volatile markets by avoiding high-risk trades during ambiguous information states.

Experiment Plan: - Setup: Modify LiveTradeBench agents to incorporate DeLLMa's multi-step uncertainty reasoning and Tayebati et al.'s conformal abstention thresholds.

• Data: Use LiveTradeBench's live feeds + synthetic volatility events (e.g., Fed announcements) to stress-test agents.
• Metrics: Compare Sharpe ratios, maximum drawdown, and "abstention frequency" vs. benchmark LLMs.
• Expected Outcome: Uncertainty-aware agents will show 15%+ lower drawdowns during high-volatility periods by dynamically reducing exposure.

References: ['Yu, H., Li, F., & You, J. (2025). LiveTradeBench: Seeking Real-World Alpha with Large Language Models.', 'Liu, O., Fu, D., Yogatama, D., & Neiswanger, W. (2024). DeLLMa: Decision Making Under Uncertainty with Large Language Models. ICLR.', 'Tayebati, S., et al. (2025). Learning Conformal Abstention Policies for Adaptive Risk Management in LLMs/VLMs. arXiv.']