Meta-Tool Use: Adaptive Tool Selection and Synthesis During Test-Time

Much of the current literature (e.g., Mosbach & Behnke, 2023) focuses on adapting the use of a fixed toolset. But what if, at test time, agents could not only adapt how they use tools, but which tools they use—or even synthesize entirely new tool-like behaviors by recombining existing primitives? Inspired by the diversity of capuchin monkey tool use when exposed to new environments (Falótico & Ottoni, 2023) and the compositional zero-shot learning in vision-language models (Zhou & Ma, 2024), this research would develop a dynamic meta-tool layer. At test time, the system could autonomously select, sequence, or blend available tools (physical or algorithmic) based on the demands of the novel scenario, possibly using reinforcement learning or meta-learning. This approach would challenge the assumption of a static toolset, opening the door to truly open-ended adaptation and innovation.

References:

1. Greater tool use diversity is associated with increased terrestriality in wild capuchin monkeys.. Tiago Falótico, E. Ottoni (2023). American Journal of Biological Anthropology.
2. Learning Generalizable Tool Use with Non-Rigid Grasp-Pose Registration. Malte Mosbach, Sven Behnke (2023). 2023 IEEE 19th International Conference on Automation Science and Engineering (CASE).
3. Test-Time Prompt Tuning for Compositional Zero-Shot Learning. Sicong Zhou, Jianhui Ma (2024). IEEE International Conference on Electro/Information Technology.