Adaptive Deliberation Policies for Large Language Models in Tool-Using Agents: Balancing Reasoning Cost and Decision Quality
Keywords:
large language models, tool-using agents, adaptive reasoning, deliberation policy, reasoning cost, decision quality, cognitive architecture, governanceAbstract
Large language models are increasingly deployed as central reasoning components in autonomous agents that interact with external tools and APIs. While these agents benefit from the vast knowledge and generative capabilities of foundation models, their operational efficiency and decision quality are critically dependent on how much reasoning effort is expended before selecting an action. Existing approaches often apply a fixed reasoning budget, leading to either wasteful computation in simple scenarios or insufficient deliberation in complex, high-stakes environments. This paper develops a systems-level framework for adaptive deliberation policies in tool-using agents, drawing inspiration from dual-process theories of human cognition. We propose that reasoning depth can be dynamically modulated based on task difficulty, time constraints, and the availability of tool feedback. The framework integrates a meta-cognitive controller that monitors intermediate reasoning states and decides whether to commit to an action or to continue deliberation via further tool calls or internal chain-of-thought expansion. We analyze structural trade-offs between reasoning cost—including latency, token consumption, and API usage—and decision quality, defined as accuracy, robustness, and fairness of outcomes. Governance implications are explored, including the need for transparency in allocation of computational resources, mitigation of reasoning-induced biases, and alignment with human oversight norms. Through cross-domain comparisons spanning knowledge retrieval, software engineering, and medical diagnosis, we illustrate how adaptive policies can yield Pareto improvements over static baselines. Deployment challenges around infrastructure heterogeneity, safety constraints, and real-time monitoring are discussed. The paper concludes with a research agenda for socio-technical design of deliberative agent systems that are both effective and responsible.
References
1. Schick, T., Dwivedi-Yu, J., Dessì, R., Raileanu, R., Lomeli, M., Zettlemoyer, L., ... & Scialom, T. (2024). Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems, 36.
2. Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., & Cao, Y. (2023). ReAct: Synergizing reasoning and acting in language models. International Conference on Learning Representations.
3. Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., ... & Liang, P. (2022). On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258.
4. Kahneman, D. (2011). Thinking, fast and slow. Farrar, Straus and Giroux.
5. Schuster, T., Fisch, A., Gupta, J., Dehghani, M., Radev, D., & Kozareva, Z. (2022). Confident adaptive language modeling. Advances in Neural Information Processing Systems, 35, 17056-17069.
6. Zhou, D., Schärli, N., Hou, L., Wei, J., Scales, N., Wang, X., ... & Chi, E. H. (2023). Least-to-most prompting enables complex reasoning in large language models. International Conference on Learning Representations.
7. Liang, P., Bommasani, R., Lee, T., Tsipras, D., Soylu, D., Yasunaga, M., ... & Hashimoto, T. (2023). Holistic evaluation of language models. Annals of the New York Academy of Sciences, 1525(1), 140-146.
8. Dou, Z., Cui, D., Yan, J., Wang, W., Chen, B., Wang, H., ... & Zhang, S. (2025). Dsadf: Thinking fast and slow for decision making. arXiv preprint arXiv:2505.08189.
9. Komeili, M., Shuster, K., & Weston, J. (2022). Internet-augmented dialogue generation. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, 7410-7420.
10. Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, S., ... & Clark, P. (2024). Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems, 36.
11. Griffiths, T. L., Lieder, F., & Goodman, N. D. (2015). Rational use of cognitive resources: Levels of analysis between the computational and the algorithmic. Topics in Cognitive Science, 7(2), 217-229.
12. Khattab, O., Santhanam, K., Li, X., Hall, D., & Potts, C. (2023). Demonstrate-search-predict: Composing retrieval and language models for knowledge-intensive NLP. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics, 5561-5577.
13. Shinn, M., Cassano, F., Gopinath, A., Narasimhan, K., & Yao, S. (2024). Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36.
14. Lightman, H., Kosaraju, V., Burda, Y., Edwards, H., Baker, B., Lee, T., ... & Sutskever, I. (2024). Let's verify step by step. International Conference on Learning Representations.
15. Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., Narang, S., ... & Zhou, D. (2023). Self-consistency improves chain of thought reasoning in language models. International Conference on Learning Representations.
16. Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., & Iwasawa, Y. (2022). Large language models are zero-shot reasoners. Advances in Neural Information Processing Systems, 35, 22199-22213.
17. Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., ... & Barnes, P. (2020). Closing the AI accountability gap: Defining an end-to-end framework for internal algorithmic auditing. Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, 33-44.
18. Barocas, S., Hardt, M., & Narayanan, A. (2019). Fairness and machine learning: Limitations and opportunities. MIT Press.
19. Lakkaraju, H., Bastani, O., & Leskovec, J. (2016). "How can I choose a doctor?" Learning to explain and to recommend. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 1275-1284.
20. Fu, L., Chen, X., Gao, K., Huang, X., & Tong, K. (2025, October). Memory-Augmented Knowledge Fusion with Safety-Aware Decoding for Domain-Adaptive Question Answering. In 2025 6th International Conference on Machine Learning and Computer Application (ICMLCA) (pp. 1-6). IEEE.
21. Bubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., ... & Zhang, Y. (2023). Sparks of artificial general intelligence: Early experiments with GPT-4. arXiv preprint arXiv:2303.12712.
22. Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. D. O., Kaplan, J., ... & Zaremba, W. (2021). Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374.
23. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., ... & Zhou, D. (2022). Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35, 24824-24837.
24. Park, J. S., O'Brien, J., Julian, C., & Bernstein, M. S. (2023). Generative agents: Interactive simulacra of human behavior. Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, 1-22.
25. Hendrycks, D., Burns, C., Kadavath, S., Basart, S., Chen, S., Di Troll, C., ... & Steinhardt, J. (2021). Measuring mathematical problem solving with the MATH dataset. arXiv preprint arXiv:2103.03874.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Journal of Data Intelligence and AI Systems
This work is licensed under a Creative Commons Attribution 4.0 International License.
This article is published under the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
