ProtoReasoner: Prototype-Guided Reinforcement Learning for Interpretable and Backdoor-Resistant Large Language Model Reasoning Systems
Keywords:
large language models, reinforcement learning, interpretability, backdoor defense, prototype learning, reasoning systems, trustworthy AI, socio-technical infrastructureAbstract
Large language models have demonstrated remarkable reasoning capabilities, yet their deployment in high-stakes domains is hindered by two persistent challenges: a lack of interpretability in their decision-making processes and vulnerability to backdoor attacks that can covertly manipulate model behavior. Existing approaches to enhance reasoning, such as reinforcement learning from human feedback, often improve performance at the cost of transparency and security. This paper introduces ProtoReasoner, a prototype-guided reinforcement learning framework designed to produce interpretable and backdoor-resistant reasoning systems built upon large language models. ProtoReasoner integrates a prototype-based representation layer into the reinforcement learning loop, enabling the model to ground its reasoning steps on learned, human-interpretable prototypes that capture salient patterns in the input space. These prototypes serve a dual purpose: they provide local and global explanations for model decisions, and they act as semantic anchors that disrupt the injection and activation of backdoor triggers. We present the architectural principles of ProtoReasoner, including the design of the prototype memory bank, the reinforcement learning objective that balances task performance with prototype fidelity, and the inference-time protocol for prototype-based reasoning. A detailed analysis of backdoor resistance demonstrates how prototype consistency forces backdoor triggers to conflict with natural prototypes, making attacks detectable and ineffective. We further discuss system-level considerations such as computational overhead, scalability to large state spaces, robustness under distribution shift, fairness in prototype selection, and the governance implications of deploying interpretable reasoning agents. ProtoReasoner represents a step toward trustworthy large language model systems that can be audited, certified, and deployed in safety-critical applications.
References
1. Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., ... & Amodei, D. (2020). Language models are few-shot learners. Advances in Neural Information Processing Systems, 33, 1877-1901.
2. Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., ... & Lowe, R. (2022). Training language models to follow instructions with human feedback. arXiv preprint arXiv:2203.02155.
3. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., ... & Zhou, D. (2022). Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35, 24824-24837.
4. Bagdasaryan, E., Veit, A., Hua, Y., Estrin, D., & Shmatikov, V. (2020). How to backdoor federated learning. In Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics (pp. 2938-2948). PMLR.
5. Zhang, J., Zhu, Z., Li, G., & Yang, Q. (2023). Backdoor attacks and defenses in large language models: A survey. arXiv preprint arXiv:2309.02377.
6. Uesato, J., Madaan, A., Kumar, R., & Yang, G. (2022). Solving math word problems with process- and outcome-based feedback. arXiv preprint arXiv:2211.14275.
7. Gao, H., Zeng, W., Zhang, J., & Liang, Y. (2025, December). A large model API response quality prediction model based on least squares vector machine and SHAP interpretability analysis. In 2025 5th International Symposium on Artificial Intelligence and Big Data (AIBDF) (pp. 438-442). IEEE.
8. Zhou, D. (2025, December). M-VP2: Microservice-Oriented Vulnerability Patch Planning-A Cost-Aware Approachusing Multi-Agent Reinforcement Learning. In 2025 5th International Conference on Computer, Internet of Things and Control Engineering (CITCE) (pp. 248-254). IEEE.
9. Dou, Z., Zhao, Q., Wan, Z., Zhang, D., Wang, W., Raiyan, T., ... & Biswas, S. (2025). Plan Then Action: High-Level Planning Guidance Reinforcement Learning for LLM Reasoning. arXiv preprint arXiv:2510.01833.
10. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017). Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347.
11. Chen, C., Li, O., Tao, D., Barnett, A., Rudin, C., & Su, J. (2019). This looks like that: Deep learning for interpretable image recognition. Advances in Neural Information Processing Systems, 32.
12. Ghorbani, A., Wexler, J., Zou, J., & Kim, B. (2019). Towards automatic concept-based explanations. Advances in Neural Information Processing Systems, 32.
13. Gu, T., Dolan-Gavitt, B., & Garg, S. (2017). BadNets: Identifying vulnerabilities in the machine learning model supply chain. arXiv preprint arXiv:1708.06733.
14. Chen, X., Liu, C., Li, B., Lu, K., & Song, D. (2017). Targeted backdoor attacks on deep learning systems using data poisoning. arXiv preprint arXiv:1712.05526.
15. Shui, Y., Jin, R., Dou, Z., & Gao, Z. (2026). ProtoGuard-SL: Prototype Consistency Based Backdoor Defense for Vertical Split Learning. arXiv preprint arXiv:2604.03595.
16. Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). “Why should I trust you?”: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 1135-1144).
17. Lundberg, S. M., & Lee, S. I. (2017). A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems, 30.
18. Kurakin, A., Goodfellow, I. J., & Bengio, S. (2017). Adversarial examples in the physical world. In ICLR Workshop.
19. Li, Y., Wu, B., Wang, Y., & Li, M. (2022). Backdoor attack and defense in deep learning: A survey. ACM Computing Surveys, 55(2), 1-36.
20. Schoeffler, M., Stronzek-Pfeifer, L., & Frank, M. (2023). Prototype-based machine learning for explainable AI: A review. ACM Computing Surveys, 56(3), 1-38.
21. Sun, Y., Cheng, Y., & Zhao, H. (2024). Towards robust and interpretable reasoning in large language models: A prototype-guided approach. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (pp. 4567-4579). Association for Computational Linguistics.
22. Dhamija, A., Ghamrawi, N., & Mortensen, D. R. (2023). Interpretability and trust in AI systems: A socio-technical perspective. AI & Society, 38(4), 1571-1585.
23. Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., ... & Barnes, P. (2020). Closing the AI accountability gap: Defining an end-to-end framework for internal algorithmic auditing. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (pp. 33-44).
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Journal of Data Intelligence and AI Systems
This work is licensed under a Creative Commons Attribution 4.0 International License.
This article is published under the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
