Explainable Prompt Selection Mechanisms for Interpretable Large Language Model Adaptation
Keywords:
explainable artificial intelligence, prompt engineering, large language models, model adaptation, selective prompting, interpretability, system governanceAbstract
The rapid proliferation of large language models (LLMs) has driven a paradigm shift in downstream task adaptation through prompt engineering, where carefully crafted textual or learned continuous instructions steer model behavior without retraining the entire parameter set. As prompt-based adaptation matures, systems increasingly incorporate dynamic selection mechanisms that automatically choose, compose, or insert prompts from a pre-defined library or a learned continuous space. While such mechanisms improve accuracy and efficiency, they introduce new layers of opacity that hinder interpretability, accountability, and trust. This paper presents a system-level examination of explainable prompt selection mechanisms designed for interpretable LLM adaptation. We analyze the architectural foundations that underpin prompt selection, spanning soft prompt tuning, prefix-based adaptation, mixture-of-experts routing, and selective insertion strategies. We then map the landscape of interpretability techniques—ranging from post-hoc attribution to concept-based explanations—onto these selection processes, identifying structural impediments to transparency and proposing design principles for intrinsically interpretable selection modules. Through a governance lens, we evaluate trade-offs among computational efficiency, fairness, robustness, and sustainability, highlighting how opaque prompt selection can amplify biases or create adversarial vulnerabilities. Drawing on cross-domain use cases in clinical decision support and content moderation, we illustrate the practical stakes of explainable prompt selection and discuss the alignment of such systems with emerging regulatory frameworks. The paper concludes by charting open challenges, including the need for standardized evaluation protocols for selective interpretation, the tension between sparsity and explainability, and the role of human-in-the-loop oversight. By treating prompt selection not as a purely algorithmic optimization but as a socio-technical control interface, we provide a roadmap toward responsible adaptation of LLMs at scale.
References
1. Houlsby, N., Giurgiu, A., Jastrzebski, S., Morrone, B., de Laroussilhe, Q., Gesmundo, A., Attariyan, M., & Gelly, S. (2019). Parameter-efficient transfer learning for NLP. In International Conference on Machine Learning (pp. 2790–2799). PMLR.
2. Lester, B., Al-Rfou, R., & Constant, N. (2021). The power of scale for parameter-efficient prompt tuning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (pp. 3045–3059). Association for Computational Linguistics.
3. Li, X. L., & Liang, P. (2021). Prefix-tuning: Optimizing continuous prompts for generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) (pp. 4582–4597). Association for Computational Linguistics.
4. Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., & Chen, W. (2022). LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations. OpenReview.
5. Shin, T., Razeghi, Y., Logan IV, R. L., Wallace, E., & Singh, S. (2020). AutoPrompt: Eliciting knowledge from language models with automatically generated prompts. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 4222–4235). Association for Computational Linguistics.
6. Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). “Why should I trust you?”: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 1135–1144). ACM.
7. Lundberg, S. M., & Lee, S.-I. (2017). A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems (pp. 4765–4774). Curran Associates, Inc.
8. Jain, S., & Wallace, B. C. (2019). Attention is not explanation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) (pp. 3543–3556). Association for Computational Linguistics.
9. Kim, B., Wattenberg, M., Gilmer, J., Cai, C., Wexler, J., Viegas, F., & Sayres, R. (2018). Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (TCAV). In International Conference on Machine Learning (pp. 2668–2677). PMLR.
10. Liu, J., Shen, D., Zhang, Y., Dolan, B., Carin, L., & Chen, W. (2022). What makes good in-context examples for GPT-3? In Proceedings of Deep Learning Inside Out (DeeLIO 2022): The 3rd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures (pp. 100–114). Association for Computational Linguistics.
11. Zhu, W., & Tan, M. (2023, December). SPT: Learning to selectively insert prompts for better prompt tuning. In Proceedings of the 2023 conference on empirical methods in natural language processing (pp. 11862-11878).
12. Abid, A., Farooqi, M., & Zou, J. (2021). Persistent anti-Muslim bias in large language models. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society (pp. 298–306). ACM.
13. Wallace, E., Feng, S., Kandpal, N., Gardner, M., & Singh, S. (2019). Universal adversarial triggers for attacking and analyzing NLP models. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) (pp. 2153–2162). Association for Computational Linguistics.
14. Patterson, D., Gonzalez, J., Le, Q., Liang, C., Munguia, L., Rothchild, D., Texier, D., & Dean, J. (2021). Carbon emissions and large neural network training. arXiv preprint arXiv:2104.10350.
15. Shazeer, N., Mirhoseini, A., Maziarz, K., Davis, A., Le, Q., Hinton, G., & Dean, J. (2017). Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. In International Conference on Learning Representations. OpenReview.
16. Jobin, A., Ienca, M., & Vayena, E. (2019). The global landscape of AI ethics guidelines. Nature Machine Intelligence, 1(9), 389–399.
17. Veale, M., & Borgesius, F. Z. (2021). Demystifying the draft EU artificial intelligence act. Computer Law Review International, 22(4), 97–112.
18. Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., & Pedreschi, D. (2018). A survey of methods for explaining black box models. ACM Computing Surveys, 51(5), Article 93 (1–42).
19. Shi, W., Min, S., Yasunaga, M., Seo, M., James, R., Lewis, M., Zettlemoyer, L., & Riedel, S. (2023). RePrompt: Planning by automatic prompt engineering for large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (pp. 562–580). Association for Computational Linguistics.
20. Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., Payne, P., Shah, N., Liu, Y., Gottweis, J., Vijayakumar, A., Seneviratne, M., Gamble, P., Ghosn, D., Oakden-Rayner, L., ... & Natarajan, V. (2023). Large language models encode clinical knowledge. Nature, 620, 172–180.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Journal of Data Intelligence and AI Systems
This work is licensed under a Creative Commons Attribution 4.0 International License.
This article is published under the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
