Human-in-the-Loop Interpretability for Detecting Cultural Underrepresentation in AI-Generated Visual Content
Keywords:
human-in-the-loop interpretability, cultural underrepresentation, generative AI, text-to-image models, fairness auditing, sociotechnical infrastructureAbstract
Generative artificial intelligence models that synthesize photorealistic visual content from natural language prompts are increasingly embedded in global media pipelines, yet they frequently exhibit systematic cultural underrepresentation, amplifying existing representational harms and narrowing the visual vocabulary of shared digital culture. Detecting these subtle gaps demands more than automated metrics; it requires interpretive frameworks that connect model outputs to the richly contextual, often tacit criteria by which communities recognize adequate cultural portrayal. This paper presents a system-level analysis of human-in-the-loop interpretability architectures designed to surface cultural underrepresentation in AI-generated imagery. We examine the structural trade-offs involved in coupling large-scale generative models with structured human review, focusing on annotation protocol design, interpretability toolchains, incentive alignment, and feedback integration mechanisms. The discussion situates these architectures within broader sociotechnical governance regimes, considering how scalable human oversight can inform model retraining cycles, auditing standards, and public accountability without collapsing into extractive data labor or superficial diversity theater. By synthesizing insights from fairness auditing, human-computer interaction, and infrastructure studies, we articulate a multidimensional framework that treats interpretability not as a post-hoc transparency instrument but as an ongoing socio-cognitive infrastructure for renegotiating representation across cultural boundaries. Policy implications for documentation practices, regulatory oversight, and sustainable deployment are discussed, emphasizing the need to embed participatory interpretability in the lifecycle of generative visual systems.
References
1. Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021). On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (pp. 610–623). ACM.
2. Buolamwini, J., & Gebru, T. (2018). Gender shades: Intersectional accuracy disparities in commercial gender classification. In Proceedings of the Conference on Fairness, Accountability and Transparency (pp. 77–91). PMLR.
3. Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019). Model cards for model reporting. In Proceedings of the Conference on Fairness, Accountability, and Transparency (pp. 220–229). ACM.
4. Doshi-Velez, F., & Kim, B. (2017). Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608.
5. Holstein, K., Wortman Vaughan, J., Daumé III, H., Dudík, M., & Wallach, H. (2019). Improving fairness in machine learning systems: What do industry practitioners need? In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (pp. 1–16). ACM.
6. C. Shi, S. Li, S. Guo, S. Xie, W. Wu, J. Dou, C. Wu, C. Xiao, C. Wang, Z. Cheng, et al. (2025)Where culture fades: revealing the cultural gap in text-to-image generation.arXiv preprint arXiv:2511.17282.
7. Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., & Sutskever, I. (2021). Learning transferable visual models from natural language supervision. In Proceedings of the 38th International Conference on Machine Learning (pp. 8748–8763). PMLR.
8. Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022). High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 10684–10695). IEEE.
9. Bianchi, F., Kalluri, P., Durmus, E., Ladhak, F., Cheng, M., Nozza, D., Hashimoto, T., Jurafsky, D., Caliskan, A., & Zou, J. (2023). Easily accessible text-to-image generation amplifies demographic stereotypes at large scale. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency (pp. 1493–1504). ACM.
10. Luccioni, A. S., Akiki, C., Mitchell, M., & Jernite, Y. (2023). Stable bias: Analyzing societal representations in diffusion models. arXiv preprint arXiv:2303.11408.
11. Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). “Why should I trust you?” Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 1135–1144). ACM.
12. Lundberg, S. M., & Lee, S.-I. (2017). A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems 30 (pp. 4765–4774).
13. Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017). Grad-CAM: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE International Conference on Computer Vision (pp. 618–626). IEEE.
14. Miller, T. (2019). Explanation in artificial intelligence: Insights from the social sciences. Artificial Intelligence, 267, 1–38.
15. DeVries, T., Misra, I., Wang, C., & van der Maaten, L. (2019). Does object recognition work for everyone? In Proceedings of the CVPR Workshop on Fairness, Accountability, Transparency, and Ethics in Computer Vision. IEEE.
16. Shankar, S., Halpern, Y., Breck, E., Atwood, J., Wilson, J., & Sculley, D. (2017). No classification without representation: Assessing geodiversity issues in open data sets for machine learning. arXiv preprint arXiv:1711.08536.
17. Prabhu, V. U., & Birhane, A. (2020). Large image datasets: A pyrrhic win for computer vision? In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society (pp. 153–160). ACM.
18. Hutchinson, B., Smart, A., Hanna, A., Denton, E., Greer, C., Kjartansson, O., Barnes, P., & Mitchell, M. (2021). Towards accountability for machine learning datasets: Practices from software engineering and documentation. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (pp. 560–575). ACM.
19. Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2021). A survey on bias and fairness in machine learning. ACM Computing Surveys, 54(6), 1–35.
20. Jobin, A., Ienca, M., & Vayena, E. (2019). The global landscape of AI ethics guidelines. Nature Machine Intelligence, 1(9), 389–399.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Journal of Data Intelligence and AI Systems
This work is licensed under a Creative Commons Attribution 4.0 International License.
This article is published under the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
