Human-Centric Visual Search Framework Integrating Person Re-Identification and Open-Vocabulary Object Detection
Keywords:
human-centric visual search, person re-identification, open-vocabulary object detection, multi-modal fusion, socio-technical systems, deployment governanceAbstract
The growing demand for intelligent visual surveillance, assistive technologies, and large-scale forensic search has driven the need for human-centric visual search systems that can simultaneously identify individuals and interpret the objects they interact with. While person re-identification and open-vocabulary object detection have matured as separate domains, their integration into a coherent, deployable framework remains underexplored. This paper presents a system-level investigation of a unified human-centric visual search architecture that jointly leverages clothing-robust person features and open-set object semantics. We argue that such a framework must be designed not only around algorithmic accuracy but also around structural trade-offs in scalability, multi-modal fusion, real-time performance, and ethical governance. The discussion traverses the architectural decomposition of the search pipeline, the challenges of decoupling identity-relevant and object-relevant visual cues under clothing variation, and the infrastructure required for cloud-edge deployment at urban scales. Particular attention is devoted to fairness concerns arising from demographic and contextual biases, the governance of biometric and object metadata, and the policy implications of deploying such systems across jurisdictions with differing privacy norms. By synthesizing perspectives from computer vision, systems engineering, and socio-technical scholarship, we propose design principles that balance technical capability with societal accountability. The paper concludes with forward-looking reflections on foundation models, self-supervised adaptation, and the emergence of internationally harmonized standards for responsible visual search.
References
1. Redmon, J., & Farhadi, A. (2018). YOLOv3: An incremental improvement. arXiv preprint arXiv:1804.02767.
2. Ye, M., Shen, J., Lin, G., Xiang, T., Shao, L., & Hoi, S. C. H. (2021). Deep learning for person re-identification: A survey and outlook. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(6), 2872–2893.
3. Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., ... & Sutskever, I. (2021). Learning transferable visual models from natural language supervision. In International Conference on Machine Learning (pp. 8748–8763). PMLR.
4. Gu, X., Lin, T. Y., Kuo, W., & Cui, Y. (2022). Open-vocabulary object detection via vision and language knowledge distillation. In International Conference on Learning Representations.
5. Liu, S., Zeng, Z., Ren, T., Li, F., Zhang, H., Yang, J., ... & Zhang, L. (2024). Grounding DINO: Marrying DINO with grounded pre-training for open-set object detection. In European Conference on Computer Vision (pp. 38–55). Springer.
6. Li, T., Li, J., & Chen, X. (2023). Person search with natural language description: A survey. IEEE Transactions on Biometrics, Behavior, and Identity Science, 5(1), 15–31.
7. Gu, X., Chang, H., Ma, B., Zhang, H., & Chen, X. (2021). Clothes-changing person re-identification with RGB modality only. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 1060–1069).
8. He, S., Luo, H., Wang, P., Wang, F., Li, H., & Jiang, W. (2021). TransReID: Transformer-based object re-identification. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 15013–15022).
9. Zhu, P., Huang, Z., & Yang, Y. (2023). Towards unified person search with open-vocabulary queries. In Proceedings of the AAAI Conference on Artificial Intelligence, 37(3), 3605–3613.
10. Zhang, W., He, Z., Liu, L., & Jia, Z. (2022). Edge-cloud collaborative real-time video analytics for smart cities: A taxonomy and survey. IEEE Internet of Things Journal, 9(20), 19556–19574.
11. Johnson, J., Douze, M., & Jégou, H. (2019). Billion-scale similarity search with GPUs. IEEE Transactions on Big Data, 7(3), 535–547.
12. Cheng, Y., Wang, D., Zhou, P., & Zhang, T. (2018). Model compression and acceleration for deep neural networks: The principles, progress, and challenges. IEEE Signal Processing Magazine, 35(1), 126–136.
13. Vijayaraghavan, P., & Roy, D. (2022). A fairness analysis of person re-identification systems across demographic subgroups. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society (pp. 730–740).
14. Zhuang, Z., Wei, X., Xie, L., & Tian, Q. (2022). Federated person re-identification: A privacy-preserving benchmark. IEEE Transactions on Information Forensics and Security, 17, 2031–2045.
15. Whittaker, M., Crawford, K., Dobbe, R., Fried, G., Kaziunas, E., Mathur, V., ... & Schwartz, O. (2018). AI now report 2018. AI Now Institute at New York University.
16. Ding, Y., Wang, X., Yuan, H., Qu, M., & Jian, X. (2025). Decoupling feature-driven and multimodal fusion attention for clothing-changing person re-identification. Artificial Intelligence Review, 58(8), 241.
17. European Commission. (2021). Proposal for a regulation of the European Parliament and of the Council laying down harmonised rules on artificial intelligence (Artificial Intelligence Act). COM/2021/206 final.
18. Strubell, E., Ganesh, A., & McCallum, A. (2019). Energy and policy considerations for deep learning in NLP. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (pp. 3645–3650).
19. Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., ... & Liang, P. (2021). On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258.
20. Vestri, M., Cascarano, P., Tamburro, R., & Gualandi, F. (2023). Embedding fairness into large-scale biometric search systems: A governance architecture. Computer Law & Security Review, 48, 105797.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Journal of Data Intelligence and AI Systems
This work is licensed under a Creative Commons Attribution 4.0 International License.
This article is published under the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
