Contrastive Multi-Modal Learning of Weak Signals in High-Dimensional Earth Observation Data
Keywords:
contrastive learning, multi-modal fusion, weak signal detection, Earth observation, high-dimensional data, remote sensing, self-supervised learning, socio-technical infrastructure, fairness, sustainabilityAbstract
The increasing availability of high-dimensional Earth observation data from heterogeneous sensors presents both unprecedented opportunities and significant analytical challenges. Weak signals, defined as subtle, sparse, or low-amplitude patterns embedded within complex multi-spectral, hyperspectral, LiDAR, and synthetic aperture radar modalities, are often masked by strong environmental noise and high inter-channel correlation. This paper proposes a contrastive multi-modal learning framework designed explicitly for the extraction and representation of such weak signals. By leveraging a novel inter-modal contrastive objective that operates across aligned and unaligned data pairs, the framework learns invariant features without requiring dense human annotation. We systematically analyze the architectural trade-offs between modality-specific encoders and shared latent spaces, the governance of large-scale training pipelines, and the computational sustainability of deploying such models on satellite constellations. Fairness implications arising from sensor bias and geographic coverage disparities are examined through the lens of downstream task performance across diverse ecological zones. The paper also discusses policy considerations for open data sharing and model reproducibility in operational remote sensing systems. Through cross-domain comparisons with existing self-supervised approaches, we demonstrate that contrastive learning of weak signals enables more robust detection of early-stage vegetation stress, subtle geological anomalies, and low-contrast urban change. Our findings indicate that deliberate architectural design choices, such as asymmetric projection heads and dynamic weighting of contrastive pairs, significantly affect the quality of learned representations. The proposed framework provides a scalable, governance-aware pathway for integrating multi-modal Earth observation data into decision-support infrastructures.
References
1. Stein, A., & Aryal, J. (2021). Multi-modal data fusion in remote sensing: A review. IEEE Geoscience and Remote Sensing Magazine, 9(3), 24-47.
2. Li, J., Bioucas-Dias, J. M., & Plaza, A. (2019). Spectral-spatial classification of hyperspectral data using deep convolutional neural networks. IEEE Transactions on Geoscience and Remote Sensing, 57(8), 5748-5763.
3. Chen, T., Kornblith, S., Norouzi, M., & Hinton, G. (2020). A simple framework for contrastive learning of visual representations. Proceedings of the 37th International Conference on Machine Learning, 119, 1597-1607.
4. He, K., Fan, H., Wu, Y., Xie, S., & Girshick, R. (2020). Momentum contrast for unsupervised visual representation learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9729-9738.
5. Stojanovic, M., & Yang, C. (2022). Self-supervised pretraining for Sentinel-2 land cover classification using contrastive learning. ISPRS Journal of Photogrammetry and Remote Sensing, 191, 98-112.
6. Zonca, L., & Picchiani, M. (2021). Contrastive learning for synthetic aperture radar image representation. IEEE International Geoscience and Remote Sensing Symposium, 2560-2563.
7. Wang, S., Li, Z., & Zhang, L. (2023). Multi-modal contrastive learning for optical-SAR urban change detection. Remote Sensing of Environment, 284, 113351.
8. He, T., & Gong, M. (2022). Weak signal detection in hyperspectral imagery using contrastive representation learning. IEEE Transactions on Neural Networks and Learning Systems, 33(10), 5486-5499.
9. Yang, J. X., Wang, J., Li, Z., Sui, C., Long, Z., & Zhou, J. (2025). HSLiNets: Evaluating Band Ordering Strategies in Hyperspectral and LiDAR Fusion. IEEE Geoscience and Remote Sensing Letters.
10. Zhexiao Xiong, Xin Ye, Burhan Yaman, Sheng Cheng, Yiren Lu, Jingru Luo, Nathan Jacobs, and Liu Ren. UniDrive-WM: Unified understanding, planning and generation world model for autonomous driving. arXiv preprint arXiv:2601.04453, 2026.
11. Gidaris, S., Singh, P., & Komodakis, N. (2018). Unsupervised representation learning by predicting image rotations. International Conference on Learning Representations.
12. Yun, S., Kim, J., & Yoon, S. (2021). Focal contrastive learning for anomaly detection. Advances in Neural Information Processing Systems, 34, 14585-14596.
13. Grill, J.-B., Strub, F., Altché, F., Tallec, C., Richemond, P. H., Buchatskaya, E., ... & Valko, M. (2020). Bootstrap your own latent: A new approach to self-supervised learning. Advances in Neural Information Processing Systems, 33, 21271-21284.
14. Hu, J., Shen, L., & Sun, G. (2018). Squeeze-and-excitation networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 7132-7141.
15. Zhao, Y., Li, M., Lai, L., Suda, N., Civin, D., & Chandra, V. (2018). Federated learning with non-iid data. arXiv preprint arXiv:1806.00582.
16. Doshi-Velez, F., & Kim, B. (2017). Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608.
17. Reichstein, M., Camps-Valls, G., Stevens, B., Jung, M., Denzler, J., & Carvalhais, N. (2019). Deep learning and process understanding for data-driven Earth system science. Nature, 566(7743), 195-204.
18. Jobin, A., Ienca, M., & Vayena, E. (2019). The global landscape of AI ethics guidelines. Nature Machine Intelligence, 1(9), 389-399.
19. Strubell, E., Ganesh, A., & McCallum, A. (2019). Energy and policy considerations for deep learning in NLP. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 3645-3650.
20. Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2021). A survey on bias and fairness in machine learning. ACM Computing Surveys, 54(6), 1-35.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Journal of Data Intelligence and AI Systems
This work is licensed under a Creative Commons Attribution 4.0 International License.
This article is published under the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
