SpringerLink
Log in

Hybrid feature constraint with clustering for unsupervised person re-identification

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Unsupervised person re-identification (Re-ID) has better scalability and usability in real-world deployments due to the lack of annotations, which is more challenging than supervised methods. State-of-the-art approaches mainly employ clustering algorithms to generate pseudo-labels for transferring the process into a supervised operation. However, the clustering algorithm depends on discriminative pedestrian features. Only using the clustering algorithm produces low-quality labels and hinders the performance of the Re-ID model. In the paper, we propose the hybrid feature constraint network (HFCN) to adequately restrict the pedestrian feature distribution for unsupervised person Re-ID. Specifically, we first define a feature constraint loss to restrict the feature distribution so that different pedestrians can be clearly distinguished at the first step. And then, we design a multi-task operation with the iterative update for clustering algorithm to further implement the feature constraint. This can adequately utilize predicted label information and identify complex samples. Finally, we integrate the feature constraint loss and multi-task operation to optimize the Re-ID model, which could promote the clustering to generate high-quality labels and learn valuable information. Extensive experiments prove that the proposed HFCN is effective and outperforms the state-of-the-art.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (France)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Data availability

The database used is: Market1501, DukeMTMC-reID and MSMT17. They are already public.

References

  1. Wang, X.: Intelligent multi-camera video surveillance: a review. Pattern Recognit. Lett. 34(1), 3–19 (2013)

    Article  Google Scholar 

  2. Ristani, E., Solera, F., Zou, R., Cucchiara, R., Tomasi, C.: Performance measures and a data set for multi-target, multi-camera tracking. In: European Conference on Computer Vision, pp. 17–35 (2016)

  3. Chen, Z., Lv, X., Sun, T., Zhao, C., Chen, W.: Flag: feature learning with additional guidance for person search. Vis. Comput. 37(4), 685–693 (2021)

    Article  Google Scholar 

  4. Si, T., He, F., Zhang, Z., Duan, Y.: Hybrid contrastive learning for unsupervised person re-identification. IEEE Trans. Multimed. (2022). https://doi.org/10.1109/TMM.2022.3174414

    Article  Google Scholar 

  5. Fan, X., Jiang, W., Luo, H., Mao, W.: Modality-transfer generative adversarial network and dual-level unified latent representation for visible thermal person re-identification. Vis. Comput. 38, 279–294 (2022)

    Article  Google Scholar 

  6. Wei, D., Wang, Z., Luo, Y.: Video person re-identification based on RGB triple pyramid model. Vis. Comput. (2022). https://doi.org/10.1007/s00371-021-02344-7

    Article  Google Scholar 

  7. Pervaiz, N., Fraz, M., Shahzad, M.: Per-former: rethinking person re-identification using transformer augmented with self-attention and contextual map**. Vis. Comput. (2022). https://doi.org/10.1007/s00371-022-02577-0

    Article  Google Scholar 

  8. Wei, L., Zhang, S., Gao, W., Tian, Q.: Person transfer GAN to bridge domain gap for person re-identification. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 79–88 (2018)

  9. Zhong, Z., Zheng, L., Zheng, Z., Li, S., Yang, Y.: Camstyle: a novel data augmentation method for person re-identification. IEEE Trans. Image Process. 28(3), 1176–1190 (2018)

    Article  MathSciNet  Google Scholar 

  10. Zhong, Z., Zheng, L., Luo, Z., Li, S., Yang, Y.: Invariance matters: exemplar memory for domain adaptive person re-identification. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 598–607 (2019)

  11. Zhong, Z., Zheng, L., Luo, Z., Li, S., Yang, Y.: Learning to adapt invariance in memory for person re-identification. IEEE Trans. Pattern Anal. Mach. Intell. 43(8), 2723–2738 (2021)

    Google Scholar 

  12. Zhai, Y., Ye, Q., Lu, S., Jia, M., Ji, R., Tian, Y.: Multiple expert brainstorming for domain adaptive person re-identification. In: European Conference on Computer Vision, pp. 594–611 (2020)

  13. Chen, H., Lagadec, B., Bremond, F.: Enhancing diversity in teacher–student networks via asymmetric branches for unsupervised person re-identification. In: IEEE Winter Conference on Applications of Computer Vision, pp. 1–10 (2021)

  14. Wang, D., Zhang, S.: Unsupervised person re-identification via multi-label classification. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 10981–10990 (2020)

  15. Lin, Y., **e, L., Wu, Y., Yan, C., Tian, Q.: Unsupervised person re-identification via softened similarity learning. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3390–3399 (2020)

  16. Zhang, S., He, F.: DRCDN: learning deep residual convolutional dehazing networks. Vis. Comput. 36(9), 1797–1808 (2020)

    Article  Google Scholar 

  17. Pan, Y., He, F., Yu, H.: Learning social representations with deep autoencoder for recommender system. World Wide Web 23(4), 2259–2279 (2020)

    Article  Google Scholar 

  18. Liu, T., Cai, Y., Zheng, J., Thalmann, N.M.: Beacon: a boundary embedded attentional convolution network for point cloud instance segmentation. Vis. Comput. (2021). https://doi.org/10.1007/s00371-021-02112-7

    Article  Google Scholar 

  19. Tulsulkar, G., Mishra, N., Thalmann, N.M., Lim, H.E., Lee, M.P., Cheng, S.K.: Can a humanoid social robot stimulate the interactivity of cognitively impaired elderly? a thorough study based on computer vision methods. Vis. Comput. 37(12), 3019–3038 (2021)

    Article  Google Scholar 

  20. Arora, S., Bhatia, M., Mittal, V.: A robust framework for spoofing detection in faces using deep learning. Vis. Comput. 38(7), 2461–2472 (2022)

    Article  Google Scholar 

  21. Wei, T., He, F., Liu, Y.: YDTR: infrared and visible image fusion via y-shape dynamic transformer. IEEE Trans. Multimed. (2022). https://doi.org/10.1109/TMM.2022.3192661

    Article  Google Scholar 

  22. Liang, Y., He, F., Zeng, X., Luo, J.: An improved loop subdivision to coordinate the smoothness and the number of faces via multi-objective optimization. Integr. Comput. Aided Eng. 29(1), 23–41 (2021)

    Article  Google Scholar 

  23. Li, H., He, F., Chen, Y., Pan, Y.: MLFS-CCDE: multi-objective large-scale feature selection by cooperative coevolutionary differential evolution. Memet. Comput. 13(1), 1–18 (2021)

    Article  Google Scholar 

  24. Zhang, Z., Si, T., Liu, S.: Integration convolutional neural network for person re-identification in camera networks. IEEE Access 6, 36887–36896 (2018)

    Article  Google Scholar 

  25. Luo, H., Gu, Y., Liao, X., Lai, S., Jiang, W.: Bag of tricks and a strong baseline for deep person re-identification. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 4321–4329 (2019)

  26. Liu, S., Huang, W., Zhang, Z.: Learning hybrid relationships for person re-identification. In: Association for the Advance of Artificial Intelligence, pp. 2172–2179 (2021)

  27. **e, J., Ge, Y., Zhang, J., Huang, S., Chen, F., Wang, H.: Low-resolution assisted three-stream network for person re-identification. Vis. Comput. (2021). https://doi.org/10.1007/s00371-021-02127-0

    Article  Google Scholar 

  28. Ding, Y., Duan, Z., Li, S.: Source-free unsupervised multi-source domain adaptation via proxy task for person re-identification. Vis. Comput. 38(6), 1871–1882 (2022)

    Article  Google Scholar 

  29. Si, T., He, F., Wu, H., Duan, Y.: Spatial-driven features based on image dependencies for person re-identification. Pattern Recognit. 124, 108462 (2022)

    Article  Google Scholar 

  30. Chen, D., Xu, D., Li, H., Sebe, N., Wang, X.: Group consistent similarity learning via deep CRF for person re-identification. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 8649–8658 (2018)

  31. Liu, J., Zha, Z., Chen, D., Hong, R., Wang, M.: Adaptive transfer network for cross-domain person re-identification. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 7202–7211 (2019)

  32. Zhang, Z., Wang, Y., Liu, S., **ao, B., Durrani, T.: Cross-domain person re-identification using heterogeneous convolutional network. IEEE Trans. Circuits Syst. Video Technol. 32(3), 1160–1171 (2022)

    Article  Google Scholar 

  33. Jia, Z., Li, Y., Tan, Z., Wang, W., Wang, Z., Yin, G.: Domain-invariant feature extraction and fusion for cross-domain person re-identification. Vis. Comput. (2022). https://doi.org/10.1007/s00371-022-02398-1

    Article  Google Scholar 

  34. Zhong, Z., Zheng, L., Li, S., Yang, Y.: Generalizing a person retrieval model hetero-and homogeneously. In: European Conference on Computer Vision, pp. 172–188 (2018)

  35. Ge, Y., Chen, D., Li, H.: Mutual mean-teaching: pseudo label refinery for unsupervised domain adaptation on person re-identification. In: International Conference on Learning Representations (2020)

  36. Ge, Y., Zhu, F., Chen, D., Zhao, R., Li, H.: Self-paced contrastive learning with hybrid memory for domain adaptive object RE-ID. In: Advances in Neural Information Processing Systems (2020)

  37. Gray, D., Tao, H.: Viewpoint invariant pedestrian recognition with an ensemble of localized features. In: European Conference on Computer Vision, pp. 262–275 (2008)

  38. Liao, S., Hu, Y., Zhu, X., Li, S.Z.: Person re-identification by local maximal occurrence representation and metric learning. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2197–2206 (2015)

  39. Zheng, L., Shen, L., Tian, L., Wang, S., Wang, J., Tian, Q.: Scalable person re-identification: a benchmark. In: IEEE International Conference on Computer Vision, pp. 1116–1124 (2015)

  40. Lin, Y., Dong, X., Zheng, L., Yan, Y., Yang, Y.: A bottom-up clustering approach to unsupervised person re-identification. In: Association for the Advance of Artificial Intelligence, pp. 8738–8745 (2019)

  41. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  42. Wang, W., Wu, Y., Tang, C., Hor, M.: Adaptive density-based spatial clustering of applications with noise (DBSCAN) according to data. In: International Conference on Machine Learning and Cybernetics, pp. 445–451 (2015)

  43. Song, L., Wang, C., Zhang, L., Du, B., Zhang, Q., Huang, C., Wang, X.: Unsupervised domain adaptive re-identification: theory and practice. Pattern Recognit. 102, 107173 (2020)

    Article  Google Scholar 

  44. Zhong, Z., Zheng, L., Kang, G., Li, S., Yang, Y.: Random erasing data augmentation. In: Association for the Advancement of Artificial Intelligence, pp. 13001–13008 (2020)

  45. Zhong, Z., Zheng, L., Zheng, Z., Li, S., Yang, Y.: Camera style adaptation for person re-identification. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 5157–5166 (2018)

  46. Zhai, Y., Lu, S., Ye, Q., Shan, X., Chen, J., Ji, R., Tian, Y.: Ad-cluster: augmented discriminative clustering for domain adaptive person re-identification. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 9021–9030 (2020)

  47. Dai, Z., Chen, M., Gu, X., Zhu, S., Tan, P.: Batch dropblock network for person re-identification and beyond. In: IEEE International Conference on Computer Vision, pp. 3691–3701 (2019)

  48. Si, T., Zhang, Z., Liu, S.: Compact triplet loss for person re-identification in camera sensor networks. Ad Hoc Netw. 95, 101984 (2019)

    Article  Google Scholar 

  49. Ye, M., Shen, J., Lin, G., **ang, T., Shao, L., Hoi, S.C.: Deep learning for person re-identification: a survey and outlook. IEEE Trans. Pattern Anal. Mach. Intell. 44(6), 2872–2893 (2021)

    Article  Google Scholar 

  50. Zhuang, W., Wen, Y., Zhang, S.: Joint optimization in edge-cloud continuum for federated unsupervised person re-identification. In: ACM International Conference on Multimedia, pp. 433–441 (2021)

  51. Li, J., Zhang, S.: Joint visual and temporal consistency for unsupervised domain adaptive person re-identification. In: European Conference on Computer Vision, pp. 483–499 (2020)

  52. Zeng, K., Ning, M., Wang, Y., Guo, Y.: Hierarchical clustering with hard-batch triplet loss for person re-identification. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 13657–13665 (2020)

  53. Prasad, M.V., Balakrishnan, R., Ramadoss, B.: Spatio-temporal association rule based deep annotation-free clustering (STAR-DAC) for unsupervised person re-identification. Pattern Recognit. 122, 108287 (2022)

    Article  Google Scholar 

  54. Pang, B., Zhai, D., Jiang, J., Liu, X.: Fully unsupervised person re-identification via selective contrastive learning. ACM Trans. Multimed. Comput. Commun. Appl. 18(2), 1–15 (2022)

    Article  Google Scholar 

  55. **e, K., Wu, Y., **ao, J., Li, J., **ao, G., Cao, Y.: Unsupervised person re-identification via k-reciprocal encoding and style transfer. Int. J. Mach. Learn. Cybern. 12, 1–18 (2021)

    Article  Google Scholar 

  56. Ji, H., Wang, L., Zhou, S., Tang, W., Zheng, N., Hua, G.: Meta pairwise relationship distillation for unsupervised person re-identification. In: IEEE International Conference on Computer Vision, pp. 3661–3670 (2021)

  57. Yang, F., Zhong, Z., Luo, Z., Cai, Y., Lin, Y., Li, S., Sebe, N.: Joint noise-tolerant learning and meta camera shift adaptation for unsupervised person re-identification. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 4855–4864 (2021)

  58. Li, Q., Peng, X., Qiao, Y., Hao, Q.: Unsupervised person re-identification with multi-label learning guided self-paced clustering. Pattern Recognit. 125, 108521 (2022)

    Article  Google Scholar 

  59. Djebril, M., Amran, B., George, E., Eric, G.: Unsupervised domain adaptation in the dissimilarity space for person re-identification. In: European Conference on Computer Vision, pp. 159–174 (2020)

  60. Ji, Z., Zou, X., Lin, X., Liu, X., Huang, T., Wu, S.: An attention-driven two-stage clustering method for unsupervised person re-identification. In: European Conference on Computer Vision, pp. 20–36 (2020)

  61. Yang, F., Li, K., Zhong, Z., Luo, Z., Sun, X., Cheng, H., Guo, X., Huang, F., Ji, R., Li, S.: Asymmetric co-teaching for unsupervised cross-domain person re-identification. In: Association for the Advance of Artificial Intelligence, pp. 12597–12604 (2020)

  62. **, X., Lan, C., Zeng, W., Chen, Z.: Global distance-distributions separation for unsupervised person re-identification. In: European Conference on Computer Vision, pp. 735–751 (2020)

  63. Wang, G., Lai, J., Liang, W., Wang, G.: Smoothing adversarial domain attack and p-memory reconsolidation for cross-domain person re-identification. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 10568–10577 (2020)

  64. Li, H., Dong, N., Yu, Z., Tao, D., Qi, G.: Triple adversarial learning and multi-view imaginative reasoning for unsupervised domain adaptation person re-identification. IEEE Trans. Circuits Syst. Video Technol. 32(5), 2814–2830 (2021)

    Article  Google Scholar 

  65. Zhang, H., Cao, H., Yang, X., Deng, C., Tao, D.: Self-training with progressive representation enhancement for unsupervised cross-domain person re-identification. IEEE Trans. Image Process. 30, 5287–5298 (2021)

    Article  Google Scholar 

  66. Deng, J., Dong, W., Socher, R., Li, L., Li, K., Li, F.: Imagenet: a large-scale hierarchical image database. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009)

  67. Ainam, J.P., Qin, K., Owusu, J.W., Lu, G.: Unsupervised domain adaptation for person re-identification with iterative soft clustering. Knowl. Based Syst. 212, 106644 (2021)

    Article  Google Scholar 

  68. Sun, J., Li, Y., Chen, H., Peng, Y., Zhu, J.: Unsupervised cross domain person re-identification by multi-loss optimization learning. IEEE Trans. Image Process. 30, 2935–2946 (2021)

    Article  Google Scholar 

  69. Liang, W., Wang, G., Lai, J., **e, X.: Homogeneous-to-heterogeneous: unsupervised learning for RGB-infrared person re-identification. IEEE Trans. Image Process. 30, 6392–6407 (2021)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China under Grant No. 62072348, National Key R &D Program of China under Grant No. 2019YFC1509604, the Science and Technology Major Project of Hubei Province (Next-Generation AI Technologies) under Grant No. 2019AEA170.

Author information

Authors and Affiliations

  1. School of Computer Science, Wuhan University, Bayi Road, Wuhan, 430072, China

    Tongzhen Si & Fazhi He

  2. Institute of Geophysics and Geomatics, China University of Geosciences, Lumo Road, Wuhan, 430074, China

    Penglei Li

Authors
  1. Tongzhen Si
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fazhi He
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Penglei Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fazhi He.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Si, T., He, F. & Li, P. Hybrid feature constraint with clustering for unsupervised person re-identification. Vis Comput 39, 5121–5133 (2023). https://doi.org/10.1007/s00371-022-02649-1

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-022-02649-1

Keywords

Search

Navigation