SpringerLink
Log in

FMDADA: Federated multi-discriminative adversarial domain adaptation

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Federated domain adaptation system aims to address the problem of domain shift in a federated learning (FL) framework, where knowledge learned from distributed source domains can be readily transferred to the target domain. However, federated domain adaptation suffers from two challenges: (1) Inefficient assignment of source domain weights. (2) The joint distributions of feature and category across domains are poorly aligned. To solve the above problems, we propose a novel unsupervised federated domain adaptation (UFDA) approach called Federated Multi-Discriminative Adversarial Domain Adaptation (FMDADA). Firstly, we propose a FL aggregation scheme (F-DIS), which assigns weights to distributed source domains with different contribution rates based on a measure of cross-domain discrepancy. Secondly, we facilitate the joint distribution alignment of feature and category by designing multiple tightly coupled joint classifiers, which facilitates the positive transfer of source domain knowledge. Finally, extensive experimental results on three datasets demonstrate the effectiveness of FMDADA for UFDA problem. Compared to the currently advanced comparison approaches, the accuracy of FMDADA is significantly improved, reaching 54.7% and achieving an improvement of 5.9% on the large-scale dataset DomainNet.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price includes VAT (Canada)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Algorithm 1
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Availability of data and materials

All data is publicly available.

Code availability

Contact the authors for access to the code.

References

  1. McMahan B, Moore E, Ramage D, Hampson S, Arcas BA (2017) Communication-efficient learning of deep networks from decentralized data. In: Proceedings of the artificial intelligence and statistics, pp 1273–1282

  2. Yurochkin M, Agarwal M, Ghosh S, Greenewald K, Hoang N, Khazaeni Y (2019) Bayesian nonparametric federated learning of neural networks. In: Proceedings of the international conference on machine learning, pp 7252–7261

  3. Park J, Han D-J, Choi M, Moon J (2021) Sageflow: robust federated learning against both stragglers and adversaries. Proc Adv Neural Inf Process Syst 34:840–851

    Google Scholar 

  4. Quinonero-Candela J, Sugiyama M, Schwaighofer A, Lawrence ND (2008) Dataset shift in machine learning, pp 1–299

  5. Liang X, Lin Y, Fu H, Zhu L, Li X (2022) Rscfed: random sampling consensus federated semi-supervised learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 10154–10163

  6. Yu T, Bagdasaryan E, Shmatikov V (2020) Salvaging federated learning by local adaptation. Preprint at ar**v:2002.04758

  7. Liu Y, Kang Y, **ng C, Chen T, Yang Q (2020) A secure federated transfer learning framework. IEEE Intell Syst 35:70–82

    Article  Google Scholar 

  8. Li T, Sahu AK, Zaheer M, Sanjabi M, Talwalkar A, Smith V (2020) Federated optimization in heterogeneous networks. Proc Mach Learn Syst 2:429–450

    Google Scholar 

  9. Huang Y, Chu L, Zhou Z, Wang L, Liu J, Pei J, Zhang Y (2021) Personalized cross-silo federated learning on non-iid data. In: Proceedings of the AAAI conference on artificial intelligence, pp 7865–7873

  10. Dinh TC, Tran N, Nguyen J (2020) Personalized federated learning with moreau envelopes. Proc Adv Neural Inf Process Syst 21394–21405

  11. Gao L, Fu H, Li L, Chen Y, Xu M, Xu C-Z (2022) Feddc: Federated learning with non-iid data via local drift decoupling and correction. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 10112–10121

  12. HassanPour Zonoozi M, Seydi V (2023) A survey on adversarial domain adaptation. Neural Process Lett 55:2429–2469

    Article  Google Scholar 

  13. Westfechtel T, Yeh H-W, Zhang D, Harada T (2024) Gradual source domain expansion for unsupervised domain adaptation. In: Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp 1946–1955

  14. Lee J, Jung D, Yim J, Yoon S (2022) Confidence score for source-free unsupervised domain adaptation. In: International conference on machine learning, pp 12365–12377

  15. Peng X, Huang Z, Zhu Y, Saenko K (2020) Federated adversarial domain adaptation. Proceedings of the international conference on learning representations, 1–19

  16. Ben-David S, Blitzer J, Crammer K, Kulesza A, Pereira F, Vaughan JW (2010) A theory of learning from different domains. Mach Learn 79:151–175

    Article  MathSciNet  Google Scholar 

  17. Ben-David S, Blitzer J, Crammer K, Pereira F (2006) Analysis of representations for domain adaptation. Proceedings of the Advances in neural information processing systems, 137–144

  18. Xu M, Zhang J, Ni B, Li T, Wang C, Tian Q, Zhang W (2020) Adversarial domain adaptation with domain mixup. In: Proceedings of the AAAI conference on artificial intelligence, pp 6502–6509

  19. Pei Z, Cao Z, Long M, Wang J (2018) Multi-adversarial domain adaptation. In: Proceedings of the Thirty-second AAAI conference on artificial intelligence, pp 3934–3941

  20. Tang H, Jia K (2020) Discriminative adversarial domain adaptation. In: Proceedings of the AAAI conference on artificial intelligence, pp 5940–5947

  21. Yeganeh Y, Farshad A, Navab N, Albarqouni S (2020) Inverse distance aggregation for federated learning with non-iid data. In: Domain adaptation and representation transfer, and distributed and collaborative learning, pp 150–159

  22. Li X, Jiang M, Zhang X, Kamp M, Dou Q (2021) Fedbn: Federated learning on non-iid features via local batch normalization, 1–12

  23. Ge P, Ren C-X, Xu X-L, Yan H (2023) Unsupervised domain adaptation via deep conditional adaptation network. Pattern Recognit 134:4488–4503

    Article  Google Scholar 

  24. Wei G, Lan C, Zeng W, Chen Z (2021) Metaalign: Coordinating domain alignment and classification for unsupervised domain adaptation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 16643–16653

  25. Du Z, Li J, Su H, Zhu L, Lu K (2021) Cross-domain gradient discrepancy minimization for unsupervised domain adaptation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3937–3946

  26. Mei Z, Ye P, Ye H, Li B, Guo J, Chen T, Ouyang W (2023) Automatic loss function search for adversarial unsupervised domain adaptation. IEEE Trans Circ Syst Video Tech 33:5868–5881

    Article  Google Scholar 

  27. Akkaya IB, Altinel F, Halici U (2021) Self-training guided adversarial domain adaptation for thermal imagery. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 4322–4331

  28. Gao Z, Zhang S, Huang K, Wang Q, Zhong C (2021) Gradient distribution alignment certificates better adversarial domain adaptation. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 8937–8946

  29. Long M, Cao Z, Wang J, Jordan MI (2018) Conditional adversarial domain adaptation. Proceedings of the Advances in neural information processing systems, 1647–1657

  30. Cheng Z, Wang S, Yang D, Qi J, **ao M, Yan C (2024) Deep joint semantic adaptation network for multi-source unsupervised domain adaptation. Pattern Recognit 151:1–11

    Article  Google Scholar 

  31. Park GY, Lee SW (2021) Information-theoretic regularization for multi-source domain adaptation. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 9214–9223

  32. Zhao S, Wang G, Zhang S, Gu Y, Li Y, Song Z, Xu P, Hu R, Chai H, Keutzer K (2020) Multi-source distilling domain adaptation. In: Proceedings of the AAAI conference on artificial intelligence, pp 12975–12983

  33. Yan H, Ding Y, Li P, Wang Q, Xu Y, Zuo W (2017) Mind the class weight bias: Weighted maximum mean discrepancy for unsupervised domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2272–2281

  34. Peng X, Bai Q, **a X, Huang Z, Saenko K, Wang B (2019) Moment matching for multi-source domain adaptation. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 1406–1415

  35. Zhao H, Zhang S, Wu G, Gordon GJ et al (2018) Multiple source domain adaptation with adversarial learning. Proceedings of the 2018 international conference on learning representations, 1–24

  36. Feng H, You Z, Chen M, Zhang T, Zhu M, Wu F, Wu C, Chen W (2021) Kd3a: Unsupervised multi-source decentralized domain adaptation via knowledge distillation. In: Proceedings of the international conference on machine learning, pp 3274–3283

  37. Gong B, Shi Y, Sha F, Grauman K (2012) Geodesic flow kernel for unsupervised domain adaptation. In: Proceedings of the 2012 IEEE conference on computer vision and pattern recognition, pp 2066–2073

  38. Ganin Y, Lempitsky V (2015) Unsupervised domain adaptation by backpropagation. In: Proceedings of the international conference on machine learning, pp 1180–1189

  39. Yang L, Balaji Y, Lim S-N, Shrivastava A (2020) Curriculum manager for source selection in multi-source domain adaptation. In: Proceedings of the european conference on computer vision, pp 608–624

  40. Liu Y-H, Ren C-X (2022) A two-way alignment approach for unsupervised multi-source domain adaptation. Pattern Recognit 124:1–13

  41. Liang J, Hu D, Feng J (2020) Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation. In: International conference on machine learning, pp 6028–6039

  42. Wang B, Li G, Wu C, Zhang W, Zhou J, Wei Y (2022) A framework for self-supervised federated domain adaptation. EURASIP J Wirel Commun Netw 2022:1–17

    Article  Google Scholar 

  43. Wei Y, Han Y (2023) Exploring instance relation for decentralized multi-source domain adaptation. In: Proceedings of the IEEE International conference on acoustics, speech and signal processing, pp 5031–5040

  44. Zhang Y, Tang H, Jia K, Tan M (2019) Domain-symmetric networks for adversarial domain adaptation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 5031–5040

  45. Cui S, Wang S, Zhuo J, Su C, Huang Q, Tian Q (2020) Gradually vanishing bridge for adversarial domain adaptation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 12455–12464

  46. Li S, **e M, Lv F, Liu CH, Liang J, Qin C, Li W (2021) Semantic concentration for domain adaptation. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 9102–9111

  47. Wang R, Wu Z, Weng Z, Chen J, Qi G-J, Jiang Y-G (2022) Cross-domain contrastive learning for unsupervised domain adaptation. IEEE Trans Multimedia, 1–10. https://doi.org/10.1109/TMM.2022.3146744

  48. Saenko K, Kulis B, Fritz M, Darrell T (2010) Adapting visual category models to new domains. In: Proceedings of the european conference on computer vision, pp 213–226

Download references

Acknowledgements

This research is supported by the National Natural Science Foundation of China (NSFC) under grant number 62172377, the Taishan Scholars Program of Shandong province under grant numbers tsqn202312102, and the Startup Research Foundation for Distinguished Scholars under grant number 202112016.

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Ocean University of China, 1299 Sansha Street, Qingdao, 266100, Shandong, China

    Hao Chi, Hui **a, Shuo Xu & Yusheng He

  2. School of Big Data & Software Engineering, Chongqing University, 174 Shazheng Street, Chongqing, 400044, China

    Chunqiang Hu

Authors
  1. Hao Chi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hui **a
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shuo Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yusheng He
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Chunqiang Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to writing and research.

Corresponding author

Correspondence to Hui **a.

Ethics declarations

Conflict of interest

There is no conflict of interest.

Ethics approval

Not Applicable.

Consent to participate

Not Applicable.

Consent for publication

Not Applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chi, H., **a, H., Xu, S. et al. FMDADA: Federated multi-discriminative adversarial domain adaptation. Appl Intell (2024). https://doi.org/10.1007/s10489-024-05592-x

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10489-024-05592-x

Keywords

Search

Navigation