SpringerLink
Log in

Portfolio dynamic trading strategies using deep reinforcement learning

  • Application of soft computing
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Using the constituent stocks of the iShares MSCI US ESG Select Index ETF, a matrix of technical indicators, returns, and covariance is incorporated to represent the inherent information characteristics of the stock market. In this study, based on the proposed Deep Reinforcement Learning for Portfolio Management on Environmental, Social, and Governance (DRLPMESG) architecture model, investors who use active portfolio management reap the greatest rewards, as the portfolio with 5 stocks performing the best, with an annualized return of 46.58%, a Sharpe ratio of 1.37, and a cumulative return of 115.18%, indicating that the results have the potential to win the market and generate excess profits. In contrast to the efficient market hypothesis, this new understanding of proven effectiveness in obtaining satisfactory rewards would help improve investment strategies for portfolio management. Furthermore, this study proposed that holding 5 stocks in a portfolio can lead to higher returns, laying the foundation for future research on the number of holdings. Moreover, when compared to previous static strategies, this model offering a dynamic strategy may generate a more stable return in the face of market fluctuations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the first author on reasonable request at myday@gm.ntpu.edu.tw.

Notes

  1. In deep learning, Recurrent Neural Network (RNN), Deep Multilayer Perceptron (DMLP), and Convolutional Neural Network (CNN) models are frequently used.

References

  • Aggarwal S, Aggarwal S (2017) Deep investment in financial markets using deep learning models. Int J Comput Appl 162:40–43

    Google Scholar 

  • Ahmed S, Alshater MM, ElAmmari A, Hammami H (2022) Artificial intelligence and machine learning in finance: a bibliometric review. Res Int Bus Finance 61:101646

    Article  Google Scholar 

  • Al-Aradi A, Jaimungal S (2021) Active and passive portfolio management with latent factors. Quant Finance 21:1437–1459

    Article  MathSciNet  Google Scholar 

  • Alexander C (2008) Market risk analysis, practical financial econometrics. Wiley

    Google Scholar 

  • Arulkumaran K, Deisenroth MP, Brundage M, Bharath AA (2017) Deep reinforcement learning: a brief survey. IEEE Signal Process Mag 34:26–38

    Article  Google Scholar 

  • Betancourt C, Chen W-H (2021) Deep reinforcement learning for portfolio management of markets with a dynamic number of assets. Expert Syst Appl 164:114002

    Article  Google Scholar 

  • Bodnar T, Schmid W (2009) Econometrical analysis of the sample efficient frontier. Eur J Finance 15:317–335

    Article  Google Scholar 

  • Castiglioni I, Rundo L, Codari M et al (2021) AI applications to medical images: From machine learning to deep learning. Phys Med 83:9–24

    Article  Google Scholar 

  • Chandra P (2017) Investment analysis and portfolio management. McGraw-Hill Education, New York

    Google Scholar 

  • Charpentier A, Elie R, Remlinger C (2023) Reinforcement learning in economics and Finance. Comput Econ 62:425–462

  • Chen YF, Huang SH (2021) Sentiment-influenced trading system based on multimodal deep reinforcement learning. Appl Soft Comput 112:107788

    Article  Google Scholar 

  • Chen R, Ren J (2021) Do AI-powered mutual funds perform better? Finance Res Lett 47:102616

    Article  Google Scholar 

  • Chen J, Luo C, Pan L, Jia Y (2021) Trading strategy of structured mutual fund based on deep learning network. Expert Syst Appl 183:115390

    Article  Google Scholar 

  • Choueifaty Y, Coignard Y (2008) Toward Maximum Diversification. J Portf Manag 35:40–51

    Article  Google Scholar 

  • Craja P, Kim A, Lessmann S (2020) Deep learning for detecting financial statement fraud. Decis Support Syst 139:113421

    Article  Google Scholar 

  • Da Silva AS, Lee W, Pornrojnangkool B (2009) The Black-Litterman model for active portfolio management. J Portf Manag 35:61

    Article  Google Scholar 

  • Darapaneni N, Basu A, Savla S, et al (2020) Automated portfolio rebalancing using Q-learning. In: 2020 11th IEEE annual ubiquitous computing, electronics & mobile communication conference (UEMCON). IEEE, pp 0596–0602

  • DeMiguel V, Gil-Bazo J, Nogales FJ, Santos AA (2021) Can machine learning help to select portfolios of mutual funds?, SSRN

  • Deng Y, Bao F, Kong YY et al (2017) Deep direct reinforcement learning for financial signal representation and trading. IEEE Trans Neural Netw Learn Syst 28:653–664

    Article  Google Scholar 

  • Escrig-Olmedo E, Muñoz-Torres MJ, Fernandez-Izquierdo MA (2010) Socially responsible investing: sustainability indices, ESG rating and information provider agencies. Int J Sustain Dev 2:442–461

    Google Scholar 

  • Evans JL, Archer SH (1968) Diversification and the reduction of dispersion: an empirical analysis. J Finance 23:761–767

    Google Scholar 

  • Huang G, Zhou X, Song Q (2020) Deep reinforcement learning for portfolio management. ar**v preprint ar**v:201213773

  • Ieda M, Fu**o N, Sasaki H (2019) Active portfolio management with conditioning information. J Investig 28:51–65

    Google Scholar 

  • Kocmanová A, Dočekalová M (2012) Construction of the economic indicators of performance in relation to environmental, social and corporate governance (ESG) factors. Acta Universitatis Agriculturae Et Silviculturae Mendelianae Brunensis 60:195–206

    Article  Google Scholar 

  • Liang Z, Chen H, Zhu J, et al (2018) Adversarial deep reinforcement learning in portfolio management. ar**v preprint ar**v:180809940

  • Lim QYE, Cao Q, Quek C (2022) Dynamic portfolio rebalancing through reinforcement learning. Neural Comput Appl 34:7125–7139

    Article  Google Scholar 

  • Lintner J (1965) The valuation of risk assets and the selection of risky investments in stock portfolios and capital budgets. Rev Econ Stat 47:13–37

    Article  Google Scholar 

  • Liu FR, Li Y, Li BT et al (2021a) Bitcoin transaction strategy construction based on deep reinforcement learning. Appl Soft Comput 113:107952

    Article  Google Scholar 

  • Lucarelli G, Borrotti M (2020) A deep Q-learning portfolio management framework for the cryptocurrency market. Neural Comput Appl 32:17229–17244

    Article  Google Scholar 

  • Ma YL, Han RZ, Wang WZ (2021b) Portfolio optimization with return prediction using deep learning and machine learning. Expert Syst Appl 165:113973

    Article  Google Scholar 

  • Malkiel BG (2003) The efficient market hypothesis and its critics. J Econ Perspect 17:59–82

    Article  Google Scholar 

  • Markowitz H (1952) Portfolio selection. J Finance 7:77–91

    Google Scholar 

  • Meng TL, Khushi M (2019) Reinforcement learning in financial markets. Data 4:110

    Article  Google Scholar 

  • Ozbayoglu AM, Gudelek MU, Sezer OB (2020) Deep learning for financial applications: a survey. Appl Soft Comput 93:106384

    Article  Google Scholar 

  • Peterson S (2012) Active portfolio management. In: Investment theory and risk management, pp 187–196

  • Pham U, Luu Q, Tran H (2021) Multi-agent reinforcement learning approach for hedging portfolio problem. Soft Comput 25:7877–7885

    Article  Google Scholar 

  • Reilly FK, Akhtar RA (1995) The benchmark error problem with global capital markets. J Portf Manag 22:33

    Article  Google Scholar 

  • Rundo F (2019) Deep LSTM with reinforcement learning layer for financial trend prediction in FX high frequency trading systems. Appl Sci 9:4460

    Article  Google Scholar 

  • Sharpe WF (1964) Capital asset prices: a theory of market equilibrium under conditions of risk. J Finance 19:425–442

    Google Scholar 

  • Shi S, Li JJ, Li GH et al (2022) GPM: a graph convolutional network based reinforcement learning framework for portfolio management. Neurocomputing 498:14–27

    Article  Google Scholar 

  • Stoilov T, Stoilova K, Vladimirov M (2021) Application of modified Black-Litterman model for active portfolio management. Expert Syst Appl 186:115719

    Article  Google Scholar 

  • Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT Press, Cambridge

    MATH  Google Scholar 

  • Taghian M, Asadi A, Safabakhsh R (2022) Learning financial asset-specific trading rules via deep reinforcement learning. Expert Syst Appl 195:116523

    Article  Google Scholar 

  • Théate T, Ernst D (2021) An application of deep reinforcement learning to algorithmic trading. Expert Syst Appl 173:114632

    Article  Google Scholar 

  • Torrente ML, Uberti P (2022) A rescaling technique to improve numerical stability of portfolio optimization problems. Soft Comput. https://doi.org/10.1007/s00500-021-06543-1

  • Vo NNY, He XZ, Liu SW, Xu GD (2019) Deep learning for decision making and the optimization of socially responsible investments and portfolio. Decis Support Syst 124:113097

    Article  Google Scholar 

  • Wang HN, Liu N, Zhang YY et al (2020) Deep reinforcement learning: a survey. Front Inf Technol Electron Eng 21:1726–1744

    Article  Google Scholar 

  • Wang JZ, Zhang HP, Luo H (2022) Research on the construction of stock portfolios based on multiobjective water cycle algorithm and KMV algorithm. Appl Soft Comput 115:108186

    Article  Google Scholar 

  • Weng LG, Sun XD, **a M et al (2020) Portfolio trading system of digital currencies: a deep reinforcement learning with multidimensional attention gating mechanism. Neurocomputing 402:171–182

    Article  Google Scholar 

  • Wu X, Chen H, Wang J et al (2020) Adaptive stock trading strategies with deep reinforcement learning methods. Inf Sci 538:142–158

    Article  MathSciNet  Google Scholar 

  • Wu ME, Syu JH, Lin JCW, Ho JM (2021) Portfolio management system in equity market neutral using reinforcement learning. Appl Intell 51:8119–8131

    Article  Google Scholar 

  • Xu ZY, Zhang J, Wang JY, Xu ZM (2020) Prediction research of financial time series based on deep learning. Soft Comput 24:8295–8312

    Article  Google Scholar 

  • Yu XM, Wu WJ, Liao XC, Han Y (2023) Dynamic stock-decision ensemble strategy based on deep reinforcement learning. Appl Intell 53:2452–2470

    Article  Google Scholar 

  • Yun H, Lee M, Kang YS, Seok J (2020) Portfolio management via two-stage deep learning with a joint cost. Expert Syst Appl 143:113041

    Article  Google Scholar 

  • Zhang Z, Zohren S, Roberts S (2020) Deep reinforcement learning for trading. J Financ Data Sci 2:25–40

    Article  Google Scholar 

  • Zhu X, Wu X (2004) Class noise vs. attribute noise: a quantitative study. Artif Intell Rev 22:177

    Article  MATH  Google Scholar 

Download references

Funding

This research was supported by the Ministry of Science and Technology (MOST), Taiwan [110-2410-H-305-013-MY2] and National Taipei University (NTPU), Taiwan [111-NTPU_ORDA-F-001 and 111-NTPU_ORDA-F-003].

Author information

Authors and Affiliations

  1. Graduate Institute of Information Management, National Taipei University, Taipei, Taiwan

    Min-Yuh Day & Ching-Ying Yang

  2. Department of Management Sciences, Tamkang University, No.151, Yingzhuan Rd., Tamsui Dist., New Taipei City, 25137, Taiwan

    Yensen Ni

Authors
  1. Min-Yuh Day
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ching-Ying Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yensen Ni
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yensen Ni.

Ethics declarations

Conflict of interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Day, MY., Yang, CY. & Ni, Y. Portfolio dynamic trading strategies using deep reinforcement learning. Soft Comput (2023). https://doi.org/10.1007/s00500-023-08973-5

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00500-023-08973-5

Keywords

Search

Navigation