Portfolio Allocation with Dynamic Risk Preferences via Reinforcement Learning

Abstract

In the realm of investment, the mean–variance model serves as an efficacious method for constructing investment portfolios, as it is underpinned by a robust economic theory and is ubiquitously employed in both academia and practice. Nevertheless, there is currently no satisfactory approach for ascertaining the risk preference parameters within the model for investors. This paper proposes a novel reinforcement learning (RL) framework integrated with the mean–variance model, designed to dynamically adjust investors’ risk preference parameters during the portfolio construction process. Our RL portfolio is not only readily implementable but also exhibits strong economic interpretability. In our empirical analysis employing Taiwan 50 Index market data, our designed RL portfolio outperforms both the buy-and-hold strategy and portfolios with static risk preference parameters. Concurrently, through our meticulously crafted reward function, RL demonstrates heightened accuracy in selecting suitable risk preferences when market return differences are more pronounced, underscoring the effectiveness of RL methods in dynamically adjusting risk preference parameters during periods of elevated market volatility.

Appendix

Before using reinforcement learning to dynamically adjust the risk aversion parameter, we first observe the investment results of selecting the portfolio with higher returns each day from the maximum return portfolio and the minimum variance portfolio. Figure 9 shows the cumulative returns of correctly selecting the portfolio with higher returns under different parameters. The blue, orange, and green lines represent the models with historical state n and historical return volatility days m set at 60, 120, and 240 days, respectively. The cumulative returns are stable and positive, indicating that if reinforcement learning can correctly adjust the risk aversion parameter daily, substantial gains can be achieved. This result also validates the feasibility of our experiment. Moreover, the return of the Taiwan 50 Index is negatively correlated with the risk aversion parameter in the correct investment portfolio. The correlation coefficient of the correct investment portfolio with $n, m=60$ is $-0.26$, with $n, m=120$ is $-0.29$, and with $n, m=240$ is $-0.29$. Therefore, the greater the return of the Taiwan 50 Index, the smaller the risk aversion parameter should be. These results further demonstrate the need to dynamically adjust the daily risk aversion parameter.

Fig. 9

Cumulative return of three correct portfolios. Notes The blue, orange, and green lines indicate models using the historical state n and historical return fluctuation m for 60, 120, and 240 days, respectively. The cumulative return is stable and positive, indicating that if reinforcement learning can correctly adjust the risk preference parameters every day, substantial returns can be obtained. (Color figure online)