In-depth reinforcement learning of stock trading, using a CNN with pictures of candlesticks
This article was originally published here
PLoS One. 2022 Feb 18;17(2):e0263181. doi: 10.1371/journal.pone.0263181. eCollection 2022.
Billions of dollars are automatically traded on the stock market every day, including algorithms that use neural networks, but questions remain about how neural networks trade. The black box nature of a neural network encourages entrusting it with valuable trading funds. A newer technique for studying neural networks, feature map visualizations, provide insight into how a neural network generates output. Using a convolutional neural network (CNN) with input candlestick images and feature map visualizations provides a unique opportunity to determine what in the input images causes the neural network to produce some action. In this study, a CNN is used within a Double Deep Q-Network (DDQN) to outperform S&P 500 Index returns and also analyze how the system is trading. The DDQN is trained and tested on the 30 largest stocks in the S&P 500. After the training, the CNN is used to generate feature map visualizations to determine where the neural network places its attention on the candlestick pictures. The results show that the DDQN is able to generate higher returns than the S&P 500 index between January 2, 2020 and June 30, 2020. The results also show that the CNN is able to divert its attention from all the candles of a candlestick picture to the most recent candles in the picture, based on an event such as the 2020 coronavirus stock market crash.