Create a market trading bot using Open AI Gym Anytrading


The main focus of AI is the development of computational functions associated with human intelligence, such as reasoning, learning and problem solving, which can be particularly useful for markets. Trading and investing in the market only requires a series of reasoning and calculations, based on data and solving the problem of predicting the future direction of current stock prices. Fundamental and manual technical analysis is going out of fashion these days. The application of machine learning technology in trading or stock market is used so that the system automatically learns the complexity of the trade and improves its algorithms to assist with the best trading gift. In the last decade, there seemed to be a use of the wallet of traders, so that everyone could earn their profits. But, with the help of AI, one can perfectly analyze the underlying data points presented very quickly and accurately.

Using such data points, we can analyze current market trends and train high speed patterns, which are the two necessary elements generally used for smart trading. Using headlines from news channels and news sources, reviews from social media, and comments on other platforms, AI can analyze the action by performing sentiment analysis on that data. Machine learning usually stores the results and metrics that gave those results and can better analyze the stock market.

Data often helps to find a better solution, especially in probability-based and sentiment-based activities, such as stock trading. But to this day, financial engineers also believe that it is impossible for a machine, left to itself, to beat the stock market. With the rise of technology, incredibly powerful computers can process almost countless data points in a matter of minutes. This means that they are also very capable of detecting historical and replicating patterns for intelligent trading in the market which are often hidden from ordinary human investors. We humans are simply not able to process such data or see these patterns at the same rate as a technologically capable machine. AI can evaluate and analyze thousands of stocks in a matter of moments, and so this technology adds even more speed to trading. Today, every millisecond counts, and with AI as a means of automated trading, it’s a wonder. AI is already learning to continually improve on its own mistakes. It deploys automated trading assistant robots and is constantly working to improve its performance by refining programming and entering huge masses of new data.

What is Open AI Gym Anytrading?

AnyTrading is an Open Source collection of OpenAI Gym environments for reinforcement learning based trading algorithms. The trading algorithms are mainly implemented on the basis of two of the biggest markets present: FOREX and Stock. AnyTrading aims to provide Gym environments to improve and facilitate the process of developing and testing reinforcement learning based algorithms in the field of market trading. This is achieved by implementing it on three Gym environments: TradingEnv, ForexEnv and StocksEnv. AnyTrading can help you learn about stock market trends and perform powerful analysis, providing in-depth insights for data-driven decisions.

Getting started with the code

In this article, we will implement a reinforcement learning based market trading model, in which we will create a trading environment using OpenAI Gym AnyTrading. We will use historical GME price data, then train and evaluate our model using reinforcement learning agents and the gymnastics environment. The following code is partly inspired by a video tutorial on Gym Anytrading, the link of which can be found here.

Library installation

The first essential step would be to install the necessary library. To do this, you can run the following lines of code,

! pip install tensorflow-gpu == 1.15.0 tensorflow == 1.15.0 stable-baselines gym-anytrading gym

Stable-Baselines will provide us with the reinforcement learning algorithm and Gym Anytrading will provide us with our trading environment

Import dependencies

Now let’s install the required dependencies to create a basic framework for our model, and we’ll use the A2C reinforcement learning algorithm to build our market trading model.

# Importing Dependencies
import gym
import gym_anytrading
# Stable baselines - rl stuff
from stable_baselines.common.vec_env import DummyVecEnv
from stable_baselines import A2C
# Processing libraries
import numpy as np
import pandas as pd
from matplotlib import pyplot as plt
Processing of our dataset

Now, with our pipeline setup, let’s load our GME Market data. You can download the dataset using the link here. You can also use other relevant data sets such as Bitcoin data to run this model.

#loading our dataset
df = pd.read_csv('/content/gmedata.csv')
#viewing first 5 columns
#converting Date Column to DateTime Type
df['Date'] = pd.to_datetime(df['Date'])

Go out :

Date      datetime64[ns]
Open             float64
High             float64
Low              float64
Close            float64
Volume            object
dtype: object

#setting the column as index
df.set_index('Date', inplace=True)

We will now transmit the data and create our gym environment for our agent to train later.

See also
How Fujitsu uses artificial intelligence
#passing the data and creating our environment
env = gym.make('stocks-v0', df=df, frame_bound=(5,100), window_size=5)

Setting the window size parameter will specify how many previous price references our trading bot will have so that it can decide to place a trade.

Test our environment

Now with our model setup, let’s test our basic environment and deploy our reinforcement learning agent.

#running the test environment
state = env.reset()
while True: 
    action = env.action_space.sample()
    n_state, reward, done, info = env.step(action)
    if done: 
        print("info", info)

As we can see, our agent RL bought and sold stocks at random. Our profit margin appears to be greater than 1, so we can determine that our bot has made us profit from the trades it has made. But these were random steps, now let’s properly train our model to get better trades.

Shaping our environment

Configure our environment to train our reinforcement learning agent,

#setting up our environment for training 
env_maker = lambda: gym.make('stocks-v0', df=df, frame_bound=(5,100), window_size=5)
env = DummyVecEnv([env_maker])

#Applying the Trading RL Algorithm
model = A2C('MlpLstmPolicy', env, verbose=1) 
#setting the learning timesteps
| explained_variance | 0.0016   |
| fps                | 3        |
| nupdates           | 1        |
| policy_entropy     | 0.693    |
| total_timesteps    | 5        |
| value_loss         | 111      |
| explained_variance | -2.6e-05 |
| fps                | 182      |
| nupdates           | 100      |
| policy_entropy     | 0.693    |
| total_timesteps    | 500      |
| value_loss         | 2.2e+04  |
| explained_variance | 0.0274   |
| fps                | 244      |
| nupdates           | 200      |
| policy_entropy     | 0.693    |
| total_timesteps    | 1000     |
| value_loss         | 0.0663   |
#Setting up the Agent Environment
env = gym.make('stocks-v0', df=df, frame_bound=(90,110), window_size=5)
obs = env.reset()
while True: 
    obs = obs[np.newaxis, ...]
    action, _states = model.predict(obs)
    obs, rewards, done, info = env.step(action)
    if done:
        print("info", info)

#Plotting our Model for Trained Trades

As we can see here, our skilled agent is now doing much better trades and a lot less random trades, giving us profit at the same time with a lot more awareness of when to buy and when to sell the stock.

End Notes

In this article, we have tried to understand how artificial intelligence can be applied to market trading to help leverage the art of buying and selling. We have also created a reinforcement learning model whereby our skilled agent can buy and sell stocks, reserving us profits simultaneously. The following implementation above can be found as a Colab notebook, accessible using the link here.

Good learning!

The references

Join our Discord server. Be part of an engaging online community. Join here.

Subscribe to our newsletter

Receive the latest updates and relevant offers by sharing your email.

Leave A Reply

Your email address will not be published.