Need OpenAI Gym trading environment class debugged in python
Budget: $30 – $250 USD
I am looking for someone who is skilled in python/re enforcement learning via OpenAI gym to help me resolve a few final bugs with the trading environment I have built. I am pretty new to the coding field so I'm sure it's mostly due to my lack of competency.
Currently, the environment is 95% complete, however there are a few issues that I have been unable to solve. The reward and trade profit is not being calculated correctly and due to how complex the class has gotten at this point, and I have been unable to resolve it myself.
Here was my goal:
The environment is meant to be a simulation of the NinjaTrader Platform. It was intended to have the same functionalities and conditions. The environment charges a commission per trade, as well as forces us to close out all positions 2 minutes before the market closes UTC. Trading can then resume when the market re opens about an hour later. PnL is calculated the same was as NQ futures contracts(as this is what this model will trade). 5$ for each .25 price move, 20$ per 1$ price movement or "point". Also, the agent should not be able to chose the buy or sell actions while a position is open. It should only be able to chose "wait" or "close". Similarly, when the agent is flat and has no open position, it should not be able to chose the 'close' action and only be able to chose buy, sell, or wait while flat. This is to prevent order spamming, it should only be able to hold 1 contract at a time. Each trade it takes should have a fixed 30 point stop loss, which should be checked each step to see if it was tagged out by the low/high of a candle. If so, we can assume that that trade was stopped at the max loss of 30 points, equating to a 600$ loss + trade fees (4.46). Lastly, there should be a daily loss limit of 615$, which should be based on the closed PnL + unrealized PnL of the day. If the loss limit is hit, all trades should be closed, and we can skip to the next day and continue from there.
This logic has already been coded into the environment, however there are a few sections that are not working as I intended. It is crucial the the environment is 100% accurate, and functions as a live NT8 platform would.
The current issues are as follows:
- It does not appear that the environment is keeping track of cumulative PnL, or registering when a trade should be stopped out.
- The daily stoploss (max loss per day) is not being updated nor respected. It should make it that if at any time cumulative PnL (both closed and unrealized) reaches -615, all positions should be closed and trading should be halted until the next trading day. I have added print statements and daily_loss seems to always be 0 even when the chart shows lots of red trades.
- I want the agent to be rewarded based on when it closes out a profitable trade, however it's current trade_reward is always "0" even after it closes out profitable trades. I think at the moment, it is only being rewarded for unrealized PnL each step along the way. However I want its main rewards to be based off the closed PnL and the trades that it exits.
- It seems that when we are plotting the trades on the chart, they seem to be correct and reflect the proper PnL. However it feels like the total PnL for the session is incorrect to me, and it makes me hesitant to move forward until I can verify the accuracy of this environment. As I previously stated, this must be an accurate representation of the platform I am aiming to emulate.
I'm sure to someone who knows what they are doing this will be a relatively simple fix, however I just don't have this skill developed enough to get it to the finish line myself. Please see the code attached to this post, we will be working on the CustomFuturesTradingEnv class. Please ask any questions you may have.
Priority will be given to bids that
A. Read this description and look at the code before sending a bid
and
B. Can turnaround the adjusted environment ideally as soon as possible
Currently, the environment is 95% complete, however there are a few issues that I have been unable to solve. The reward and trade profit is not being calculated correctly and due to how complex the class has gotten at this point, and I have been unable to resolve it myself.
Here was my goal:
The environment is meant to be a simulation of the NinjaTrader Platform. It was intended to have the same functionalities and conditions. The environment charges a commission per trade, as well as forces us to close out all positions 2 minutes before the market closes UTC. Trading can then resume when the market re opens about an hour later. PnL is calculated the same was as NQ futures contracts(as this is what this model will trade). 5$ for each .25 price move, 20$ per 1$ price movement or "point". Also, the agent should not be able to chose the buy or sell actions while a position is open. It should only be able to chose "wait" or "close". Similarly, when the agent is flat and has no open position, it should not be able to chose the 'close' action and only be able to chose buy, sell, or wait while flat. This is to prevent order spamming, it should only be able to hold 1 contract at a time. Each trade it takes should have a fixed 30 point stop loss, which should be checked each step to see if it was tagged out by the low/high of a candle. If so, we can assume that that trade was stopped at the max loss of 30 points, equating to a 600$ loss + trade fees (4.46). Lastly, there should be a daily loss limit of 615$, which should be based on the closed PnL + unrealized PnL of the day. If the loss limit is hit, all trades should be closed, and we can skip to the next day and continue from there.
This logic has already been coded into the environment, however there are a few sections that are not working as I intended. It is crucial the the environment is 100% accurate, and functions as a live NT8 platform would.
The current issues are as follows:
- It does not appear that the environment is keeping track of cumulative PnL, or registering when a trade should be stopped out.
- The daily stoploss (max loss per day) is not being updated nor respected. It should make it that if at any time cumulative PnL (both closed and unrealized) reaches -615, all positions should be closed and trading should be halted until the next trading day. I have added print statements and daily_loss seems to always be 0 even when the chart shows lots of red trades.
- I want the agent to be rewarded based on when it closes out a profitable trade, however it's current trade_reward is always "0" even after it closes out profitable trades. I think at the moment, it is only being rewarded for unrealized PnL each step along the way. However I want its main rewards to be based off the closed PnL and the trades that it exits.
- It seems that when we are plotting the trades on the chart, they seem to be correct and reflect the proper PnL. However it feels like the total PnL for the session is incorrect to me, and it makes me hesitant to move forward until I can verify the accuracy of this environment. As I previously stated, this must be an accurate representation of the platform I am aiming to emulate.
I'm sure to someone who knows what they are doing this will be a relatively simple fix, however I just don't have this skill developed enough to get it to the finish line myself. Please see the code attached to this post, we will be working on the CustomFuturesTradingEnv class. Please ask any questions you may have.
Priority will be given to bids that
A. Read this description and look at the code before sending a bid
and
B. Can turnaround the adjusted environment ideally as soon as possible