I replied to @SeaPhil post here to bump the issue of pulling lagging OHLC data. For some reason, @BWeis decided to make it a personal conversation between the two of us and then told me to “post more details on community forum” like I already was doing so I’m making a new post I guess.
In the attached image is my data streamer pulling OHLC data via getChart function. The yellow box is the time stamp from my computer time and the blue box is the time stamp for the OHLC data getChart function is pulling from the broker. They are, as @SeaPhil stated, 15 minutes delayed. This is an insanely critical issue that is denying my algorithm’s autonomy for longer than 5~ hours.
Anyone else encounter this issue?
1 Like
What environment are you using to create this? Node, browser, or something else? And also what does your getChart request body look like? It’s possible, if you’re using a browser, that you could be getting throttled if you’re seeing delays.
If you’re using MinuteBars and elementSize: 15 that would only be 22 seconds late however. The blue box looks like the bar timestamp, is that correct?. If it is, the bar timestamp stays the same once you’re in the most recent bar (also there’s the EoH flag to tell you when you’re in the latest bar). When the latest bar moves forward, you’ll receive the next timestamp which will have a timestamp elementSize minutes later.
We are using a python websocket to retrieve data. The data is receiving in “on_message” event. we are using getChart MinuteBar with elementSize=2.
It only happens after some time though? If so, try to restart your getChart subscription when you notice the bar time isn’t within 2 minutes of the current time. On each message, check if the bar-time is more than 2 minutes from the current time. If yes run a procedure to restart the subscription, else keep accepting messages.
Yes, I record the data streamer after setting it up to run overnight and when nearly 2 hours pass it starts delaying. I will try restarting getChart subscription. Will post back by tomorrow afternoon, thanks.
@Alexander So we got the reconnection function implemented and it works. I am still getting bad OHLC data and it is causing my algorithm to poorly perform. I run backtests on the OHLC data from TradeStation and compare them to the papertrade results from running on Tradovate (since I cannot download OHLC data on Tradovate) and the results differ greatly. Looking back on screen recording of the data streamer, closing prices are incorrect.
My algorithm needs the exact closing price of a candle and it’s been pulling incorrect closing values nearly 90% of the time.
Here’s another example of the closing value being incorrect.
What are my solutions here? My computer time is sync’d every 5 minutes and the data streamer is operating correctly using getChart function. This is crucial and I need help pulling correct OHLC data.
Do you have specifically Norton software running on your PC by chance? We encountered a scenario recently where a client running Norton without the ‘BTC mining’ option disabled encountered significant data delay.
I do not have Norton software installed. My CPU is at 30% load when I have multiple programs running too so the speed shouldn’t be affected since I have 16 i9 processors with 64GB ram.
So you asked for options… here’s a couple that’s not ideal but might help.
- Have you tried running on a different platform? Perhaps rent a VM for cheap and watch how things perform. (tests that your local settings aren’t a cause)
- Have you tried using the quote stream instead of the chart stream?
`md/subscribequote\n${messageCount}\n\n{"symbol":${contract.id}}`
instead of
`md/getchart\n${messageCount}\n\n{"symbol":"${contract.id}","chartDescription":...
I’m not excusing bad data… but using an alternate api/socket endpoint may work better.
And of course, I realize it’s not as simple since you’ll have to calculate the close price given the last price of the minute (or whatever increment)… but it should be easy enough to do.
Like @EOD_Trader said, you may want to try running the code a different platform. The quote stream suggestion could work, but charts shouldn’t bog down your connection any more than quotes would, and you’d really be trying to reconstruct a feature that exists (find close for a bar on getChart).
I know that I can run a websocket for several hours and retrieve accurate data right up until my token expires. I even have tried plotting some of my favorite indicators using the getChart data, and I can get 100% accuracy compared to the Trader application. So I think it’s probably not our data stream itself, but something else going wrong.
We coded a re-subscribe function that has been working, thank you for the suggestions none the less. I will keep the solutions in mind if issues continue.