Websockets: not able to retrieve market data

I am new to Tradovate and trying to retrieve market data using Python and the Websockets library via the API. Unfortunately, I keep getting the same error message and can’t find a solution even after a lot of research and reading the docs. I have a ToB subscription to the entire CME Group.

I’m using this request body:

body = {
    "symbol": "MES",
    "chartDescription": {
        "underlyingType": "MinuteBar",
        "elementSize": 1,
        "elementSizeUnit": "UnderlyingUnits",
        "withHistogram": False,
    },
    "timeRange": {
        "asMuchAsElements": 20
    }
}

Request:

ws.send(f"md/getChart\n{request_id}\n\n{json.dumps(body)}")

The printed string:

md/getChart
2

{"symbol": "MES", "chartDescription": {"underlyingType": "MinuteBar", "elementSize": 1, "elementSizeUnit": "UnderlyingUnits", "withHistogram": false}, "timeRange": {"asMuchAsElements": 20}}

Response:

a[{"s":200,"i":2,"d":{"errorText":"Symbol is inaccessible", "errorCode":"UnknownSymbol", "mode":"None"}}]

I also tried to send via an iterable.
Request:

ws.send([
    "md/getChart",
    str(request_id),
    "",
    json.dumps(body)
])

Response:

a[{"s":400,"i":,"d":"Invalid JSON: missing required field \"symbol\", offset: 0x00000001"}]

What am I doing wrong?

The symbol field is incorrect, you need to include a month code as well. For the nearest maturity use the MESM2 symbol (for the contract period ending June 2022). This code will change at the maturity of each contract. We use the CME group’s month code system. MES in particular uses H, M, U, and Z codes.

Thanks for the quick reply, @Alexander!
Unfortunately, I also have exactly the same errors with the MESM2 symbol.

{"symbol": "MESM2", "chartDescription": {"underlyingType": "MinuteBar", "elementSize": 1, "elementSizeUnit": "UnderlyingUnits", "withHistogram": false}, "timeRange": {"asMuchAsElements": 20}}
a[{"s":200,"i":2,"d":{"errorText":"Symbol is inaccessible", "errorCode":"UnknownSymbol", "mode":"None"}}]

Can you tell me what endpoint you use to connect the WebSocket that you send this message through?

wss://md.tradovateapi.com/v1/websocket

And you definitely have an active market data subscription correct? How about just requesting ES data (use ESM2)? Just to know if you can get anything at all.

Yes, I have an active market data subscription.

Which symbol I use does not matter. It is always the same problem.

a[{"s":200,"i":2,"d":{"errorText":"Symbol is inaccessible","errorCode":"UnknownSymbol","mode":"None"}}]

Without divulging your personal information, could you show me how you are structuring your access token request?

I have generated a Python client using the OpenAPI specification. I can successfully authenticate and get an access token. With this I can also query my account data. The websocket connection also seems to be up and running as I can send and receive the heartbeat. Furthermore the opening of the websocket is confirmed with an o and an a[{"s":200, "i":1}].

tradovate_client = TradovateClient(base_url=TRADOVATE_URL)

access_token_response = access_token_request.sync(
    client=tradovate_client,
    json_body=AccessTokenRequest(
        name=TRADOVATE_NAME,
        password=TRADOVATE_API_PW,
        app_id=TRADOVATE_APP_ID,
        app_version=APP_VERSION,
        device_id=TRADOVATE_API_DEVICE_ID,
        cid=TRADOVATE_API_CID,
        sec=TRADOVATE_API_SECRET
    )
)
access_token = access_token_response.access_token
authenticated_client = AuthenticatedClient(
    base_url=TRADOVATE_URL,
    token=access_token
)

account_list_response = account_list.sync(client=authenticated_client)

Hmm that all looks OK. You are using your personal API Key and CID, your username and password, etc? And not like a demo key or something (like the one we have posted up on our API docs Accessing the API section, it works for some things but its highly restricted - it’s a teaser key so you can make test calls without investing in the API).

Next question if that is all OK, is where are you creating the WebSocket and calling the chart request from? Can I see some of that calling code? Are you using multiple sockets (one for market data, one for user data) or are you just using the single market data socket? You must use separate sockets if you want to use both Market Data and Real Time User Data protocols together.

Yes, I use my own API credentials. No demo data published anywhere. I have set and use an API password. I use only one websocket, through which I send the heartbeat and try to fetch the chart data.

Here is the code:

async def heartbeat(websocket):
    while True:
        await asyncio.sleep(2.5)
        await websocket.send('[]')


async def chart(websocket, request_id):
    while True:
        await asyncio.sleep(5)
        body = {
            "symbol": TRADOVATE_SYMBOL,
            "chartDescription": {
                "underlyingType": "MinuteBar",
                "elementSize": 1,
                "elementSizeUnit": "UnderlyingUnits",
                "withHistogram": False,
            },
            "timeRange": {
                "asMuchAsElements": 20
            }
        }
        await websocket.send(f"md/getChart\n{request_id}\n\n{json.dumps(body)}")


async def receive(websocket):
    while True:
        response = await websocket.recv()
        print(f"receive: {response}")


async def open_ws(access_token):
    request_id = 1
    async with websockets.connect(TRADOVATE_WEBSOCKET_URL) as websocket:

        receive_task = asyncio.create_task(receive(websocket), name='receive')

        await websocket.send(f"authorize\n{request_id}\n\n{access_token}")
        request_id += 1

        asyncio.create_task(heartbeat(websocket), name='heartbeat')

        asyncio.create_task(chart(websocket, request_id), name='chart')
        request_id += 1

        await asyncio.Future()  # run forever

await open_ws(access_token)

The only thing I can think of is that TRADOVATE_SYMBOL is being assigned to the wrong thing possibly? Can you log its value before the call?

I have already done that. In my 2nd answer is the output. And here again:

{"symbol": "MESM2", "chartDescription": {"underlyingType": "MinuteBar", "elementSize": 1, "elementSizeUnit": "UnderlyingUnits", "withHistogram": false}, "timeRange": {"asMuchAsElements": 20}}

And the response:

a[{"s":200,"i":2,"d":{"errorText":"Symbol is inaccessible","errorCode":"UnknownSymbol","mode":"None"}}]

You’re going to have trouble with this setup. Python native websocket client will choke when you try to send the heartbeat because it blocks the asyncio loop. You can try to put the heartbeat in another thread or run from futures but those all fail.

Use this websocket client: websocket-client · PyPI

You can use a partial to create the callback functions to pass additional data, then you can create a thread specifically for the heartbeat and have that sleep close to the 2.5 seconds between iterations. That will leave your main thread to get the most work done.

Also with the above websocket-client, you can dump the raw data, with the flag, websocket.enableTrace(True)

It will help give you a true picture of what’s going on.

1 Like

Thanks for the answer @beebee. I am relatively new to Python and websockets and have now tried around a bit, but still seem to have comprehension issues. Would you have some sample code on how to authenticate with Tradovate and then send some request? I can’t get any further at this point.

Thanks in advance!

I am going to dump this here.

import os
import sys
import uuid
import json
import orjson
import requests
import datetime
import websockets

import asyncio

from config import API_LIVE_URL
from config import API_DEMO_URL
from my_secrets import APP_ID, APP_VERSION, ACCOUNT, ACCOUNT_P, CID, CLIENT_SECRET, DEVICE_ID



def get_endpoint(endpoint, _data, DEMO, query=None):
    url                = None
    json_response_data = None
    if(DEMO):
        __api_url = API_DEMO_URL
        print("__TV_API -- get_endpoint -- DEMO URL -- [{0}]".format(__api_url))
    else:
        __api_url = API_LIVE_URL
        print("__TV_API -- get_endpoint -- LIVE URL -- [{0}]".format(__api_url))

    if(query):
        url = "{0}/{1}/{2}".format(__api_url, endpoint, query)
    else:
        url = "{0}/{1}".format(__api_url, endpoint)

    head = { 'content-type':'application/json', 'accept':'application/json', }

    print("__TV_API -- get_endpoint -- header  | {0}".format(head))
    print("__TV_API -- get_endpoint -- payload | {0}".format(_data))
    print("__TV_API -- get_endpoint -- url     | {0}".format(url))


    p = requests.post(url, headers=head, data=_data)
    status = p.status_code
    if(status == 200):
        print("__TV_API -- get_endpoint -- status code 200 -- [{0}]".format(p.status_code))
        print("__TV_API -- get_endpoint -- status code 200 -- [{0}]".format(p.text))
        print("__TV_API -- get_endpoint -- status code 200 -- [{0}]".format(p.content))
        try:
            json_response_data = json.loads(p.text)
        except Exception as e:
            print("get_endpoint -- exception [{0}]\n[{1}]\n".format(str(e), p.text))
        return json_response_data

    else:
        print("get_endpoint -- status code {0} -- [{1}]".format(status, p.text))
        return json_response_data

def __get_access_token(DEMO):
    _filename      = "./accessToken.py"
    diff           = None
    accessToken    = None
    mdAccessToken  = None
    expirationTime = None

    _update_at     = False
    _              = None
    package        = None


    if(os.path.isfile(_filename) and not(os.stat(_filename).st_size==0)):
        time = None
        with open(_filename) as infile:
            package = json.load(infile)
        try:
            time           = package['expirationTime']
        except Exception as e:
            print("__TV_API -- exception [{0}]\n".format(str(e)))
            time = None

        if(time == None):
            print("FILE EXCEPTION\nRE-CREATING FILE")
            _ = __authenticate(DEMO)
            package = _
            with open(_filename, "w") as outfile:
                json.dump(_, outfile)

            return package
        else:
            expirationTime = datetime.datetime.strptime(time, "%Y-%m-%dT%H:%M:%S.%fZ")
            diff = (expirationTime - datetime.datetime.now()).total_seconds()
            if(diff <= 0.0):
                a = "__TV_API -- __get_access_token -- ACCESS TOKEN EXPIRED [{0}], UPDATE --\n[{1}]\n"
                a = a.format(diff, package['accessToken'])
                print(a)
                _ = __authenticate(DEMO)

                package = _
                with open(_filename, "w") as outfile:
                    json.dump(_, outfile)
            else:
                a = "__TV_API -- DO NOT create file -- YET | [{0}] - [{1}] - [{2}]\n"
                a = a.format(expirationTime, diff, _filename)
                print(a)
    else:
        a = "__TV_API -- create file -- | [{0}] - [{1}]\n"
        a = a.format(diff, _filename)
        print(a)
        _ = __authenticate(DEMO)
        package = _
        with open(_filename, "w") as outfile:
            json.dump(_, outfile)
        print(_)

    return package


def __authenticate(DEMO):
    print("__TV_API -- __authenticate -- BEGIN\n")
    #"deviceId":"{}".format(str(uuid.getnode())),
    data = json.dumps({
                       "name":str(ACCOUNT),
                       "password":str(ACCOUNT_P),
                       "appId":str(APP_ID),
                       "appVersion":str(APP_VERSION),
                       "deviceId":str(DEVICE_ID),
                       "cid":int(CID),
                       "sec":str(CLIENT_SECRET)
                      })
 
    _ = get_endpoint("auth/accesstokenrequest", data, DEMO)
    print("__TV_API -- __authenticate -- END  [{0}]\n".format(_))
    return _


if __name__ == "__main__":
    _ = __get_access_token()
    token = _["accessToken"]
    print(token)

There are a few major sections…

If you are mainly going to do get/post requests the function get_endpoint will be helpful and it’s similar to the example javascript code.

Take note of how I create post request url, header and data to create a valid http post request.

The function __get_access_token relies on get_endpoint. get_access_token takes a variable DEMO to see if you’ll be making demo or live requests.

For simplicity sake, I have a file called accessToken in the same directory, this is similar to the storage mechanism in the javascript example. There are rate limits on the get_access_token endpoint and one access token is valid for about 4000 seconds.

The my_secrets file is very important and should not be shared publicly, the contents are below.

APP_ID                 = "xxx"
APP_VERSION            = "xxx"
ACCOUNT                = "xxx"
ACCOUNT_P              = "xxx"
CID                    = "xxx"
CLIENT_SECRET          = "xxx"
DEVICE_ID              = "xxx"

Focus on the get_endpoint function to see how to create a http request with python’s requests module. Try to create simple http get and post requests. If they are still failing post what you have and the errors that you are getting and we can assist you further.

Best,

1 Like

Thanks a lot, I’ll give it a try!

Hey @beebee,

I have already managed the authentication. I am currently only failing with the websockets. Do you have an example for that as well?

Thanks :slight_smile:

Did you get the websockets setup? Can you share what you have so far?

Of course @beebee, I hope you can help me with this.

I have used the suggested library and have now tried around some more until I finally get a response from the websocket server. Unfortunately, it’s exactly the same as before. Here is the code:

import json
from threading import Thread
import websocket
import time

_ws = None
_request_id = 0
_access_token = None

def on_message(ws, message):
    print(message)


def on_error(ws, error):
    print(error)


def on_close(ws, close_status_code, close_msg):
    print("### closed ###")


def send(ws, endpoint, query=None, body=None):
    global _request_id
    _request_id += 1
    print(f"{endpoint}, {_request_id}, {query}, {body}")
    ws.send(f"{endpoint}\n{_request_id}\n{query}\n{body}")


def on_open(ws):
    print('### on_open ###')
    send(ws=ws, endpoint="authorize", body=_access_token)


def chart(*args):
    time.sleep(5)
    # send the message, then wait
    # so thread doesn't exit and socket
    # isn't closed
    print('### chart ###')
    body = {
        "symbol": TRADOVATE_SYMBOL,
        "chartDescription": {
            "underlyingType": "MinuteBar",
            "elementSize": 1,
            "elementSizeUnit": "UnderlyingUnits",
            "withHistogram": False,
        },
        "timeRange": {
            "asMuchAsElements": 20
        }
    }
    send(ws=_ws, endpoint="md/getChart", body=json.dumps(body))


if __name__ == "__main__":
    tradovate_auth()
    websocket.enableTrace(True)
    _ws = websocket.WebSocketApp(TRADOVATE_WEBSOCKET_URL,
                                 on_message=on_message,
                                 on_error=on_error,
                                 on_close=on_close,
                                 on_open=on_open)
    Thread(target=chart).start()
    _ws.run_forever()

_access_token and _ws are global.
tradovate_auth performs authentication using the REST API and sets the global _access_token.

And here are the responses from the server:

o
a[{"s":200,"i":1}]
h
a[{"s":200,"i":2,"d":{"errorText":"Symbol is inaccessible","errorCode":"UnknownSymbol","mode":"None"}}]

There is a lot of debug information. Posting them here would be a bit much.

1 Like