Hm... This is hard stuff I suppose, doesn't have a complex knoledge in crypto and in reverse engeneer but maybe I can help..
I think you should see how the client generates those keys that are sent in the heartbeat packet to the server. Try to figure how those keys are generated.
Then, (This I don't know, Its an assumption) When the server sends back the heartbeat to the client, try to find if the client doesn't do something with the output keys you said, like, checking to see if they are valid or not.
Then, knowing how to generate a key, and knowing how to verify the output key, maybe those things are enough to you to find yout what crypto they use and implement any method you like, like, generating the keys, and verifying them...