eSRO released?

[RevoLand]

I hope you know that xsense and evestu's skillset isn't even close to that of jmerlin and pushedx.

No one tells their knowledge are same also we are not discussing about their knowledges in there.

Either way, a project like this is way too complicated for a team on the current sro "level", and while pushedx has currently left sro, I guess jMerlin has to work on it on his own (and that is probably what he desires)

No one can refuse an offer to organized & team based project, i heard about pushedx but never meet him. As i heard about him he will not refuse it. (at least if he is not working on something else)

While I don't agree with every point he makes, I'd love to see a working emulator for silkroad, but I doubt the MIT license would be the way to go (if you see how greedy silkroad has become), I doubt even anybody would care about the license lol.

You're right about that part but i don't think it will be open-source until all team members accept it to be open-source, it must be team-call and will be.

EDIT: RevoLand, please refrain from deleting my posts that actually bring value into the discussion. You are not the one to teach me about respect if you give me warning messages like this:

I still think i did a right call but if you think it wasn't you are free to talk about it with a super moderator and if he / she decides i did a wrong call then i will be more careful.

(if you plan on deleting this post too, I'm seriously gonna lol)

As a moderator we don't have rights to delete posts randomly.

Thanks for your attention to this thread.

[yorick671]

No one tells their knowledge are same also we are not discussing about their knowledges in there.


No one can refuse an offer to organized & team based project, i heard about pushedx but never meet him. As i heard about him he will not refuse it. (at least if he is not working on something else)

lol'd

[jMerliN]

If you have a rough concept of how you are planning on starting this, I'm sure we can arrange you a copy of the packet archive

I already have an idea. It's pretty simple. A single library that packages all of the features needed to build an emulator in Node (https://github.com/just2n/mmo.js) and then a simple application (https://github.com/just2n/silkroad.js) that uses that library but acts primarily as an adapter from the game-specific concepts to use of the mmo library (such as encryption, handshaking, then packet encoding/decoding to fire events into the mmo library).

[megaman963]

Wow, this code is far worse than I gathered from a cursory glance. This belongs on TDWTF for sure. It's as if someone (...). What I planned to do (but didn't and still don't have the time for) was to build a basic framework for programmatic debugging and checksumming that would result in a tool that could allow a reverser to reverse individual packet building/parsing sites, do dataflow analysis to trace where data ends up within in-memory structures, and to mark code paths as completed to get an idea of packet coverage, and maintain this by checksums of basic blocks. When a client update happened, beyond doing basic things like re-running parsing tools on PK2 text resources to update the emulator, one would simply load the tool on the new client and see if any basic blocks involved in packet building/parsing have changed. If so, you get a nice display of those modified blocks in red and a new readout of packet coverage, and all you have to do is do a minimal amount of reversing to update the emulator to the latest version (something like a few hours, worst case, for any capable reverse engineer). This tool would take at least a few months to get working well and right, but would be usable on any game (not just SRO), which is why it'd still be a valuable tool to build, and I may get around to it one day but I'm far too busy these days. In absence of such a tool, the task of gathering packet information for an up to date version isn't a task someone can fully achieve in their spare time while building an emulator that is 100% functional. This is why I asked the community for packet data. (...)

Framework for reverse individual packet is really great idea for reversers in any kind of application that sends packets in Server/Client or P2P methods.

The question is, are you really want to do that for community?

[Jangan]

I already have an idea. It's pretty simple. A single library that packages all of the features needed to build an emulator in Node (https://github.com/just2n/mmo.js) and then a simple application (https://github.com/just2n/silkroad.js) that uses that library but acts primarily as an adapter from the game-specific concepts to use of the mmo library (such as encryption, handshaking, then packet encoding/decoding to fire events into the mmo library).

Tree (data structure) - Wikipedia, the free encyclopedia
:D i never liked nodes... and i also hate gathering packets.. but oh well :P new adventures are fun!

merlin... i think you are hinting that you want to start an emulator... what language + framework are you planning to use? I'd love to build the database/designs + stored procedures/functions + connections and others!



//// Off:topic -- i built a pretty intense combat script "automatives/bots/whatever" for another community while i was bored long time ago i Java - used nodes and other stuff but here to look:
https://github.com/RealCombat/RealCo...ee/master/V0.9

for (Node n : nodeList) {
    if (n.activate()) {
        System.out.println("Activating: " + n.toString());
        n.execute();
        Task.sleep(100);
        break;
    }             
}

[yorick671]

Tree (data structure) - Wikipedia, the free encyclopedia
:D i never liked nodes... and i also hate gathering packets.. but oh well :P new adventures are fun!

merlin... i think you are hinting that you want to start an emulator... what language + framework are you planning to use? I'd love to build the database/designs + stored procedures/functions + connections and others!



//// Off:topic -- i built a pretty intense combat script "automatives/bots/whatever" for another community while i was bored long time ago i Java - used nodes and other stuff but here to look:
https://github.com/RealCombat/RealCo...ee/master/V0.9

for (Node n : nodeList) {
    if (n.activate()) {
        System.out.println("Activating: " + n.toString());
        n.execute();
        Task.sleep(100);
        break;
    }             
}

read the post you just quoted

[Jangan]

read the post you just quoted

You do realize that your reply is useless right?

[jMerliN]

If you have a rough concept of how you are planning on starting this, I'm sure we can arrange you a copy of the packet archive

So where's this packet stuff?

[jMerliN]

Still hwaiting...

[Jangan]

hmm, i might be confused or possible right...


an idea: i believe microsoft wrote a code somewhere about a client/server connection system, and if i remember right, the server side, reads all kind of data , and send it back to the client..


Now i am thinking, why not just setup a read server, where silkroad tried to connect to the server, by connecting it sends specific packets, where the server will read those packets, save them to a file, then close/decline the connection.

I think this way is good to just keep reading packets and figuring out what they are for, trial and error kind of thing..


-------------------------------------------


2 Possible Problems:
-----------------

1) Not sure if the packets are encrypted, not even sure what kind of encryption they are in, not sure if my method will work.
2) If my method works, there is still a problem with sending back the right packets/commands to the client in order to move on forward. Because this method is suppose to just read incoming packets, and no idea how to figure out what are the outgoing packets.



Forgive me, i am not an expert networker, so i do not 100% understand client/server connections and protocols.


/edit/
https://gist.github.com/leandrosilva/656054

- Found the code.

[jMerliN]

Much of what I'm about to say came from a time when I had a near infinite amount of free time, followed sharply by a period of 0 free time, followed by my near 0 free time today, which is why I never finished my emulator (primarily because gathering packets is a huge time investment that almost requires a dedicated person because we didn't build good enough tools).

The packets are encrypted. Much of the initial days of building emulators was similar to that. We started by analyzing the first few packets only to realize that some of them were horribly encrypted (before they added the DH handshake).

Then, as with all reversing, it was a quick PEiD to find that there was blowfish and some other encryption. So a few breakpoints and a few minutes of stepping later, followed by a copy of a reference implementation, and you had a server that could encrypt/decrypt packets (the client never expected encrypted packets, they were flagged by a 0x8000 mask on the packet size field, effectively limiting any possible packet length to 0x7FFF bytes).

So then that let client proxies decrypt packets on the fly and spit them all to disk, as well as the server to echo all packets it received with timestamps to disk. Then packets were replayed initially, and slowly disassembled for information. For instance, you'd get a server list packet and see some text that was obviously the name of a server, but there was other stuff you didn't know about, so you'd replay anything you didn't understand and update what you did. This eventually got us in-game. There are a lot of packets and some packets have a lot of fields, while others have encrypted or very awkward values that make no sense to a person looking at it from the outside but all the sense in the world from the internal workings of the game with how it renders geometry and determines your location in the world (talking about big blocks of landscape, not just X/Y/Z coords). Then there's a lot to parse.

It didn't help that they confused things more with some nonsensically stupid multi-packet packets that can "package" multiple packets together (... because.. that makes sense, it's really a clear sign that the engineers had no understanding of TCP/IP or streaming), or that they actually changed the initial encryption, breaking everything, and that every server locale had its own different opcode values, meaning efforts in csro/tsro/ksro to reverse their packets were confused with isro in that nobody knew which packets were the same.

Around that time, cls, myself, pushedx, and a few other (more independent developers, primarily the guys behind rev6 and cw who was doing 0x33) were starting to get more into reverse engineering the client itself since packet sniffing was too slow and error prone, and with the client updating, we had to have a more robust way of finding packet information, and because they just changed the login encryption so our proxies and client bots were broken on the real servers, and the new clients didn't like our old packets so they'd get an exception and crash. So either we kept using old client versions or we'd have to break into the client and figure things out -- which is what we wanted. This was at 60 cap, back when new things were being added like dw cave and cap 70, and we really wanted this new stuff, we were not satisfied with working on some ancient version of the game with 60 cap and 0 content while the game was moving ahead.

So I, while still learning how to reverse engineer, took a stab at the new encryption stuff. Others worked on better tools. cls and pushedx were working hardcore on packet logging tools and reversing their packet functions (on older versions of the client, but it worked fine on the new client, too). cw was working on hacking the bots and starting softmod (I suspect he made a lot of money doing it, too). It's easier to crack a bot than it is to build a bot, typically, so that's smart.

I reversed a good chunk of the auth code into some ugly C that did work. I didn't complete the reversing, but I provided this to others. pushedx finished the reversing where I had left off, which is why you see me credited in the authing code in emus that is still being used today. The tools we then had prior worked by just dumping what was seen in send/recv hooks. pushedx and cls improved this significantly by reversing where the data went after a recv call and where the data came from in a send call. This resulted in packet serialization/deserialization functions that were then hooked and permitted output logs to include the precise size of each field in each packet, which made it significantly simpler to guess what each field was doing. This is when Eckoro got involved and used those tools to reverse most of the packets -- it was time consuming and tedious, and much of the sremu team was busy in school or with jobs, so we had already lost the time needed to apply those tools and get information. Others besides Eckoro had already done this work, and others since have done this work, but nobody has ever publicly released it.

The problem was, in hindsight, that our tools were one generation too immature to have been imminently useful. pushedx worked a lot more on PK2 things, did some interesting work elsewhere, but nobody really advanced the state of packet reversing tools, which was essential to gathering the information. The only subsequent work done was to locate individual packet serialization/deserialization blocks (such as the code that would completely deserialize a character data packet, or the code that would serialize a create character packet). This meant you could open up IDA and reverse that code, but you had 0 context, which made it very difficult unless you attached a breakpoint and actually stepped through it.

A few years ago (this being many years after I stopped working on SRO related stuff), I decided, with a lot more knowledge, to learn more about reverse engineering, especially theoretical techniques used in malware analysis, code quality testing, and automated debugging. What I learned literally revolutionized how I thought about reversing, and it wasn't anything new, it was just the first time such information was so easily available (a lot of papers on the subjects are available on ACM from way before we ever did SRO things, and tools that did instrumentation and dynamic dataflow and branch analysis were long since available, too).

What we needed to do was make one more iteration. A programmatic debugging framework would've made it very simple to have automated much of the reverse engineering tasks (which were mostly the same), and could provide snapshots of live data for dead code analysis, providing you with a context, which meant you could play the game, capture information, then without loading the game and dealing with disconnecting, etc, just step through the code as it actually happened in a live replay. Another concept was dataflow analysis, in which you literally programmatically step through the code and watch how data moves between registers and memory. You start by marking the data you want to watch in a packet as dirty. We'd start with one field, mark all of the bits of that field (say 8 bits for a 1 byte field) as dirty. Now anything that ever assumes that value becomes marked. You mark all basic blocks that touch that data, and all memory addresses which hold that data. Then later, you can harvest that information by overlaying the hot blocks on top of a dead code listing (say in IDA) to see what code actually touched that data. You'd see immediately in the deserialization code what touched it, which would make it very obvious if it was a flag, a single value, etc. If it was a flag about how to process data in the packet, it would be immediately clear by some form of bitwise operations and conditionals based on them. If it was a flag used by the client to determine something (like are you a GM?), it would show up being stored in client state at some address, and being accessed by code and tested conditionally when you did something (like try to open the GM console). This would immediately tell you WHAT it did, where that state was stored in the client, and which byte in what packet stored the original data. Then one simply has to go through this information, which is a lot more data than you had with 0 context, and flag each byte as 'reversed', along with providing ample notes on what it did.

Then, once every byte of every packet has been analyzed, your logs would be clean, as the tool would never see anything it didn't already know. But as soon as a client update comes out, and a new field is added to a packet, for example, the code paths would change, basic block checksums wouldn't, but the new field would be realized as unknown and fields displaced within the packet or that have had their basic blocks modified would be marked as dirty. The reverser then has essentially a very relevant diff of what changed that they need to actually reverse. So a few minutes later, they finish the reversing, and everything is marked as reversed again, the server/bot code is updated to reflect the changes, and things are done.

That's what we SHOULD have done, but I suspect none of us knew enough at the time to actually do it. While I could do it now, I don't have the time to build something like that. An emulator, though, would be pretty simple to build.