Welcome!

Join our community of MMO enthusiasts and game developers! By registering, you'll gain access to discussions on the latest developments in MMO server files and collaborate with like-minded individuals. Join us today and unlock the potential of MMO server development!

Join Today!

Why not using protobuf

Initiate Mage
Joined
Jun 20, 2014
Messages
3
Reaction score
0
Ive been looking in some server sources and I saw some nasty packet parsing... the mplew thing, you are readding short then int and then string but you write it and you need to recompile it over and over again.
Why dont you all just take a week and write change the sources to work properly with protobuf and then you also dont need to recompile it? Im 100% sure it will be much faster then any other sources.
 
Custom Title Activated
Loyal Member
Joined
Jan 18, 2010
Messages
3,109
Reaction score
1,139
I know I hate on Odin a lot too, but.. you can't just say that this Odin has "terrible parsing" and then generalize stuff as a "thing"..

For starters: "mplew" is just the common variable name short for MaplePacketLittleEndianWriter. You could have mentioned the actual class(es) involved. Now, a Writer is NOT a Parser. If you're talking about parsing, then you would be referring to "slea", a SeekableLittleEndianAccessor. This parses the incoming data, it does not write it.

Second. We are not "readding" anything, we are actually in reality just appending bytes. The writer is writing and adding bytes into a buffer, and this class is just a wrapper in general. The bytes are also not random at all, each are the physical packets that require bytes (short, string, and so on) in a specific order. These are then returned as a MaplePacket in older sources, which is actually kinda silly I will agree, that's why it eventually turned into just byte[] instead of it calling it when it's sent. Anyways, in sources nowadays it just returns a byte-array of the buffer that we appended all these bytes of data into.

After the data has been constructed, we send the byte-array of data (remember that these bytes are in a Little Endian order, as well) through a client's session and into an Encoder which will encrypt the data (the packet) and send this packet to the client (networking, yay!).

Third. Doing any of this will not make anything "100% faster then any other sources" lol. Now if what you want is to improve the networking and writer's, you could upgrade MINA and your manually handled bytes to Netty and their ByteBuf. It's not a necessity at all, just preference really. I love Netty and would recommend to use it, and it seems more newer devs now are switching to it. Even when this technically makes it "faster", it is nothing really noticeable or worth anything in this type of.. field or w/e. It's a big improvement if you have a massive player base and tons of active packets being sent back and forth and such, but this likely won't be the case.

Lastly. I'm really not a fan of this protobuf design in Java, and did you even read what it does? Below is the description of a Protocol Buffer. I don't see any reason why we would personally use this for all of our packet structures.

"With protocol buffers, you write a .proto description of the data structure you wish to store. From that, the protocol buffer compiler creates a class that implements automatic encoding and parsing of the protocol buffer data with an efficient binary format. The generated class provides getters and setters for the fields that make up a protocol buffer and takes care of the details of reading and writing the protocol buffer as a unit. Importantly, the protocol buffer format supports the idea of extending the format over time in such a way that the code can still read data encoded with the old format."

Also.. if we are in the wrong here, why not show us some code and benchmarks of your version with Google's Protocol Buffer implemented proving your theory that it will basically make all of our sources "100% faster"?
 
Upvote 0
Skilled Illusionist
Joined
Jul 28, 2009
Messages
339
Reaction score
200
I know I hate on Odin a lot too, but.. you can't just say that this Odin has "terrible parsing" and then generalize stuff as a "thing"..

For starters: "mplew" is just the common variable name short for MaplePacketLittleEndianWriter. You could have mentioned the actual class(es) involved. Now, a Writer is NOT a Parser. If you're talking about parsing, then you would be referring to "slea", a SeekableLittleEndianAccessor. This parses the incoming data, it does not write it.

Second. We are not "readding" anything, we are actually in reality just appending bytes. The writer is writing and adding bytes into a buffer, and this class is just a wrapper in general. The bytes are also not random at all, each are the physical packets that require bytes (short, string, and so on) in a specific order. These are then returned as a MaplePacket in older sources, which is actually kinda silly I will agree, that's why it eventually turned into just byte[] instead of it calling it when it's sent. Anyways, in sources nowadays it just returns a byte-array of the buffer that we appended all these bytes of data into.

After the data has been constructed, we send the byte-array of data (remember that these bytes are in a Little Endian order, as well) through a client's session and into an Encoder which will encrypt the data (the packet) and send this packet to the client (networking, yay!).

Third. Doing any of this will not make anything "100% faster then any other sources" lol. Now if what you want is to improve the networking and writer's, you could upgrade MINA and your manually handled bytes to Netty and their ByteBuf. It's not a necessity at all, just preference really. I love Netty and would recommend to use it, and it seems more newer devs now are switching to it. Even when this technically makes it "faster", it is nothing really noticeable or worth anything in this type of.. field or w/e. It's a big improvement if you have a massive player base and tons of active packets being sent back and forth and such, but this likely won't be the case.

Lastly. I'm really not a fan of this protobuf design in Java, and did you even read what it does? Below is the description of a Protocol Buffer. I don't see any reason why we would personally use this for all of our packet structures.

"With protocol buffers, you write a .proto description of the data structure you wish to store. From that, the protocol buffer compiler creates a class that implements automatic encoding and parsing of the protocol buffer data with an efficient binary format. The generated class provides getters and setters for the fields that make up a protocol buffer and takes care of the details of reading and writing the protocol buffer as a unit. Importantly, the protocol buffer format supports the idea of extending the format over time in such a way that the code can still read data encoded with the old format."

Also.. if we are in the wrong here, why not show us some code and benchmarks of your version with Google's Protocol Buffer implemented proving your theory that it will basically make all of our sources "100% faster"?
Mina uses NIO already so I don't think the packet processing stuff would improve much
 
Upvote 0
Everything is possible~
Loyal Member
Joined
Jan 9, 2008
Messages
818
Reaction score
847
Mina uses NIO already so I don't think the packet processing stuff would improve much

Wow, who has awakened you? I haven't seen you in ages.

OT: If you really want to improve networking logic: don't use Java in the first place
 
Upvote 0
Skilled Illusionist
Joined
Jul 28, 2009
Messages
339
Reaction score
200
Wow, who has awakened you? I haven't seen you in ages.

OT: If you really want to improve networking logic: don't use Java in the first place

New phone who dis
/ I don't think the difference would be that much worse for ms server scale
 
Upvote 0
Joined
Jun 5, 2010
Messages
567
Reaction score
598
Ive been looking in some server sources and I saw some nasty packet parsing... the mplew thing, you are readding short then int and then string but you write it and you need to recompile it over and over again.
Why dont you all just take a week and write change the sources to work properly with protobuf and then you also dont need to recompile it? Im 100% sure it will be much faster then any other sources.

I use this at work and I'm pretty sure it won't work in this case. Here we are not serializing objects and sending them anywhere and rematerializing them. We are just building up a packet by appending info onto an existing byte array. In fact for every packet there would need to be a message with this, and you'll have to end up doing the same thing in the end as we are doing here. Also you do need to recompile with protobufs, pretty sure since it generates code (like setters and getters). Also there's downsides to using it too, if you're going to change a required field to optional for example. It'll ruin your previously parsed binaries. Of course it has its uses, just not for this stuff. It might be useful to serialize data such as stats or something that isn't ordered, is constant.
 
Upvote 0
Initiate Mage
Joined
Nov 20, 2016
Messages
3
Reaction score
0
Protobufs won't work because the MapleStory client does not parse the protobufs binary format.
 
Upvote 0
Back
Top