ColexDev (re-coding Lithium, cuz it sucks)

[Novak]

Edit: So thinking off it, re-coding isn't the right word (sorry) It's basically re-structuring it completely.

So as people may know I got a v145 source, but it's based of astral.

(Which is based of vicious? Which is based of Lithium? Which went bat-s**t-crazy on odin?)

Earlier I had no clue how maple worked and just mindlessly tried updated. It helped me learn about some general things and got me quite into packets. But now it's time to just get something worthwhile going.

My to do list so far:

1. Re-do the handling of packets, odin/moople alike. (done!)
2. Go over all Handlers, remove useless files/ handlers, re-code bad code (45% done)
3. Re-structure the entire thing: some things just really don't make sense. They often just mashed a couple things into one file. (10% done?)
4. Change how loading (wz) data works. (idk how yet, but it HAS to be changed)
5. Make map scripts into actual scripts, not hardcoded (got this from that weird v153 thread, was 100% right tho)
6. Just generally get rid of switches in most things that shouldn't process more then once? (commands, skills,..?)
7. Become an overall slightly better coder, cuz I really am not at all. I know a bit about how to write stuff in Java, but know little about how java itself actually works

Any suggestions on what to change? Let me know!

I want to make something better here. And something different. Cuz right now, most people are just adding sh*t to something that's allready sh*t. And it stinks.

Don't ask for a (git) link. I'll provide one if I feel the need.

(P.S. This thread was my main motivation: https://forum.ragezone.com/f683/packetprocessor-switch-1046056/)

 

[Saii]

I just want to point out that if you truly want to fully reconstruct the source, it will take you ETERNITY.
like seriously, it's not worth your time. I did it 2 years ago and I had some really pro guy that did stuff for me like handlers and map scripts. (and we were not even done by the time we dropped it)
updating it, adding features and making stuff work is just not worth it.
after all, assuming you're using ok hardware you will not feel any performance differences. if you're planning for bigger changes, might as well make your own source based on something different.

disclaimer:
this was not meant to demotivate you, I wish you best of luck. just shared my thoughts from personal experience.

 

[Five]

If you can't even make a v145 source with any features of the game working, how do you plan on pursuing this little development project of yours?

 

[Linkerzz]

I wish you the best of luck with this, as your project here seems rather difficult. :mellow:

 

[Novak]

I just want to point out that if you truly want to fully reconstruct the source, it will take you ETERNITY.
like seriously, it's not worth your time. I did it 2 years ago and I had some really pro guy that did stuff for me like handlers and map scripts. (and we were not even done by the time we dropped it)
updating it, adding features and making stuff work is just not worth it.
after all, assuming you're using ok hardware you will not feel any performance differences. if you're planning for bigger changes, might as well make your own source based on something different.

Indeed, it won't turn out to be much of an improvement, and for bigger changes coding my own source would be better.

But I'm pretty noob, but quite interested still in maplestory. I'm trying to get knowledge on all fronts of how it works, and server wise that includes understanding the loading of the wz data, the scripting interface, the handling of packets and the general structure. I don't know how much I'll be changing during this project, but it will probably not come to a 100% completely restructured lithium. But every little improvement and new implementation will be a way for me to learn a bit in my free time

I do want to start my own server from scratch somewhere in the feature! But I'm 100% sure I will have to learn more about everything first in order to produce something that is worthwhile!

p.s. no demotivation taken, i got my own reasons for attempting this

If you can't even make a v145 source with any features of the game working, how do you plan on pursuing this little development project of yours?

Honoustly you are confusing me a bit.
I mean, I totally agree, if you can't even get basic gameplay functions working on a source that basically works, then you can't get anything done since you obviously have no clue how things work. Buy how do you mean "not any feautures of the game working" :/ ? Is it targeted at me, or a general statement? If you mean that I can't code my own source from scratch, you are right. But the v145 is working quite well gameplay function wise. Nearly all skills are handled right, I used IDA to update all opcodes and a few packets that changed, scripted some missing instances, coded some beast tamer quests, etc.. It plays quite well..

No offense or anything! I just have honoustly don't know what you are refering to!

 

[ExtremeDevilz]

Indeed, it won't turn out to be much of an improvement, and for bigger changes coding my own source would be better.

But I'm pretty noob, but quite interested still in maplestory. I'm trying to get knowledge on all fronts of how it works, and server wise that includes understanding the loading of the wz data, the scripting interface, the handling of packets and the general structure. I don't know how much I'll be changing during this project, but it will probably not come to a 100% completely restructured lithium. But every little improvement and new implementation will be a way for me to learn a bit in my free time

I do want to start my own server from scratch somewhere in the feature! But I'm 100% sure I will have to learn more about everything first in order to produce something that is worthwhile!

p.s. no demotivation taken, i got my own reasons for attempting this



Honoustly you are confusing me a bit.
I mean, I totally agree, if you can't even get basic gameplay functions working on a source that basically works, then you can't get anything done since you obviously have no clue how things work. Buy how do you mean "not any feautures of the game working" :/ ? Is it targeted at me, or a general statement? If you mean that I can't code my own source from scratch, you are right. But the v145 is working quite well gameplay function wise. Nearly all skills are handled right, I used IDA to update all opcodes and a few packets that changed, scripted some missing instances, coded some beast tamer quests, etc.. It plays quite well..

No offense or anything! I just have honoustly don't know what you are refering to!

It definely going to be very time consuming not forgetting how terrible it was rewritten and given that it is almost as no use for it. Unless you have a team like Odin it is going be quite hard to change it back also I hope you have understanding of RMI and able to code Java intermediate level and is able to write a source from scratch then attempt the change if not don't waste your time you can learn something usefully about Java itself

 

[Novak]

It definely going to be very time consuming not forgetting how terrible it was rewritten and given that it is almost as no use for it. Unless you have a team like Odin it is going be quite hard to change it back also I hope you have understanding of RMI and able to code Java intermediate level and is able to write a source from scratch then attempt the change if not don't waste your time you can learn something usefully about Java itself

I'm actually learning alot about java and RMI right now! But don't worry I won't turn any of this into a time waster. It's more of a front to steer me to new things to learn, and try to implement the things I've learned directly to see if I can. Cuz after reading about something, that won't mean I can immedeatly apply it.

Some of the things I might be trying to implement are purely for fun, such as a new server managing interface that gives more control then a mere .bat without even a command line.

So far it's already fun


I think it's also good to say this: I didn't put a timestamp on this project on purpose. I'll be treating this more like maplebit these days. Stuff is happening, but updates will not just appear on a regular base. I will be trying to implement new things and improve the old, but I won't let a hobby like this stand in the way of my personal life. Will I finish this project? Yes. Will I finish the project soon? Hell no. I'll only work on free time. (with hell no i mean, not even in the forseeable future)

 

[Snopboy]

Sounds good, good luck with your development and I hope you'll learn from it.
Guys don't discourage him, best way to learn is by actual practice.

 

[sunnyboy]

If you're going to restructure a source, you might as well just make a new source. That way you gain a lot of more experience and you can do things all your way

 

[oxysoft]

I'm actually learning alot about java and RMI right now

dude duck RMI

stay away from this cancer

also here are two ideas if you want to make things nicer:

• Restructure minigame stuff because it is poorly implemented in all MS sources
• Some sort of higher-level special command parsing rather than keep accessing poop through unclean array accessors

 

[even]

dude duck RMI

stay away from this cancer

also here are two ideas if you want to make things nicer:

• Restructure minigame stuff because it is poorly implemented in all MS sources
• Some sort of higher-level special command parsing rather than keep accessing poop through unclean array accessors

To add fuel to the fire.

You must be registered to see links

 

[Zygon]

dude duck RMI

stay away from this cancer

Explain your rationale? RMI is perfectly fine if used properly. The only issues with it ends up being that people simply neglect the security and leave the port open with an SSL certificate completely unchanged from OdinMS Rev 988.

 

[oxysoft]

Explain your rationale? RMI is perfectly fine if used properly. The only issues with it ends up being that people simply neglect the security and leave the port open with an SSL certificate completely unchanged from OdinMS Rev 988.

no rationale, I just find it annoying having to run all the different components (login,world,channels) separate from each other; plus, last time I tried, I couldn't run the server inside of my IDE for debugging purposes because you had to run 3 different things at once. (IntelliJ can actually do that but it's still a pain having to run them all, especially when you need to re-deploy quite often)

I always found it more convenient to have everything clumped together

 

[Arcus]

If you can't even make a v145 source with any features of the game working, how do you plan on pursuing this little development project of yours?

stone cold savage.

 

[Novak]

I've been searching around reading up on RMI and there appears to be no solid conclusion on what is better. Let alone if RMI or mina are the best options to go with at all. As far as I can tell the best way to go is performance testing. I do however have to agree with oxysoft that running separate instances can be rather annoying.

As for now I'm getting started on making small scripting interfaces for both map scripts and item scripts. That will be handled just like other scripts and no longer via a long switch somewhere in the source. (just 1 call to the script)

I do have a question!:
The reason that the packet processor is better then the lithium switch is because it sorts the handlers into a linear array with the opcode as the index number. (the shortest possible deduction (leaving assembly out) from a series of posts SuperLol made regarding how this works)
This is possible because no opcode has the same number.

So my question is: how bout skills...
If you don't get what I'm going at: What if you process them (buffs w/ statups) the same way?
1. Create a file with all skills where you (sorted by jobs) name them as you would do with opcodes (ex. MAGIC_PROTECT = 51000012 ;(just a rand num))
2. Create a skill processor that functions the same as the packet processor would (no skillID is the same right?)
3. Create java packages for all jobs
4. Put handlers for skills, (basically calling onto a statup and basic skills packets) into said java packages.

Would this be in any way worthwile? Since a lot of skills handle in the same way, with the only major difference being the statups. But since they get called on a shitload (with only recvs being handled more often then skills) it might be a fun little optimazation. Let alone that I prefer sorting statups by their job instead of creating one hughe file with random skillID's you have to reference with a handbook or w/e to figure out what it belongs to and what it should do.

So! What do you think??

 

[Five]

I've been searching around reading up on RMI and there appears to be no solid conclusion on what is better. Let alone if RMI or mina are the best options to go with at all. As far as I can tell the best way to go is performance testing. I do however have to agree with oxysoft that running separate instances can be rather annoying.

Running separate instances isn't annoying and allows you to scale your server. I would much rather be able to run my server on multiple machines vs one machine so I am not completely limited in this aspect.

I would advise the switch to Netty instead of Mina. I did this a long time ago and I know a few others who did as well. You may not notice much performance difference local or with a small server, but you should notice it on a large scale server as well as it fixes those random mina session exits (never figured out why these occured)

So my question is: how bout skills...
If you don't get what I'm going at: What if you process them (buffs w/ statups) the same way?
1. Create a file with all skills where you (sorted by jobs) name them as you would do with opcodes (ex. MAGIC_PROTECT = 51000012 ;(just a rand num))
2. Create a skill processor that functions the same as the packet processor would (no skillID is the same right?)
3. Create java packages for all jobs
4. Put handlers for skills, (basically calling onto a statup and basic skills packets) into said java packages.

Would this be in any way worthwile? Since a lot of skills handle in the same way, with the only major difference being the statups. But since they get called on a shitload (with only recvs being handled more often then skills) it might be a fun little optimazation. Let alone that I prefer sorting statups by their job instead of creating one hughe file with random skillID's you have to reference with a handbook or w/e to figure out what it belongs to and what it should do.

I don't really get why you would do this. I never understood why people made buffs so unorganized to begin with. Personally I made a file with every skill name and their id (like you suggested in this post) and then rewrote the skill handling files (Passive and Buffs) to read from 1 new file I have that just organizes job by job each skill and their corresponding effect.

I don't think you need to over complicate skills if you are just looking for organization. God forbid you actually make 50 files for 50 jobs...

 

[Novak]

Personally I made a file with every skill name and their id (like you suggested in this post) and then rewrote the skill handling files (Passive and Buffs) to read from 1 new file I have that just organizes job by job each skill and their corresponding effect.

I don't think you need to over complicate skills if you are just looking for organization. God forbid you actually make 50 files for 50 jobs...

No that's the deal. handling each one in an individual file would be annoying. The way you do it sounds better. I think I'll sort it like that too.

 

[Shmoconut]

No that's the deal. handling each one in an individual file would be annoying. The way you do it sounds better. I think I'll sort it like that too.

Handling each one in an individual file could be good design, assuming there is a solid enough abstraction for it. Jobs and skills definitely can be designed better than they are in any existing source right now.
I don't know why people are so averse to having lots of files. When justified by design, having abstractions and many files is good and expected.


There will be a time when you realize the importance of separate "instances" of things. Not only because of the various performance or scale benefits, but also because it's the right thing to do.

The use of RMI (and RMI-esque RPC in general, really) is dying out as people tackle problems at high scale. There is nothing wrong with using RMI in a MapleStory server, and it's definitely better than a single JVM for the server, but it's not up to date with current tech.

Netty vs Mina doesn't matter in a MapleStory server. I don't know the architectural differences between the two, but I do know that benchmarks between them and throughput conclusions are irrelevant because MapleStory servers will never have the load to make it matter. Before Netty became the standard Java networking library, Mina was used commercially for projects that had way more load than a MapleStory server ever did.

With all that said, it's still good to update old tech, but the disclaimer is that you probably won't notice a difference at all, and you certainly won't be adding many new capabilities.

A lot of conversations are about changing tech, which is good, but if you want to make a source better, then adhere to good design patterns and make flexible, extensible code. It really doesn't matter if you use Mina/Netty or RMI/Messaging/Akka. The performance benefits will never be seen through real load. But what will matter is if all your job/skill stuff is in 1 file and you have to add a lot of janky code in several places to handle a new job's skills, versus extending existing capabilities by following SoC. Or if your packet handling code is procedural Java instead of taking advantage of OOP features like polymorphism.

 

[Zygon]

Netty vs Mina doesn't matter in a MapleStory server. I don't know the architectural differences between the two, but I do know that benchmarks between them and throughput conclusions are irrelevant because MapleStory servers will never have the load to make it matter. Before Netty became the standard Java networking library, Mina was used commercially for projects that had way more load than a MapleStory server ever did.

Regarding this, I found Netty much easier and more intuitive to use than MINA. I agree completely there's no difference for the servers we run performance-wise; however, I would prefer Netty over MINA in any future project I work on simply because it is easier to work with.

Handling each one in an individual file could be good design, assuming there is a solid enough abstraction for it. Jobs and skills definitely can be designed better than they are in any existing source right now.
I don't know why people are so averse to having lots of files. When justified by design, having abstractions and many files is good and expected.

I feel like this the really tough part about programming an emulator for this game: deciding when there's enough difference between two really similar features to justify splitting them or creating abstraction for them. The only really big one I can think of would be field objects since they all have some very common basic code (OID, spawning, despawning), but spread out to far more diverse functionality depending on what they are. Quests definitely deserve more love than what they have right now and could use an overhaul. Scheduling could also use some tuning since it's the main reason most Odin-based emulators can't see past about a week of online time without needing a reboot (there's no fall back if for some reason a thread deadlocks). EXP distribution also seemed really poorly designed in base Odin since it seemed needlessly convoluted for tracking damage dealt to a monster.

I think one of the really strange design choices in original Odin was their I/O which I could have honestly seen being cut down to like 2-3 files each side at max.

 

[Shmoconut]

Regarding this, I found Netty much easier and more intuitive to use than MINA. I agree completely there's no difference for the servers we run performance-wise; however, I would prefer Netty over MINA in any future project I work on simply because it is easier to work with.

Yeah don't get me wrong; MINA IS old and not actively maintained. For all intents and purposes, you should be using Netty in new projects because it's actively maintained and holistically considered better. My point is that there's no reason to replace MINA with Netty for the sake of performance. Before MS servers, MINA was used in commercial products that handled more load than any MS server has. You should only replace it for the sake of learning Netty, because it isn't really worth it otherwise. There are a couple noninvasive ways to implement Netty which don't require a lot of work, though.

Scheduling could also use some tuning since it's the main reason most Odin-based emulators can't see past about a week of online time without needing a reboot (there's no fall back if for some reason a thread deadlocks). EXP distribution also seemed really poorly designed in base Odin since it seemed needlessly convoluted for tracking damage dealt to a monster.

I think one of the really strange design choices in original Odin was their I/O which I could have honestly seen being cut down to like 2-3 files each side at max.

Scheduling can be made nicer, but at the heart of it, it's just a scheduled threadpool executor that runs Runnables wrapped in a try/catch. It depends on what kind of problems you are talking about too; monsters stopping spawning/map-related problems are usually because code is not threadsafe, and that code should be fixed instead of the scheduler. Apart from some utility code for checking scheduler health and some fallback logic (but this is already implicitly possible with the wrapped Runnable), nothing done to the executor will address the root problems.
People should be more meticulous about what scheduled tasks to keep in queue. As an example, when a poisoned monster dies, you want to cancel its poison schedule or it could lead to undesired consequences. This can be applied to everything; if a task is going to modify state, then it needs to be monitored and canceled at the right times. It's a bit more complex than that too, because even certain packets shouldn't be sent with old data or they could cause a d/c.

I don't remember original Odin exp tracking, but I thought it was mostly fine. The external synchronization is weird and error-prone, though.