Map Disposal (Lithium-based)

[QuietCrystal]

It's been a while since I've released something, so here's something small. It's been coded on MapleCrystal, which is based off EliteMS, which is based off FuckMS, which is based off Lithium. But you'll be able to get it on any source on any version if you're capable of more than just copy-pasting text.

Map Disposal
At its simplest, this release removes unused maps after 30 minutes. That's all.

Instructions

Step 1:
Under MapleMap.java, add:

    private long lastPlayerLeft = System.currentTimeMillis();

    public boolean canDelete() {
        return (System.currentTimeMillis() - lastPlayerLeft > (30 * 60 * 1000L)) && (getCharactersSize() == 0);
    }

Step 2:
Still under MapleMap.java, look for the removePlayer() function, which should look something like:

    public final void removePlayer(final MapleCharacter chr) {
        ...
        ...
        ...
    }

Add this line at the top of the function, like so:

    public final void removePlayer(final MapleCharacter chr) {
        lastPlayerLeft = System.currentTimeMillis();
        ...
        ...
        ...
    }

Step 3:
Go to World.java, and look for handleMap function:

    public static void handleMap(final MapleMap map, final int numTimes, final int size, final long now) {
        ...
        ...
        ...
    }

Again, add the following code at the top of the function:

    public static void handleMap(final MapleMap map, final int numTimes, final int size, final long now) {
        if (map.canDelete()) {
            map.resetFully(false);
            MapleMapFactory mmf = ChannelServer.getInstance(map.getChannel()).getMapFactory();
            if (mmf.isMapLoaded(map.getId())) {
                if (mmf.getMap(map.getId()) == map) {
                    // Ensure we are deleting the right map
                    mmf.removeMap(map.getId());
                }
            }
            return;
        }
        ...
        ...
        ...
    }

And that's it.

How it works is, every time a player leaves a map, the time is stored as "lastPlayerLeft". When canDelete() is called, if there are no players on the map and the last time a player left the map was more than 30 minutes ago, then the map is taken off the channel's MapleMapFactory. An extra check is added so that if maps are instanced (for instance, player houses), it will not wrongly delete the original map from the channel.

Whether or not this might cause memory leaks depends on whether or not map.resetFully(false), followed by deleting of the map causes memory leaks. If it does, then don't use this. Or clean up the leaks like any self-respecting programmer would.

Why use this?

One of the reasons why servers require quite a fair bit of RAM is because of maps (as well as the drops, mobs, reactors, whatever) taking up space. For smaller servers this might not be a problem. However, for bigger servers with more channels, having 10 instances of explored maps for 10 different channels can become quite the burden to the RAM, considering most maps are simply passed through, or explored for only a few moments before it's empty for the rest of the server's uptime.

While honestly I don't know how much RAM maps consume on servers in general, I think it is still good to remove unused maps after long periods of time.

Feel free to discuss the usefulness of this, because I still have doubts if it'll actually solve some memory problems, and if it does, why has it not been implemented in most, if not all public sources.

[MentaL]

[ExtremeDevilz]

this is beautiful :)

odin server does not has this.. but I believe Vana does..

anyway the memory problem you are talking about is mostly cause of how the provider in odin was written, It was suggested years ago to switch to something less verbose like MCDB and now you even have NX which are both very lightweight and fabulously will speed up the server loading and less memory consumption..

overall great release :)

[LazyBui]

Feel free to discuss the usefulness of this, because I still have doubts if it'll actually solve some memory problems, and if it does, why has it not been implemented in most, if not all public sources.

I can't speak to the implementation question.

However, I can say that if you have memory problems, this is most likely not a solution to the problem. This is a mitigation of the problem. It may be effective enough based on some usage patterns and map counts to be a solution, but that's situational and not addressing the core issue of consuming too much memory. Phrased differently, yes, there is a difference (possibly a large one) between having 3k maps loaded and 2k maps loaded. But everything still has high memory consumption, so all you've really done is move the goalpost.

That said, there are also a few things I think haven't been considered in the deletion mechanism.

First off, this will make bosses like Pianus respawn after 30 minutes instead of the intended 8 hours or whatever it is.

Secondly, things like boat animations (depending on how you do them) may require you to keep those maps alive regardless of how long it's been since a player was on them. Vana has them state-controlled by Lua so there's absolutely no code in those maps that refers to any particular boat. This means they have to be maintained in memory so the state can be maintained.

Another factor is that a map may be a part of a map set. In Vana, I call these instances. Things like Kerning PQ, etc. As long as the instance is active, it is a bad idea regardless of whether or not there are players on the map to unload it.

Just some things I discovered in implementing this feature myself. Hope that helps!

[Halcyon]

This could be useful for WZ editting servers, because I think when you join a map, it's loaded, so nothing can be done to afterwards, so I kind of wanted something like this for a while.

[ExtremeDevilz]

This could be useful for WZ editting servers, because I think when you join a map, it's loaded, so nothing can be done to afterwards, so I kind of wanted something like this for a while.

This isn't for wz edited server but more for the server to unload maps which don't have any characters in them it should help in performance and memory wise by tons but like what LazyBui said this would cause issues for pianus and instance map... Other then that it is a very neat feature that should have been in Lithium/Odin way back..

[resinate]

Edit: Fixed

i made a system in maplemap for instances, if any map is an instance itll skip it.

what i find funny is how some maps have hidden spawns aka no spawns in game but in wz file it shows up under "life"

[sunnyboy]

this code RECKS events would be nice to have this not trigger with PQ events

So add checks for specific maps to not execute

[QuietCrystal]

However, I can say that if you have memory problems, this is most likely not a solution to the problem. This is a mitigation of the problem. It may be effective enough based on some usage patterns and map counts to be a solution, but that's situational and not addressing the core issue of consuming too much memory. Phrased differently, yes, there is a difference (possibly a large one) between having 3k maps loaded and 2k maps loaded. But everything still has high memory consumption, so all you've really done is move the goalpost.

Actually from the way I see it, not really. From my experience in private servers, maps are loaded because:
1) They are significant maps (e.g. towns, FM)
2) They are training maps
3) They are passed-by

Point 3 is the main target of this code, mostly because these maps are of least importance out of those 3 points. People play hide and seek events, might just want to hang out in nice maps like Nine Spirit Nest, or are just passing through to training grounds. This code will remove those.

To address your point, I disagree with your "moving the goalpost" point. Imagine a popular server, vs a quiet server. Both will, after a matter of hours, load approximately the same number of maps (the popular server will probably have a higher map count, but only slightly). The reason is simple, because there will always be the same significant maps, training maps, and the maps required to pass through to reach the training grounds. Yes, the lack of activity means not so many obscure maps will be loaded on the quiet server, but the map loading is not as proportional.

To give some numbers, it's sort of like this:
Active server: 250 online players, 300 maps loaded
Inactive server: 5 online players. 230 maps loaded

This system will end up assisting these inactive servers more, so we have almost proportionately that amount of maps stored.

Inactive server: 5 online players, 30 maps loaded

Why? Because once these players AFK, or remain in their training maps, then the other maps could be safely deleted to free memory.

That said, there are also a few things I think haven't been considered in the deletion mechanism.

First off, this will make bosses like Pianus respawn after 30 minutes instead of the intended 8 hours or whatever it is.

Secondly, things like boat animations (depending on how you do them) may require you to keep those maps alive regardless of how long it's been since a player was on them. Vana has them state-controlled by Lua so there's absolutely no code in those maps that refers to any particular boat. This means they have to be maintained in memory so the state can be maintained.

Agreed, these have not been considered. But to address these concerns, I can think of solutions. (Though it will never be a perfect fix)

There's always

So add checks for specific maps to not execute

Or, you can simply increase the reload timing to 8 hours for just that map. (A sad hardcoding, yes)
Unless you make the time to respawn for specific spawn time follow the channel's global variable. This way, no matter how many deletions, Pianus will always spawn at the correct time.

About the boats, I think the boat checking is per entry in Odin. If the player enters, only then the time is calculated and the boat is shown/hidden. Then at the important intervals, the packet is again sent to show/hide the boat. There's probably going to be a way to work around that in Vana

Another factor is that a map may be a part of a map set. In Vana, I call these instances. Things like Kerning PQ, etc. As long as the instance is active, it is a bad idea regardless of whether or not there are players on the map to unload it.

Instead of loading the whole "instance", why not load each map as you enter? Monster park, PQs, etc. Load the next map upon entry. Reset mobs, shuffle reactors just as a player enters.

Or if you want them as a set, then extend the time to deletion for these maps. Alternatively, instead of just a long variable to store time, have a boolean "canOverrideDispose" or something. So once this is set, then at canDelete(), instead of just checking the time, it checks the overriding variable. If false, do not delete. Once set to true, it can be deleted.

So how it works is, set this deleting boolean to false at the start of the PQ, and once ended, set it to true, to automatically delete. Better yet, manually delete it on the spot to prevent clashes.

Just some things I discovered in implementing this feature myself. Hope that helps!

It was an interesting discussion (:

[Fraysa]

Instead of doing this, just implement lazy-load of maps in OdinMS. This is something that is done on a lot of programs that require huge amount of objects to load at the initialization process. In averge, 80% of the maps aren't visited by players, so why load them? Load them upon request only. It will reduce load dramatically, especially in OdinMS, where data load is about a minute or so.

Not it to mention lazy-loading can be implement for almost everything: Maps, Guilds, etc.

[QuietCrystal]

Instead of doing this, just implement lazy-load of maps in OdinMS. This is something that is done on a lot of programs that require huge amount of objects to load at the initialization process. In averge, 80% of the maps aren't visited by players, so why load them? Load them upon request only. It will reduce load dramatically, especially in OdinMS, where data load is about a minute or so.

Not it to mention lazy-loading can be implement for almost everything: Maps, Guilds, etc.

I'm actually targeting this system with my code. (Lithium-based, remember?)

From my experience in private servers, maps are loaded because:
1) They are significant maps (e.g. towns, FM)
2) They are training maps
3) They are passed-by

Point 3 is the main target of this code, mostly because these maps are of least importance out of those 3 points. People play hide and seek events, might just want to hang out in nice maps like Nine Spirit Nest, or are just passing through to training grounds. This code will remove those.

... Imagine a popular server, vs a quiet server. Both will, after a matter of hours, load approximately the same number of maps (the popular server will probably have a higher map count, but only slightly). The reason is simple, because there will always be the same significant maps, training maps, and the maps required to pass through to reach the training grounds. Yes, the lack of activity means not so many obscure maps will be loaded on the quiet server, but the map loading is not as proportional.

To give some numbers, it's sort of like this:
Active server: 250 online players, 300 maps loaded
Inactive server: 5 online players. 230 maps loaded

This system will end up assisting these inactive servers more, so we have almost proportionately that amount of maps stored.

Inactive server: 5 online players, 30 maps loaded

Why? Because once these players AFK, or remain in their training maps, then the other maps could be safely deleted to free memory.

[resinate]

do u mind if i released a more stable version of this code?

i moved this code to remove player under maple map

[QuietCrystal]

do u mind if i released a more stable version of this code?

i moved this code to remove player under maple map

Be my guest.

[LazyBui]

Actually from the way I see it, not really. From my experience in private servers, maps are loaded because:
1) They are significant maps (e.g. towns, FM)
2) They are training maps
3) They are passed-by

Point 3 is the main target of this code, mostly because these maps are of least importance out of those 3 points. People play hide and seek events, might just want to hang out in nice maps like Nine Spirit Nest, or are just passing through to training grounds. This code will remove those.

I know the use cases. I'm not a fetus. I didn't call into question the usefulness.

To address your point, I disagree with your "moving the goalpost" point. Imagine a popular server, vs a quiet server. Both will, after a matter of hours, load approximately the same number of maps (the popular server will probably have a higher map count, but only slightly). The reason is simple, because there will always be the same significant maps, training maps, and the maps required to pass through to reach the training grounds. Yes, the lack of activity means not so many obscure maps will be loaded on the quiet server, but the map loading is not as proportional.

To give some numbers, it's sort of like this:
Active server: 250 online players, 300 maps loaded
Inactive server: 5 online players. 230 maps loaded

This system will end up assisting these inactive servers more, so we have almost proportionately that amount of maps stored.

Inactive server: 5 online players, 30 maps loaded

Why? Because once these players AFK, or remain in their training maps, then the other maps could be safely deleted to free memory.

You claim to disagree with me, but your argument is in support of what I'm saying. I said whether it's a solution or not is very situational and then you proceeded to throw out situations.

Delightfully, your first two prove the point I was making as well. The difference between 300 and 230 maps is fairly negligible. If your map costs 100 bytes of memory, it's 22.4KB versus 29.2KB. That's a difference of 6.8KB. Weighed against countless MB of memory consumption, it's almost nothing. Additionally, if you chopped off 20 bytes from each map, you'd save about the same.

I can reiterate my point here: It doesn't address the core issue and isn't really a solution. It's a mitigation that under some conditions may "solve" the problem without really solving it. It's a good idea regardless, but that doesn't make it a solution to the problem you're addressing.

EDIT: I've been told that 100 bytes per map is "too small." So we'll assume 10000 bytes per map which is probably more likely. Now replace 22.4KB with 2.19MB, 29.2KB with 2.86MB, and 20 bytes with ~3000 bytes. The .67MB is still completely negligible in the face of tens or hundreds of MB.

EDIT2: I realize now that I slightly missed the point you were going for with the way it was phrased. If you have 5 players, you shouldn't be having memory problems period, regardless of whether 5 maps, 300 maps, or 3000 maps are loaded. Yeah, if you have 1% of the maps loaded versus the other scenario, you'll be using roughly 1% of the memory. But it's all about moving the goalpost. Even if you don't have those maps, that memory could just as easily come from another variable, like number of players.

How is it not moving the goalpost? It's like changing a soccer goal 15 feet to the left. Yeah, that might mean that fewer goals are scored, but it's still the same width and height. The rules didn't change. That's literally what "moving the goalpost" means. High memory usage is still systemic and not a product of any specific count of resources.

When you reduce one bottleneck, something else becomes the bottleneck unless everything is in perfect equilibrium (which isn't really possible with respect to memory consumption). And even then, your scenario assumes binary states of playing and AFK. Really it'd be more like a percentage of activity throughout the day depending on your player base. So I say again, it becomes so situational that it's meaningless from a systems design/memory perspective. There's nothing stopping all those maps you would otherwise unload from being loaded all the time, so you haven't addressed the issue.

[QuietCrystal]

Delightfully, your first two prove the point I was making as well. The difference between 300 and 230 maps is fairly negligible. If your map costs 100 bytes of memory, it's 22.4KB versus 29.2KB. That's a difference of 6.8KB. Weighed against countless MB of memory consumption, it's almost nothing. Additionally, if you chopped off 20 bytes from each map, you'd save about the same.

No? The original comparison was without the code added. The new value of the inactive server is what it would be like after the code was added.

It's more of a proportional thing. Right now for most servers, after a few days of being up, let's say around 80% of the maps are loaded, a bit less for inactive servers, around 75% (figures here are randomly suggested in order to show proportion)

This code reduces the maps significantly for inactive servers. Meaning less players, less memory. Simple as that. So instead of running around with 75% of the maps loaded, you can get away with maybe 5%, or even 0% when nobody is on.

Active servers on the other hand probably would end up with around 50% loaded.

Because in the end, each player will only load a certain amount of new maps every 30 minutes. Let's say this number averages out to 10 (which is quite generous, considering once people start hanging out in towns, FM, or grinding they stay in the map for quite a long time).

In servers without this code, it doesn't matter how many players there are. The memory is around the same. Once this code is added, then we will have approximately (10 maps * number of players) maps loaded. Significant difference in loaded maps for servers with about 5-10 players.

Then. take into account that with more players, there will be more greater overlaps of these "10 maps" that the player loads. So this number reduces to maybe 7 maps per player. Then the number of maps loaded would be about (7 * number of players), +- a bit.

Because without the code, the number of maps is something like a logistic function. Curves up steeply, then caps off at a certain point. Of course, active servers achieve the "plateau" faster.

With the code, I make the maps loaded proportional to the number of players. Less players, less maps, less memory. More players, more maps, more memory.

Why plateau with so many maps loaded even with so little players? That's what the code is here to do.

So no, I'm not "proving your point". I didn't mean what you thought I meant.

[resinate]

some bugs i found, its constantly removing the same maps lol so heres a massive fix i made for the code. feel free to front page it.

i removed the timer cuz i felt it wasnt needed.

heres what this fixes:
if map is instanced in an event
if map was alrdy unloaded no need to unload again

MAPLEMAP.JAVA

    public boolean canDelete() {
        return this.characters.isEmpty() && !Mapinstance() && ChannelServer.getInstance(getChannel()).getMapFactory().isMapLoaded(mapid);
    }

WORLD.JAVA
inside handle map

        if (map.canDelete()) {
            map.resetFully(false);
            ChannelServer.getInstance(map.getChannel()).getMapFactory().removeMap(map.getId());
            System.out.println("Maple Map: " + map.getMapName() + " - ID: " + map.getId() + " has been unloaded from system.");
        }