Optimization of loading NPC's

[Jomex]

Hi,

Yesterday I thought: "maybe I can make loading characters better...".

So I looked into source and saw how rubbish is LoadCharacter function....

Then I decided to improve it.


This is the example of old-style character coding:

Spoiler

MaFl_Boboko
{
    setting
    {    
        AddMenu( MMI_DIALOG );
        AddMenu( MMI_TRADE  );
        AddVendorItem( 0, IK3_SUIT, 1, 15, 27, 25 );
        AddVendorItem( 0, IK3_HELMET, 1, 15, 27, 25 );
        AddVendorItem( 0, IK3_GAUNTLET, 1, 15, 27, 25 );
        AddVendorItem( 0, IK3_BOOTS, 1, 15, 27, 25 );
        AddVendorItem( 1, IK3_SUIT, 3, 15, 27, 25 );
        AddVendorItem( 1, IK3_HELMET, 3, 15, 27, 25 );
        AddVendorItem( 1, IK3_GAUNTLET, 3, 15, 27, 25 );
        AddVendorItem( 1, IK3_BOOTS, 3, 15, 27, 25 );
        AddVendorItem( 2, IK3_SUIT, 2, 15, 27, 25 );
        AddVendorItem( 2, IK3_HELMET, 2, 15, 27, 25 );
        AddVendorItem( 2, IK3_GAUNTLET, 2, 15, 27, 25 );
        AddVendorItem( 2, IK3_BOOTS, 2, 15, 27, 25 );
        m_nStructure= SRT_SHIELD;
        SetImage
        (
        IDS_CHARACTER_INC_000026
        );
        m_szDialog= "MaFl_Boboko.txt";
    }

    SetName
    (
    IDS_CHARACTER_INC_000027
    );

    AddVendorSlot( 0,
    IDS_CHARACTER_INC_000028
    );

    AddVendorSlot( 1,
    IDS_CHARACTER_INC_000029
    );

    AddVendorSlot( 2,
    IDS_CHARACTER_INC_000030
    );

    AddVendorSlot( 3,
    IDS_CHARACTER_INC_000031
    );
}

There are also other functions interpreted by source like:

Spoiler

- SetOutput( false );
- AddTeleport
- SetBuffSkill
- SetFigure
- SetEquip
- SetVenderType
- AddVenderItem2

So at First what are "setting" and "{" for??
The answer is short - nothing!

So I deleted every "setting" and "{" and the "}" of setting construction.

Also instead of "(" I decided to make source read '=' as opener and instead of ')' ';' as closer.

So the character.inc file was smaller and it looked like that:

Spoiler

MaMa_Ancimys
{
    

    AddMenu = MMI_DIALOG;
    SetImage = 
    
    IDS_CHARACTER_INC_000199
    ;
    m_szDialog = "MaMa_Ancimys.txt";


    SetName =
    
    IDS_CHARACTER_INC_000200
    ;
}

Then I Changed some references as following:

Spoiler

SetImage => Image
SetName => Name
m_szDialog => Dialog
m_nStructure => Structure
AddVendorSlot => AddShopSlot
AddVendorItem => AddShopItem
AddVendorItem2 => AddShopItem2
SetVendorType => SetShopType
SetOutPut => Enable
SetBuffSkill => AddBuffSkill

So thanks to them script now is clearer.

The next thing is About Enabling the NPC. Why should a program load info about npc if it is not used at all( 1st consequence)?
So I made script stop reading npc info if Enabled = false.

What I found is some NPC's have items but MMI_TRADE is commented so I made the script bypass adding Shop Slots if MMI_TRADE is not included( 2nd consequence ).

The same thing with slots: if there is no slot with ID AddShopItem was defined it will bypass this part. ( 3rd consequence );

Also it is the same with Buffs and Teleport.

Then while I had no other Ideas I made every NPC as short as possible so it looks now like:

Spoiler

MaFl_TELEPORTER // 3??? ????? ???? npc
{
    Enable = false;
    AddMenu = MMI_DIALOG;
    AddMenu = MMI_TELEPORTER; 
        AddTeleport = 7161, 3264; //????
        AddTeleport = 8321, 3720; //??? ??
        AddTeleport = 5593, 3920; //?? 1
        AddTeleport = 3896, 4378; //?? 2
        AddTeleport = 3221, 3416; //?? 3
        AddTeleport = 5933, 5393; //????
    Image = IDS_CHARACTER_INC_000832;
    Dialog = "MaFl_teleporter_03nomal.txt";
    Name = IDS_CHARACTER_INC_000833;
}

You should have noticed the cons. I will Explain that.

1st con is about Enable/Disable option. To make It work it must be at the beginning of the npc script like here:

MaFl_July06   // 2010? ?? ?????01
{
    Enable = false;

2nd con is about the shop. To make everything work I had to move slots right after MMI_TRADE or it wouldn't work. Example:

MaEw_Ruoboru
{
    AddMenu = MMI_DIALOG;
    AddMenu = MMI_TRADE;
        AddShopSlot = 0, IDS_CHARACTER_INC_000231;
            AddShopItem2 =0, II_SYS_SYS_SCR_BXMKNT120SET;
            AddShopItem2 =0, II_SYS_SYS_SCR_BXMBLA120SET;
            AddShopItem2 =0, II_SYS_SYS_SCR_BXFKNT120SET;
            AddShopItem2 =0, II_SYS_SYS_SCR_BXFBLA120SET;
        AddShopSlot = 1, IDS_CHARACTER_INC_000232;
            AddShopItem2 =1, II_SYS_SYS_SCR_BXMRIN120SET;
            AddShopItem2 =1, II_SYS_SYS_SCR_BXMBIL120SET;
            AddShopItem2 =1, II_SYS_SYS_SCR_BXFRIN120SET;
            AddShopItem2 =1, II_SYS_SYS_SCR_BXFBIL120SET;
        AddShopSlot = 2, IDS_CHARACTER_INC_000233;
            AddShopItem2 =2, II_SYS_SYS_SCR_BXMJST120SET;
            AddShopItem2 =2, II_SYS_SYS_SCR_BXMRAG120SET;
            AddShopItem2 =2, II_SYS_SYS_SCR_BXFJST120SET;
            AddShopItem2 =2, II_SYS_SYS_SCR_BXFRAG120SET;
        AddShopSlot = 3, IDS_CHARACTER_INC_000234;
            AddShopItem2 =3, II_SYS_SYS_SCR_BXMPSY120SET;
            AddShopItem2 =3, II_SYS_SYS_SCR_BXMELE120SET;
            AddShopItem2 =3, II_SYS_SYS_SCR_BXFPSY120SET;
            AddShopItem2 =3, II_SYS_SYS_SCR_BXFELE120SET;
}

3rd con is almost same as with shop but there must be the menu for teleport before teleports and menu for buffs before adding the buffs.

Result for now is: from 333 KB down to 136 KB character.inc file.

If you have any questions ask.
Any ideas? serve them here!

If you know: Are NPC's from character-etc and character-school.inc used?

Ideas I am thinking about:

Spoiler

AddMenus => multi AddMenu to shorten the script and fasten loading the file

 

[Tex0]

If you know: Are NPC's from character-etc and character-school.inc used?

I am not 100 %ly sure but the character-school ( and also -etc) were used for an old v6 Event System. Now they are not used.

 

[spikensbror]

The reason for the AddMenu() with parentheses notations is cause it is used as A FUNCTION.
Before you go messing around with the script-parsing you should really go about learning how actual scripts function.
That's like doing printf = 'output'; which is just a bag of silly.
m_szDialog is done using a set operator because it is a variable.
Everything is fine with the current scripts and the only way you could "optimize" it would be that you rewrite the script and implement Lua as the parser engine and implement an API which could do the same things as the current API can.

 

[ShadowDragon]

The reason for the AddMenu() with parentheses notations is cause it is used as A FUNCTION.

You can call that a function if you want, but in the actual source coding, it isn't technically used as a function at all. The source coding reads in the ( and ), and simply discards them without doing anything with them. In reality the file is just a text file. You could program the source to read it in whatever way you want.

 

[Mootie]

You can call that a function if you want, but in the actual source coding, it isn't technically used as a function at all. The source coding reads in the ( and ), and simply discards them without doing anything with them. In reality the file is just a text file. You could program the source to read it in whatever way you want.

That's incorrect, it is used as a function. Think about it, in C++ when you write

printf("output");

the compiler reads "printf(%)" then stores the variable passed to it and executes the printf function.

AddMenu is a function, even if it doesn't have a formal C++ definition. It adds a menu based on the variable passed to it.

 

[ShadowDragon]

Eh, since you put it that way, I guess you're right. I was just going on the fact that it didn't have a formal C++ definition.

 

[Jomex]

Okey, masters so I made also lua reading and here are stats:

2012/ 7/ 4 12:36:21 File: charnew1.inc, Time: 32(ms) <= method from my 1st post

2012/ 7/ 4 12:36:21 File: character.lua, Time: 54(ms) <= lua method

2012/ 7/ 4 12:36:21 File: character.inc, Time: 42(ms) <= Old method

All of these files contain exactly same NPC's.

As you can see lua is the slowest here but it has to use CScript the most....

What I mean?

In Old & 1st method we use CScript as main parser and We do not need to get Define number so we just call GetNumber function
While in lua I had to call GetDefineNum 7 to make it work correctly.

Also Those are Size of files:

1st mehtod: 135 292 bytes
lua method: 153 745 bytes
old method: 340 890 bytes

Lua NPC model:

Spoiler

AddNPC( "MaEw_Pinanoco" )
--{
    AddMenu( "MMI_DIALOG" )
    AddMenu( "MMI_TRADE" )
        AddShopSlot( 0, "IDS_CHARACTER_INC_000948" )
            AddShopItem(  0, "IK3_BCHARM", -1, 20, 20, 50 )
            AddShopItem(  0, "IK3_RCHARM", -1, 20, 20, 50 )
            AddShopItem(  0, "IK3_ARROW", -1, 5, 5, 50 )
        AddShopSlot( 1, "IDS_CHARACTER_INC_000263" )
            AddShopItem(  1, "IK3_REFRESHER", -1, 1, 5, 50 )
            AddShopItem(  1, "IK3_DRINK", -1, 1, 5, 50 )
        AddShopSlot( 2, "IDS_CHARACTER_INC_000949" )
            AddShopItem(  2, "IK3_CIDMIX", -1, 50, 50, 50 )
            AddShopItem(  2, "IK3_OPERMIX", -1, 50, 50, 50 )
            AddShopItem(  2, "IK3_MEAL", -1, 50, 50, 50 )
        AddShopSlot( 3, "IDS_CHARACTER_INC_000261" )
            AddShopItem(  3, "IK3_PILL", -1, 1, 3, 50 )
    Structure( "SRT_GENERAL" )
    SetImage( "IDS_CHARACTER_INC_000978" )
    AddDialog( "MaEw_Pinanoco.txt" )
    SetName( "IDS_CHARACTER_INC_000979" )
--}

Now tell me how I can improve lua/1st script if you know

Also Spiken in source it's not made like function but like plain text.....


@EDIT:
I had to abandon adding MultiMenus because I don't know yet how to make such a function which would take "n" amount of arguments given.

Also I think about using definitions for example of structures to make the lua faster. Tell me what ya think about it.

 

[Mootie]

I would go with luascript, but perhaps you could develop a method to load the constants(such as "IK3_CIDMIX") into the lua VM as integers. That could dramatically increase the parsing speed I'd imagine.

 

[Jomex]

I would go with luascript, but perhaps you could develop a method to load the constants(such as "IK3_CIDMIX") into the lua VM as integers. That could dramatically increase the parsing speed I'd imagine.

That's what I meant with structures...

 

[Jomex]

New info:

The time that takes so long for lua method is RunScript and it takes 47(ms) from whole 54(ms) of loading so the reading is at awesome speed as Spiken said....

But if it is like that I'd like to make a function that would prepare all the .lua files that I will be using and make some way of reloading them without client/server restart but im not sure if it is possible.

If someone know tell me how to do it

 

[Addi]

If you're really interested in making it fast, you'd rewrite it as a binary file. Script parsing is soooooo slow. x~x

Removing a few characters isn't gonna show much of a difference in speed.

 

[ShadowDragon]

If you're really interested in making it fast, you'd rewrite it as a binary file. Script parsing is soooooo slow. x~x

That's a really good idea, but it adds the complication of any future changes to the file being more difficult. The only practical way would be making a program to convert it to binary, that would be used after every change.

Also, I have a comment about how this 'project' is almost useless. You only saved 10ms by changing the format? That's seriously not much, and unless you plan to change every file to be optimized, you're not going to speed things up much.

Note: I said almost useless. If by changing more files you see some significant speed increase, then it would be useful.

 

[Jomex]

If you're really interested in making it fast, you'd rewrite it as a binary file. Script parsing is soooooo slow. x~x

Removing a few characters isn't gonna show much of a difference in speed.

Never mentioned deleting useless NPC but well it would be way more than half .

@Dragon:
Not if you are planning to make unicode/utf-8 to get all language symbols

 

[spikensbror]

The reason it is not "called as a function" is cause the script is parsed, this method is just the same as the LuaVM parses the scripts or various compilers.
A compiler does not "call" the function, it parses in the function name and arguments, then determines the appropriate action.
Also, the reason Lua got so high ms is cause it's supposed to be done for ALL scripts. Try a like read 100 files stress test and then look at those results.

 

[Jomex]

Well I already said that construction of tables took 85% of time while reading just 15% or less...

 

[spikensbror]

Oh and what you described with the Lua is entirely possible, however you'd need to patch the files that's needed on the client-side in real time.

 

[Jomex]

Well while testing I don't care about it but it's still possible to make such a mechanism.


@EDIT:
Didn't mention that but when changing the parsing engine to lua it didn't get the strings loaded before
but I already made it and now there are only little bugs to fix.

 

[Teresaty]

thanks, nice Jomex, and

@ShadowDragon42 Look MP

 

[Jomex]

Okey so I upgraded the Lua from 5.1 to 5.2 and it reads faster than anything.

 

[Langstra]

Can you tell us how much efficiency you gained from upgrading to Lua 5.2? Loading times I suggest.