The last hurdle - bof compiler

[Darq]

So now, with the latest release we have editors and compilers for

rsc/rsb
bgf
roo

the only tool missing is a working bof compiler
and eventually an uml editor export plugin

MM claimed he knew much about bofs

 

[MuteX]

The holy MM. lol

I doubt that he knows much of BOFs. Maybe he's able to change some values to make weapons stronger or something like that. But I highly bet he wouldn't be able to create a complete new BOF from scratch with new functionality.
I'm working a bit on BOFs but haven't much time currently. Also Albo is on them - and will (if he still works on BOFs) finish his work definitely earlier than me.

 

[Darq]

gods don't lie , do they ;P

i'm not doubting alba's abitily, but his motivation
still, this thread could be used to post infos about bof
i'm not sure if the srv doesn't have this feature already builtin.
./loadkod

 

[Albo]

Well, I have a bytecode-level assembler that produces BOFs, but I don't want to continue with it at the moment.

 

[Chelsea]

anyone have BOF editor?
i know that BOF's were edited by the hex editor
so good luck everyone

 

[madmonkee]

I've never made a BOF editor. I always used hex editor to do my stuff. I kind of quit Meridian... I got college and work to deal with, I truly don't have much time in my hands nowdays. poop I haven't looked into this for a loong time... uhm, I'll put together what I know, it's really easier than it seems though fellas. My two brothers still want to do meridian but I don't want to deal with "server is down, this is broken, do this, do that" poop frankly.

How I was able to do all the poop I've done on 911 was simple. I eliminated the need for BOFs

Those who played on 911 will remember MMUtils, where you "said" a command and it did something.

I made a program which everyone connected to, and the program connected everybody to the server, monitoring packets. I was able to stop people from TTGing, Speedcheating, and write my own scripts corresponding to packets which are passed through.

I charted out about 80% of ALL meridian packets, what is what and which packet is which and wrote my scripting language which I called "MM Util"

It was sort of like;

Const PCK_REST = "05"
Const PCK_STAND = "06"
Const PCK_MOVE = "64"

if
{
packetid (PCK_MOVE) is detected
get user position (row col frow fcol)
set user position (row + 5 col frow fcol)
**

and the actual move packet ends up looking like this

0A 00 7D 1C 0A 00 9E 64 12 0B AA 05 24 F2 4F 02 00

where

0A 00 is size of the data + function id
7D 1C is CRC
0A 00 is the size again
9E is the saveid (everytime system saves this ticks up a notch)
64 is the function id
12 0B rows
AA 05 cols
24 means running 12 means walking
F2 4F 02 00 is the NDS representation of the objects. Object F24F of set 2

And well, you get the idea.

And with the CRC from server to client figured, I wouldn't even need BOFs, taking the load off the server program so the object numbers also stay low.

 

[madmonkee]

Taking a look at APPLE.BOF

424F 46FF = BOF which is always constant
0500 0000 = 5 which I figure is the BOF version always constant
B000 0000 = is the offset to the end of file

uhmm... lemme remember a couple more

 

[madmonkee]

A800 0000 = is the offset to sector 1
AC00 0000 = is the offset to sector 2
0100 0000 = is constant
C229 0000 = is the class ID
2000 0000 = is constant
812D 0000 = is the class parent ID

I'll write more as I remember ... it's been a while

 

[madmonkee]

0A00 0000 = 10 class variables

(variableID value type)

2B00 0000 0700 00D0
2A00 0000 0800 00D0
0000 0000 115F 0040
0100 0000 125F 0040
0200 0000 135F 0040
2800 0000 145F 0040
0300 0000 0300 0010
0500 0000 0600 0010
0600 0000 0600 0010
1200 0000 1900 0010

the type 00D0 basically means *overwritable*

You declare the object variable as the class's ID. than you can change the actual class variable inside the game. You can declare IDs for parent class variables aswell, quite useful.

0300 0000 = object variables

0700 0000 0A00 0010
0800 0000 1800 0010
0600 0000 0200 0010

The next block would be the class messages defined but apple has none.

 

[Albo]

The BOF format is completely figured out and there is a working but very basic assembler. There is no linking yet, though (i.e. if you create a class in one source file you cannot reference it by name in another). Since typing numbers is annoying I consider that unfinished software and I won't release it until I devise a linker (or abandon Meridian completely).

In the meantime you might be interested to know that packets are constructed using BOF code. For instance, there is some code in a BOF somewhere that may cause the 65536-object bug by deliberately writing a 2-byte field to a packet.

It's kind of evil but I will post a code example. This is a list reverse function. It makes sure that the argument is indeed a list, then reverses the argument in the loop. First the relevant part of kodbase.txt:

C Test 20000 0
M   Constructor 10
...
M   SmallList 30002
...
M   ListRevIter 30004
P       src 40002
...

message 30004 2
40002 $n0
    isl     #l0 -> #l1
    jnz     inputok, #l1
    ret     #i$n0
inputok:
    mov     #l1, #i$n0
loop:
    eq      #l2, #l0, #i$n0
    jnz     done, #l2
    hd      #l0 -> #l2
    tl      #l0 -> #l0
    cons    #l2, #l1 -> #l1
    jmp     loop
done:
    ret     #l1

And here's an example session at the admin console:

create o Test
Created object 3855.
send o 3855 SmallList
:< return from OBJECT 3855 MESSAGE SmallList (30002)
: LIST 53632
:>
show list 53632
[ INT 1, INT 2, INT 3 ] 
send o 3855 ListRevIter src list 53632
:< return from OBJECT 3855 MESSAGE ListRevIter (30004)
: LIST 53635
:>
show list 53635
[ INT 3, INT 2, INT 1 ]

 

[madmonkee]

That's sexy, but wouldn't it be a witch to code straight up assembly for every single bof? Can you release the compiler? I am writing an interface for your magnificent roowrite.exe, perhaps I could write an easier, more user friendly scripting language for the bof compiler? What assembly is this btw? I am familiar with wasm, I don't recognize isl, hd and tl instructions though.

I wish you would have told me all this long ago <:\ I would have loved to do my own bofs and roos when we had 911 and still were in "good relations" with 0 and QS.

Amazing poop man, I was sittin here thinking no one knew this poop when it was Albo all along. poop we could have made a server with you back in the day, but then again I was a poop head, life taught me a lot since then

 

[Albo]

It's not very hard to code "assembly" for every message as this is some very high-level assembly. This is an assembly language specifically designed for the instructions understood by the Meridian interpreter, so it's no surprise you haven't seen them before.

With regards to a higher-than-assembly-level-language, I plan to create one for you. The assembler comes first, though, as a back end, and for testing. Don't get too carried away The reason I didn't write a full-blown ROO editor is because the amount of effort required for a quality editor is way out of proportion with the number of people that could possibly care about it. That's not the case with BOF (unless you also want an IDE for writing BOFs).

EDIT: Another reason for having an assembler is that your new BOFs will probably have to work with existing ones, which you may want to modify. You may also want to reverse-engineer them for other reasons. For that I will provide a disassembler. With an assembler, you won't have to convert the result to high-level code to make your changes.

 

[madmonkee]

So you sat down and tested to figure out what each byte instructs to the meridian interpreter? That's some impressive poop wigga, I won't need the 911 tool after that sucka is released.

Hey man, I've also been meaning to ask you. Have you ever done a program where you input a roo and it spits out your text format? There are some rooms I wish to slightly change I don't want to build them from 0 ... Or maybe you can release the roo file structure or something, I long lost all the info I had on it way back when it was being discussed on pk-hq.

Oh, and did you ever look into this thread

Maybe the CRC calculation of the function byte from server to client is there somewhere in the BOFs if not an internal function in the exe. You did figure out the ROO checksum and security number, figured yer the best man to ask this. And yes, I am humbly asking for help on something, I am not the egotistical bastard I used to be, working for my own bills have taught me a lot

 

[Albo]

Someone is working on a program that does exactly what you want. Maybe he will post it when he's done. If not, I will create such a program - it's really simple. Either that or someone might PM me the specs from PK-HQ days, because my copy is very messy and even I don't understand what it says anymore. I will update the relevant parts of the specs and post them/PM them for the sender to post.

Note, however, that ROO is a bad format for storing rooms that you want to edit later. A program like this is only useful when you don't have the source data available, which is the case with all the "stock" rooms right now (actually, the source data might be in ROO but reversing it would be rather hard).

I'm fairly sure the 'CRC' is computed by the native code in the server (not in any BOF).

So you sat down and tested to figure out what each byte instructs to the meridian interpreter?

No, I didn't test anything at all until some time after I had everything figured out It really took about 30 minutes to figure out almost all of the bytecode, but then I spent 3 days verifying it to my satisfaction and determining facts that are hard to see given the bytecode alone (I mean the semantics). Then I started writing that assembler. Another 2 days later I tested my first code.

 

[madmonkee]

Cool dude, meridian started to sound fun again. The roo visual thing I was writing for your proggy began to seem harder than I thought, the 3d test render part at least. As you mentioned I seem to be the only one who is dead hyped about making new roos anyway so I doubt anyone cares when I release it. Do ya think at some point you can look at blakserv.exe and see where it calculates the CRC for the function id part on packets? I know it's pointless but Darq doesnt wanna help, and you are the only other person I know who is familiar with assembly AND meridian Ill do the same when I get home.

 

[gaaara]

Cool dude, meridian started to sound fun again. The roo visual thing I was writing for your proggy began to seem harder than I thought, the 3d test render part at least. As you mentioned I seem to be the only one who is dead hyped about making new roos anyway so I doubt anyone cares when I release it. Do ya think at some point you can look at blakserv.exe and see where it calculates the CRC for the function id part on packets? I know it's pointless but Darq doesnt wanna help, and you are the only other person I know who is familiar with assembly AND meridian Ill do the same when I get home.

what exactly do you mean by " ... where it calculates the CRC for the function id part on packets..."?
as far as i know is there only crc in the packets?

 

[Clickslither]

Just a side note, I'm glad to see madmonkee gaining some spirit in meridian department again. All this work might just spawn a damn sexy freeserver with an actual population.

 

[madmonkee]

what exactly do you mean by " ... where it calculates the CRC for the function id part on packets..."?
as far as i know is there only crc in the packets?

That is the case for all the packets going from client to server.

simple packet structure is like so;

SIZE (2 bytes) CRC (2 bytes) SIZE (2 bytes) SaveID (1 byte) FunctionID (1 byte) Data

the Size is data + function ID

here's an example packet;

01 00 FF FF 01 00 03 06

This tells the server to do function 6 on the client, which if I remember correctly was "Space" key.

03 Means server has saved 3 times since it was up. Everytime the system saves, this number goes up a notch.

Now, I don't know how to calculate the CRC for Any of the packets however, you can stop the server from verifying client side packets by setting the server side config value "Hangup Spoofs" to false. Hence I was able to write 911 tool which made all the sexy poop 911 had like silence, free action, eagle eyes, da rescue... poop I even made the DA rooms system but I never implemented it because we took the server down that week.

Now the server does this about the same, except

the Function ID is tied to the CRC's calculation somehow.

In the assembly I could figure out that a function which was not defined in the original CRUSH.H documentation was used which is called cx_ComputeCRC32Block.

Not ( cx_ComputeCRC32Block ( functionID + Data, SizeOf (functionID + Data) ) )

And the CRC is the first two bytes of the result in hex.

Now after this, Something happens to the function ID aswell, and it has to do with the CRC I am 100% positive. Because every once in a blue moon the CRC is same on an identical packet, the function IDs are same aswell.

So maybe it's Last byte of the CRC number added to function ID or what, I have no fuggin clue, and that's what I need help with.

Now on client to server side CRC is not calculated by the same function. cx_ComputeCRC32Block is not even declared in the EXE. But since I could shut down the checking of that, I never sat down to figuring that out.

I just know it was another function called cxBuf2BufCRC or something like that, but I never looked into usage of it either, it's in the default CRUSH.H

Any help by anyone is appreciated, I am going through REAAALY difficult times in my life right now, so I don't have much time to myself. The only free times I have I am working on these nowdays.

 

[gaaara]

could you provide some packets where the crc is unclear? ... for checking/testing purpose .... never took a deeper look at it.

 

[madmonkee]

This packet is sent from the Server to the Client stating that user said "a"


11 00 C1 F9 11 00 01 4A C4 22 00 00 B6 3A 00 00 01 1E 59 00 00 01 00 61

11 00 = Size

C1 F9 = CRC

11 00 = Size

01 = Save ID (Fresh Server)

4A = Function ID (In this case, SomeoneSaid message, that could be a clue)

C4 22 00 00 = Object Number of my user

B6 3A 00 00 = Resource Number of the user's name

01 = Communication Type (1 being "say")

1E 59 00 00 = Resource 22814 user_said_echo_str = 'You say, "%q~n"'

01 00 = Size of string, which is 1 characters long

And finally

61 = "a" which is what the user said

____________________________________________________

Now here are different CRC variations of the same packet.

11 00 3E DB 11 00 01 3F C4 22 00 00 B6 3A 00 00 01 1E 59 00 00 01 00 61
11 00 43 18 11 00 01 82 C4 22 00 00 B6 3A 00 00 01 1E 59 00 00 01 00 61
11 00 8C 3F 11 00 01 86 C4 22 00 00 B6 3A 00 00 01 1E 59 00 00 01 00 61
11 00 78 00 11 00 01 6B C4 22 00 00 B6 3A 00 00 01 1E 59 00 00 01 00 61
11 00 61 BE 11 00 01 FE C4 22 00 00 B6 3A 00 00 01 1E 59 00 00 01 00 61

As we can see, the CRC and function IDs are always different, well not always. If you manage to test sending and studying the same function over and over at some point CRC does come to the same thing, and when that happens, the function ID is also exactly the same.

3F, 82, 86, 6B, FE all calculate to the same number with some sort of equation taken from 3E DB, 43 18, 8C 3F, 78 00, 61 BE ...

Having full control over the packets and the CRC will give me the ability to control new objects which don't really exist according to the server, but client will still think it does because it fully gets to interact with it. This gives me the ability to create whatever I want.

Here's another thing I discovered. You can edit the packet contents and client doesn't really care, still verifies it. You can change "a" to "b" and send it to the client it will say, "You said, "b"" when you really typed a

So the CRC must only be a security check on the function ID and that only is my assumption, because client clearly doesn't log the user off unless you change the CRC or the Function ID. Any help would be appreciated.