Looks like someone is already ahead of us.

[Yamachi]

XeNTaX • View topic - RaiderZ filesystem

I guess I can throw my code away...

[MentaL]

[Kyhoh]

Well he/she started reversing very long ago, look at the date ^^.

[Yamachi]

Exactly my point.

[Dell Honne]

His files are most likely out of date since that post was 2009...

I believe the files Dawson released are taken straight from PWE's server.

[x1nixmzeng]

That's my thread!

It was written after the second official CBT revealed new formats, but I didn't get much further - don't let it put you off!

There's some algorithm which actually decompresses the data I haven't figured out, but I generate the xor table on command (something these new client files do too).

Send me a PM or add my MSN. I'd like to get an IRC group setup around Raiderz development (who can later work on GunZ 2).

[Dell Honne]

That's my thread!

It was written after the second official CBT revealed new formats, but I didn't get much further - don't let it put you off!

There's some algorithm which actually decompresses the data I haven't figured out, but I generate the xor table on command (something these new client files do too).

Send me a PM or add my MSN. I'd like to get an IRC group setup around Raiderz development (who can later work on GunZ 2).

Add me or Dawson on MSN.

anime_master2@live.ca = me

[Yamachi]

michael@ownztheinter.net

[Kyhoh]

Most of the data are now stored on .mrf and .00x files.

So if those files structures are know game can be
edited in many ways.

I have been looking a little for .mrf files. For example system.mrf.

0013FB34   7C801A53  /CALL to CreateFileW from kernel32.7C801A4E
0013FB38   7FFDFC00  |FileName = "Data/system.mrf"
0013FB3C   80000000  |Access = GENERIC_READ
0013FB40   00000001  |ShareMode = FILE_SHARE_READ
0013FB44   00000000  |pSecurity = NULL
0013FB48   00000003  |Mode = OPEN_EXISTING
0013FB4C   00000080  |Attributes = NORMAL
0013FB50   00000000  \hTemplateFile = NULL
0013FB54   0255C170  ASCII "data/system/player_model.xml"

It seems that many of the data inside .mrf are .xml files.

After ReadFile here inside ESP+30, it's the .xml file.

00631438  |> \B0 01         MOV AL,1

player_model.xml

 <?xml version="1.0" encoding="UTF-8" ?> 
- <maiet>
- <!--  Male 
  --> 
  <DefaultModelPath sex="male">data/model/player/hm/hm.elu</DefaultModelPath> 
  <DefaultMesh sex="male" parts="hat" /> 
  <DefaultMesh sex="male" parts="chest" /> 
  <DefaultMesh sex="male" parts="hands" /> 
  <DefaultMesh sex="male" parts="legs" /> 
  <DefaultMesh sex="male" parts="feet" /> 
- <Hair sex="male" count="14">
  <HairModel id="1">hm_hair_16</HairModel> 
  <HairModel id="2">hm_hair_02</HairModel> 
  <HairModel id="3">hm_hair_13</HairModel> 
  <HairModel id="4">hm_hair_14</HairModel> 
  <HairModel id="5">hm_hair_05</HairModel> 
  <HairModel id="6">hm_hair_04</HairModel> 
  <HairModel id="7">hm_hair_07</HairModel> 
  <HairModel id="8">hm_hair_11</HairModel> 
  <HairModel id="9">hm_hair_17</HairModel> 
  <HairModel id="10">hm_hair_15</HairModel> 
  <HairModel id="11">hm_hair_12</HairModel> 
  <HairModel id="12">hm_hair_01</HairModel> 
  <HairModel id="13">hm_hair_18</HairModel> 
  <HairModel id="14">hm_hair_03</HairModel> 
  </Hair>
- <Face sex="male" count="12">
  <FaceModel id="1">hm_face_12</FaceModel> 
  <FaceModel id="2">hm_face_10</FaceModel> 
  <FaceModel id="3">hm_face_03</FaceModel> 
  <FaceModel id="4">hm_face_04</FaceModel> 
  <FaceModel id="5">hm_face_05</FaceModel> 
  <FaceModel id="6">hm_face_06</FaceModel> 
  <FaceModel id="7">hm_face_07</FaceModel> 
  <FaceModel id="8">hm_face_08</FaceModel> 
  <FaceModel id="9">hm_face_09</FaceModel> 
  <FaceModel id="10">hm_face_11</FaceModel> 
  <FaceModel id="11">hm_face_01</FaceModel> 
  <FaceModel id="12">hm_face_12_3</FaceModel> 
  </Face>
- <!--  Female 
  --> 
  <DefaultModelPath sex="female">data/model/player/hf/hf.elu</DefaultModelPath> 
  <DefaultMesh sex="female" parts="hat" /> 
  <DefaultMesh sex="female" parts="chest" /> 
  <DefaultMesh sex="female" parts="hands" /> 
  <DefaultMesh sex="female" parts="legs" /> 
  <DefaultMesh sex="female" parts="feet" /> 
- <Hair sex="female" count="14">
  <HairModel id="1">hf_hair_01</HairModel> 
  <HairModel id="2">hf_hair_02</HairModel> 
  <HairModel id="3">hf_hair_03</HairModel> 
  <HairModel id="4">hf_hair_04</HairModel> 
  <HairModel id="5">hf_hair_05</HairModel> 
  <HairModel id="6">hf_hair_12</HairModel> 
  <HairModel id="7">hf_hair_07</HairModel> 
  <HairModel id="8">hf_hair_14</HairModel> 
  <HairModel id="9">hf_hair_13</HairModel> 
  <HairModel id="10">hf_hair_15</HairModel> 
  <HairModel id="11">hf_hair_11</HairModel> 
  <HairModel id="12">hf_hair_17</HairModel> 
  <HairModel id="13">hf_hair_18</HairModel> 
  <HairModel id="14">hf_hair_19</HairModel> 
  </Hair>
- <Face sex="female" count="12">
  <FaceModel id="1">hf_face_01</FaceModel> 
  <FaceModel id="2">hf_face_02</FaceModel> 
  <FaceModel id="3">hf_face_03</FaceModel> 
  <FaceModel id="4">hf_face_12</FaceModel> 
  <FaceModel id="5">hf_face_05</FaceModel> 
  <FaceModel id="6">hf_face_06</FaceModel> 
  <FaceModel id="7">hf_face_07</FaceModel> 
  <FaceModel id="8">hf_face_08</FaceModel> 
  <FaceModel id="9">hf_face_09</FaceModel> 
  <FaceModel id="10">hf_face_10</FaceModel> 
  <FaceModel id="11">hf_face_11</FaceModel> 
  <FaceModel id="12">hf_face_04</FaceModel> 
  </Face>
  </maiet>

So the dec routine should be near.

And yeah it'll be cool an IRC group :D.

Edit---------------------------

Can be this the xor you're talking?

00630E60  /$  56            PUSH ESI
00630E61  |.  8BF1          MOV ESI,ECX
00630E63  |.  837E 04 00    CMP DWORD PTR DS:[ESI+4],0
00630E67  |.  75 06         JNZ SHORT Raiderz.00630E6F
00630E69  |.  32C0          XOR AL,AL
00630E6B  |.  5E            POP ESI
00630E6C  |.  C2 0800       RETN 8
00630E6F  |>  8B5424 08     MOV EDX,DWORD PTR SS:[ESP+8]
00630E73  |.  57            PUSH EDI
00630E74  |.  8B7C24 10     MOV EDI,DWORD PTR SS:[ESP+10]
00630E78  |.  8D47 FF       LEA EAX,DWORD PTR DS:[EDI-1]
00630E7B  |.  85C0          TEST EAX,EAX
00630E7D  |.  76 20         JBE SHORT Raiderz.00630E9F
00630E7F  |.  53            PUSH EBX
00630E80  |.  55            PUSH EBP
00630E81  |>  8A4C10 FF     /MOV CL,BYTE PTR DS:[EAX+EDX-1]
00630E85  |.  8B6E 04       |MOV EBP,DWORD PTR DS:[ESI+4]
00630E88  |.  8BD8          |MOV EBX,EAX
00630E8A  |.  D0E9          |SHR CL,1
00630E8C  |.  81E3 FFFF0000 |AND EBX,0FFFF
00630E92  |.  320C2B        |XOR CL,BYTE PTR DS:[EBX+EBP]
00630E95  |.  300C10        |XOR BYTE PTR DS:[EAX+EDX],CL
00630E98  |.  83E8 01       |SUB EAX,1
00630E9B  |.^ 75 E4         \JNZ SHORT Raiderz.00630E81
00630E9D  |.  5D            POP EBP
00630E9E  |.  5B            POP EBX
00630E9F  |>  8A443A FF     MOV AL,BYTE PTR DS:[EDX+EDI-1]
00630EA3  |.  8B4E 04       MOV ECX,DWORD PTR DS:[ESI+4]
00630EA6  |.  D0E8          SHR AL,1
00630EA8  |.  3201          XOR AL,BYTE PTR DS:[ECX]
00630EAA  |.  5F            POP EDI
00630EAB  |.  3002          XOR BYTE PTR DS:[EDX],AL
00630EAD  |.  B0 01         MOV AL,1
00630EAF  |.  5E            POP ESI
00630EB0  \.  C2 0800       RETN 8

[x1nixmzeng]

I'm on irc.esper.net as x1nixmzeng for now.


I'll try explaining again.

• MSF files are the individual files. They've been compressed and scrambled (with this xor thing I keep mentioning).
• MRF files are just MSF files which have been merged together.

In order to unpack everything in a MRF, the fileindex has to be read, because it lists the filenames, positions and sizes of the MSF files (which need to be unscrambled and uncompressed to be read).



Now that function you posted with the xor lines IS the unscrambling thing I mean. It uses a large table which is regenerated quite a lot. I've ripped out this part of the unpacking, but not the decompression (because it's HUGE).

Edit

Here's reading fileindex.msf:



That "xorTable" function translates roughly to:

bool xorData(char *data, int size)
{
        if(!m_xorTable)
        {
                return false;
        }

        for(unsigned long eax = size-1; eax > 0; eax--)
        {
                // MOV CL,BYTE PTR DS:[EAX+EDX-1]
                unsigned char cl = data[eax-1];

                //MOV EBP,DWORD PTR DS:[ESI+4]
                // ebp = m_xorTable

                // SHR CL,1
                cl >>= 1;

                //unsigned long ebx = eax; // MOV EBX,EAX
                //ebx &= 0xFFFF; // AND EBX,0FFFF

                cl ^= m_xorTable[eax & 0xFFFF]; // XOR CL,BYTE PTR DS:[EBX+EBP]

                //unsigned char edx = data[eax] ^ cl; // XOR BYTE PTR DS:[EAX+EDX],CL

                data[eax] ^= cl;
        }


//00567DFF  |> 8A443A FF      MOV AL,BYTE PTR DS:[EDX+EDI-1]
        unsigned char al = data[size-1]; // ??

//00567E03  |. 8B4E 04        MOV ECX,DWORD PTR DS:[ESI+4]
        // ecx = m_xorTable

//00567E06  |. D0E8           SHR AL,1
        al >>= 1; // SHR CL, 1

//00567E08  |. 3201           XOR AL,BYTE PTR DS:[ECX]
        al ^= m_xorTable[0];

//00567E0B  |. 3002           XOR BYTE PTR DS:[EDX],AL
        data[0] ^= al;


        // Simplified
        //data[0] ^= ((data[size-1] >> 1) ^ m_xorTable[0]);


        return true;
}

And "m_xorTable" (as I keep calling it) is this large lookup table which is regenerated works like this:

char *m_xorTable = NULL;

const unsigned long MAGIC = 0x85F24C5A;

bool GenerateTable()
{
        m_xorTable = (char *)malloc(0x10000);

        if(!m_xorTable)
                return false;

        __asm
        {
                PUSH EDX;
                PUSH ESI;
                PUSH EAX;
                PUSH ECX;

                // Loop counter
                xor EDX, EDX;

        fillBuffer:

                LEA ESI,DWORD PTR DS:[EDX+1];
                MOV EAX,ESI;
                AND EAX,1;
                DEC EAX;
                NOT EAX;

                AND EAX, MAGIC;

                MOV ECX,ESI;
                SHR ECX,1;
                XOR EAX,ECX;
                MOV ECX,EAX;
                AND ECX,1;
                DEC ECX;
                NOT ECX;
                SHR EAX,1;

                AND ECX, MAGIC;

                XOR ECX,EAX;
                MOV EAX,ECX;
                AND EAX,1;
                DEC EAX;
                SHR ECX,1;
                NOT EAX;

                AND EAX, MAGIC;

                XOR EAX,ECX;
                MOV ECX,EAX;
                AND ECX,1;
                DEC ECX;
                NOT ECX;
                SHR EAX,1;

                AND ECX, MAGIC;

                XOR ECX,EAX;
                MOV EAX,ECX;
                AND EAX,1;
                DEC EAX;
                SHR ECX,1;
                NOT EAX;

                AND EAX,MAGIC;

                XOR EAX,ECX;
                MOV ECX,EAX;
                AND ECX,1;
                DEC ECX;
                NOT ECX;
                SHR EAX,1;

                AND ECX, MAGIC;

                XOR ECX,EAX;
                MOV EAX,ECX;
                AND EAX,1;
                DEC EAX;
                SHR ECX,1;
                NOT EAX;

                AND EAX, MAGIC;

                XOR EAX,ECX;
                MOV ECX,EAX;
                AND ECX,1;
                DEC ECX;
                NOT ECX;
                SHR EAX,1;

                AND ECX, MAGIC;

                XOR ECX,EAX;

                // Set EAX to the saved buffer pointer (or more directly here w m_xorTable)
                MOV EAX, [m_xorTable];

                // Move the lower-byte value of ECX into buffer + pos
                MOV BYTE PTR DS:[EDX+EAX],CL

                MOV EDX,ESI;
                CMP EDX,0x10000;

                // Loop until EDX <= 0xFFFF (size of buffer)
                JB fillBuffer;

                POP ECX;
                POP EAX;
                POP ESI;
                POP EDX;

        }

        return true;
}

Wrap this into a program and you're as far as me. Doubt I can rip the uncompression as easily as I'd like.

[ThePhailure772]

LZMA SDK (Software Development Kit)

there's your compression / decompression.

[Kyhoh]

Mmm I think the first parameter with ? to uncompression can be a DWORD(4 bytes) just for store the result of the CALL.





So if the uncompression fails it puts 0 and if not 1.

--

The other parameters seems like more complex, it's like
a buffer with many bytes and the first 3 DWORDs are generated
like this (pseudocode):

number = 5
constant = 8F7528
-------------------

if number < 5
	return 4

result1 = ([constant + 4] << 8) + [constant+3]
result2 = (result1 << 8) + [constant+2]
result3 = (result2 << 8) + [constant+1]

if result3 < $1000
	result3 = $1000

(buffer+$C) = result3

if ([constant] >= $0E1)
	return 4
else
	tmp = [constant]
	[buffer] = tmp % 9 //Set the 3
	v6 = tmp / 9
	[buffer+8] = v6 / 5
	[buffer+4] = v6 % 5
	return 0

But no idea for what is really used inside uncompression.

---------- Post added at 03:51 PM ---------- Previous post was at 03:38 PM ----------

LZMA SDK (Software Development Kit)

there's your compression / decompression.

Maybe you're right I am checking the code and there are some
resemblances between my pseudocode and this:

	public boolean SetDecoderProperties(byte[] properties)
	{
		if (properties.length < 5)
			return false;
		int val = properties[0] & 0xFF;
		int lc = val % 9;
		int remainder = val / 9;
		int lp = remainder % 5;
		int pb = remainder / 5;
	}

[Jesemos]

Could you guys try to unpack the PWE files? It's possible by using the same method I think.

http://www.2shared.com/file/R5QnAJw2/fileindex.html
http://www.2shared.com/file/1wdyHjjy/Raiderz.html
http://www.2shared.com/file/8VcIxPRv/system.html

These files are from the Alpha Test of RaiderZ PWE.