[WIP] Launcher

[Ryfon]

I've been messing with the launcher to figure out how it works.
so far my findings are:

Directory structure:
{Domain}/CompressionGameData/version.txt
{Domain}/{version}/_GameDataFileList.txt
{Domain}/{version}/_GameDataTranslateFileList.txt
{Domain}/{version}/delfilelist.txt
{Domain}/{version}/_{Client Files}

all files beginning with an _ are compressed in a similar way to the IDX/PKG except in a single file format
more research into this is needed.

the IDX/PKG is as follows. though the _ are missing the INDEX,OFFSET,SECSTART,PKG #. (probably because its a single file) the compressed data always starts at 0x0374 too

DUMMY 260
HEADER 32
INDEX long
OFFSET long
SECSTART long
ZSIZE long
CRC long
DUMMY long
VERSION long
DUMMY long
CREATED_DATE long
DUMMY long
ACCESSED_DATE long
DUMMY long
MODIFIED_DATE long
DUMMY long
SIZE long
NAME 260
DIR 260
DUMMY long
PKG # long
DUMMY long

GameDataFileList.txt Contains:

21098 (file count)
ALAudio.dll:20150227105830:2107392:1619121554
...... all client files ......

file name : exact file modified date : file size : DecCRC

Edit: update thanks to YesOfCourse
Edit 2: compressedChecksum is wrong. its just a Dec CRC.
Edit 3: Header Is figured out now though I'm unsure as to how they are converting the DateTime to a long. made a beta program to output the compressed files but the launcher eats the last 3712 bytes causing an error on decompress.

 

[Eperty123]

Once we find out how the structure is we will be able to create our own custom launcher with automatic patcher. Great thread!
Also you can decompress the _ files using offzip. I've tried extracting and it worked, try it out! Though I can't figure out how to compress files.

Edit: I guess we can use zlib to decompress and compress as PKG unpacker uses it.

 

[YesOfCourse]

The only missing thing we have to find out is the way they calculate checksum for the compressed files. It was long ago since I messed with the files so I am not entirely sure if I just missed something but from what I remember the update files crc are calculated with a custom table/key.

filePath:lastModifyDate:compressedSize:[COLOR=#ff0000][B]compressedChecksum[/B][/COLOR]

If the checksum is generated correctly the launcher will fully work.

I will look into this as well once I can get up a test environment.

 

[Eperty123]

The only missing thing we have to find out is the way they calculate checksum for the compressed files. It was long ago since I messed with the files so I am not entirely sure if I just missed something but from what I remember the update files crc are calculated with a custom table/key.

filePath:lastModifyDate:compressedSize:[COLOR=#ff0000][B]compressedChecksum[/B][/COLOR]

If the checksum is generated correctly the launcher will fully work.

I will look into this as well once I can get up a test environment.

So you managed to extract and recompress the files?

 

[YesOfCourse]

So you managed to extract and recompress the files?

Yes sure, they follow the same format like the PKG files (well mostly the same). However I just dropped together some php codes for it so nothing special.

Edit:

GameDataFileList.txt Contains:

21098 (file Version I'm guessing)
ALAudio.dll:20150227105830:2107392:1619121554

That one is the total file count of files in the list/client (which are compressed in the pkg)

 

[Eperty123]

Yes sure, they follow the same format like the PKG files (well mostly the same). However I just dropped together some php codes for it so nothing special.

Edit:



That one is the total file count of files in the list/client (which are compressed in the pkg)

Cool! Mind giving a tut on it? Like compressing other files and such?

 

[Ryfon]

@YesOfCourse

what's the Unk 28 I'm missing? I've been messing around with them and can't seem to make any sense out of them. some times the three sections are the exact same other times only 2, I've been beating my head on this for the past few days. its also the last bit of info needed to making a decent packager for the files

 

[Ryfon]

You must be registered to see links

made a Program to generate GameDataFileList.txt

Now just need to figure out the rest of the header on compressed files

 

[Ryfon]

You must be registered to see links

tool to compress update files.

everything is figured out now except the crc32 settings. anyone know of any tools to bruteforce these?
as soon as i figure out the settings I'll update both programs.

 

[Eperty123]

You must be registered to see links

tool to compress update files.

everything is figured out now except the crc32 settings. anyone know of any tools to bruteforce these?
as soon as i figure out the settings I'll update both programs.

Thank you for your hard work Ryfon! I was wondering if you could make it possible to create a _GameDataFileList.txt after the files get compressed that includes the crc, so it's ready to go when done?

 

[Ryfon]

yep just need the crc32 polynomial, initial value, output XOR, and reflection in/out. which is just down to me entering in different values till i hit a match. this will go much faster if i can find a tool that already made for this but all i can find are crc16 tools. so i may just have to make one myself

 

[Ryfon]

Anyone that wants to Donate some CPU cycles to cracking the crc32 on this PM me.

 

[Eperty123]

Look at this.

 

[Ryfon]

yah externally you can have many crc values match but the way the client gets its crc is different. the game.bin crc needs to be 117021CA if you have the correct checksum method.




This is how you calculate a crc value.

class CRC
    {
        long[] pTable = new long[256];
        long Poly = 0xEDB88320;

        public CRC()
        {
            long CRC;
            int i, j;

            for (i = 0; i < 256; i++)
            {
                CRC = i;

                for (j = 0; j < 8; j++)
                {
                    if ((CRC & 0x1) == 1)
                    {
                        CRC = (CRC >> 1) ^ Poly;
                    }
                    else
                    {
                        CRC = (CRC >> 1);
                    }
                }
                pTable[i] = CRC;
            }

        }

        public uint GetCRC32(string FileName)
        {
            long StreamLength, CRC;
            int BufferSize;
            byte[] Buffer;

            //4KB Buffer
            BufferSize = 0x1000;

            FileStream fs = new FileStream(FileName, FileMode.Open);
            StreamLength = fs.Length;

            CRC = 0xFFFFFFFF;
            while (StreamLength > 0)
            {
                if (StreamLength < BufferSize)
                {
                    BufferSize = (int)StreamLength;
                }
                Buffer = new byte[BufferSize];

                fs.Read(Buffer, 0, BufferSize);

                for (int i = 0; i < BufferSize; i++)
                {
                    CRC = ((CRC & 0xFFFFFF00) / 0x100) & 0xFFFFFF ^ pTable[Buffer[i] ^ CRC & 0xFF];
                }

                StreamLength = StreamLength - BufferSize;

            }
            fs.Dispose();
            fs.Close();
            CRC = (-(CRC)) - 1; // !(CRC)

            return (uint)CRC;
        }

    }

now CRC and Poly can be any value and it will change the value you get, there are standard numbers used generally because they are the most effective methods. you can also have input and output reflections increasing the possibility s more. you could also just make a custom method all together but this is doubtful as generally programmers don't go out to reinvent something that already works really well (atleast not in a professional setting, at home you can get all kinds of crazy ideas

 

[nofxpunkerbrian]

Do you happen to know if the CRC value is used for the launcher only or does the client calculate it too?

I have tried the above script I keep getting the same value for any file instead of what it states in the idx or is there further calculations after?

 

[Ryfon]

CRC is calculated by launcher and client so they have to be correct for both. If you would like to know more about it and my script above pm me

 

[dTantra]

CRC is calculated by launcher and client so they have to be correct for both. If you would like to know more about it and my script above pm me

Just going to say from what I see the actual values do not mater, if you were to use a custom crc method as long as its used everywhere it would not mater. I see no actual crc calculation in the game exe.

 

[Ryfon]

You should look again because it is

 

[dTantra]

You should look again because it is.

Mind pointing me to where? I have downloaded patches using the launcher and modified the CRC in GameDataFileList.txt as well as in the file itself and had no ill effects or rejection messages. Also I have made changes to files in the pack itself without updating the CRC and the game accepts them just fine. This post is in no means hostile, I am just trying to better understand things, so feel free to show me where I am wrong.

 

[nofxpunkerbrian]

I've also modified files and added new files with random values for the CRC and had no effect in game it actually loads the files fine so I am not sure if it's necessary.