New DataCenter parser

[Gothos]

I have rewritten the DCTools parser. The new version has a few advantages over the old one:

• It doesn't require the user to hardcode section offsets and figures them out by itself. Simply point it at an unpacked data center.
• It parses the whole DataCenter in 5-10 seconds
• It preserves type information, instead of turning ints/bools/floats into strings that need to be parsed again.

Added generic dumpers for Json and XML, which produce the direct equivalent of the original DataCenter contents in these formats.

I also updated the existing Data Tools a bit, starting from coolyt's variant:

• It only contained the binary version of the Tera emulator classes, replaced that with source code taken from P5yl0's github.
• Replaced a few enums with strings, where it made sense. (e.g. NpcTitle)
• Replaced a bunch of ints with floats, where the data center uses floats
• Updated several of the parsers to work with current data, adding classes, skill-triggers, etc. and fixed handling a bunch of fields which are optional in recent versions

Decrypter / Unpacker:

• Decrypt and decompress a data center file given key and IV
• Extract the key/iv from a running Tera instance
• Outputs an info file which contains relevant information about a data center, including revision, key/iv, file name, file sizes and SHA-256 hashes.

Source:

You must be registered to see links

Binaries:

You must be registered to see links

 

[kinvao]

I have two things to say about your parser but first of all...great work have you done there
The first is the parser loads a lot the RAM... I dont think it really needs 2GB of ram to work. I know there is a lot of data but I still think that's too much.
The second is the p5ylo emulator is depreciated. That work is abbandoned and now the only emulator which have support is tera shock. I dont think to update the data structure for that emulator could be useful. I want to try a data parser for shock emulator using your project if you dont mind.
If I'm wrong in something please correct me. If you let me to work above your project I will try to upgrade everything i can.
See you

 

[[GM]Xia]

DataCenter_Final_USA
just crash all of the tools

 

[Gothos]

DataCenter_Final_USA
just crash all of the tools

Link? I haven't seen a decrypted/unpacked version of that one, yet.

 

[[GM]Xia]

You must be registered to see links

i am using tera on steam but i think it have the same datacenter its the NA Version

 

[Gothos]

You need to use a decrypted file, not the one that comes with the Tera client. Like the ones mango released.

 

[kinvao]

And nobody but mango has the tool to decrypt client's datacenter?

 

[luxtau]

You must be registered to see links

Just substitute key and IV from your version. Or share your tera.exe

 

[kinvao]

@luxtau Is there any difference between the bin files posted by mango and the .dec that I could get using your code?
A few days ago I tried to parse a dec file with gothos tool and it give me an error when loading the datacenter.

 

[mangojoe]

EUR, FRA , GER, ClientRevision: 296473

https://forum.ragezone.com/f797/release-tera-datacenter-unpacked-eur-1085276/#post8533739

Thats for

 

[Gothos]

You must be registered to see links

Just substitute key and IV from your version. Or share your tera.exe

Could you explain how you obtained the key/IV?

 

[mangojoe]

@Gothos

Tera use libcrypto for decrpyting stuff. The algo is AES 128 - Mode: CFB.
Simply look for it at in IDA (make some diff of the original Libcrpto)
Then hook the function witch sets the Key and IV.

after that use the batch luxtau posted.

Enjoy!

 

[luxtau]

Could you explain how you obtained the key/IV?

Use hex pattern from batch to find that decrypt function (IDA - Search - sequence of bytes) and go upper (Ctrl+P). Then extract keys or use the following script:

You must be registered to see links

.

@luxtau Is there any difference between the bin files posted by mango and the .dec that I could get using your code?

Mine .dec it was just copypaste.

Batch names explanation:
DataCenter_Final_RUS.dat - compressed and encrypted file, original.
dc.dec - decrypted
DC.dat - decompressed

I think, you can use DC.dat in DCTools, but I haven't tested, I use own parser to extract xml which than transformed to SQLite database.

 

[[GM]Xia]

@Gothos
while i using DCTools from your release i got this error

Yellow line in this statement "while ((ch = r.ReadInt16()) != 0)"
Additional information: Unable to read beyond the end of the stream.
An unhandled exception of type 'System.IO.EndOfStreamException' occurred in mscorlib.dll

 

[Gothos]

Use hex pattern from batch to find that decrypt function (IDA - Search - sequence of bytes) and go upper (Ctrl+P). Then extract keys or use the following script:

You must be registered to see links

.

Thank you. Based on your explanation I managed to find the keys. Thank you @mangojoe for your help as well, even if I didn't manage to follow your hints.

I've added a key finder and an unpacker to my datacenter parser.

@CrystalCoder
I can't reproduce the problem. Make sure you're using an unpacked file (about 200MB, not 38MB). You could also try unpacking it yourself using my latest release.

 

[[GM]Xia]

@Gothos
i am using release datacenter so what wrong add me to skype
lo1997xd maybe will be better when you see it

 

[kinvao]

@Gothos I used your tool to unpack the datacenter on 34.06 client and it gets wrong key/iv. With older client got nothing.

 

[Gothos]

@Gothos I used your tool to unpack the datacenter on 34.06 client and it gets wrong key/iv. With older client got nothing.

I don't have the latest EU client at hand, so I can't check if key finding works for it. Tera needs to be running and you might need to run the finder "As Administrator".

The correct key/iv for EU should be:

key: 4B9062798671360CB0C7C56686E1AE2A
iv: 7E2B1F0258B86F2FB87C6A51A8B28D70


------

DataCenter_Final_EUR - download

You must be registered to see links

Spoiler

name DataCenter_Final_EUR.dat
revision 292378
key 4B9062798671360CB0C7C56686E1AE2A
iv 7E2B1F0258B86F2FB87C6A51A8B28D70
original.size 38457162
original.sha256 BFC3C20252D69AD6006D31281BE24442E68D7E718440C2746E2BB08E9D56B139
decrypted.size 38457162
decrypted.sha256 0320EC97AF98F0BA8E99D47C9085A002E4CC4C8913E93D3FAF36BCEF781914ED
unpacked.size 200078284
unpacked.sha256 F6F5A1055A6C188CD37FCF3727B39E1C96FA90C0A4B1297C0316B1A5BA198D00

name DataCenter_Final_FRA.dat
revision 292378
key 4B9062798671360CB0C7C56686E1AE2A
iv 7E2B1F0258B86F2FB87C6A51A8B28D70
original.size 39075198
original.sha256 869925B4A5A86F2B3FDC5800FB26BFCE9771E7652E2D97349EE3F3E15D216B86
decrypted.size 39075198
decrypted.sha256 12BD72AF76F3B6B950FA1AFFF17246574332202E5FE89F09B1EE2951243DDF8D
unpacked.size 203730300
unpacked.sha256 EB261E72C401573EDB1A8D757F14C3C06CDBA36BF9EAC38C8BE57CD31CBE1462

name DataCenter_Final_GER.dat
revision 292378
key 4B9062798671360CB0C7C56686E1AE2A
iv 7E2B1F0258B86F2FB87C6A51A8B28D70
original.size 38882982
original.sha256 781E837DEA2DE48020B4B70133EE6C4E948EF2C0F0C5927A3B9BA47C51858D2E
decrypted.size 38882982
decrypted.sha256 33377077CFFDB0D8EBC3C7005B812B967CE672BB6811986AE13A7F4DD5BDB736
unpacked.size 202075244
unpacked.sha256 60C6E261BC45C2EC5724D03F5D41929C3F557A549933B07F1F131794B2C640B0

DataCenter_Final_USA - download

You must be registered to see links

Spoiler

name DataCenter_Final_USA.dat
revision 295336
key E5DC912C5A8EED4A4732CE613DB0252D
iv 99489A055973C85E6A02374C79B6E821
original.size 38528505
original.sha256 18260139D6138F78B2CF327D9E24294F42A232BAC562AF1EC07E4C3643100ECE
decrypted.size 38528505
decrypted.sha256 E0A379B1822F2C2F0777C9E9ADA519C1AE27DA3FEA026CE1C69CD332D65951FF
unpacked.size 200226764
unpacked.sha256 FFA044D0D39D8A69188C7B0FC68D17EB785B775A2F3BC042F3AA47850F16124E

 

[[GM]Xia]

@Gothos
you tool working like charm!

 

[kinvao]

It seems like I was running it from visual studio(not as admin) it get wreid keys. Now I could decompile the datacenter for rev 292811.
Thanks man, you are awesome.