Writing a tool for BDO Item Data but need some advice

[PenguinJoe]

Hi folks.

As I am unaware of any BDO dev discords since ODO shut down I have to ask this here.

I am currently writing a tool that works in conjunction with sites like BDO Codex and BDDatabase to provide offline item lookup and queries. There is already a tool by @Nopey derived by some code put out by @Lautaro that does this kind of thing in a limited fashion.

What I have in mind is a 2-part system. Part one is a tool that scrapes data from BDO Codex (or BDDatabase) and stores it in an sqlite database.

The second part is a tool very similar to Nopey's and my own that queries that database and presents it to you along with options to hop out to BDO Codex or BDDatabase to see their more comprehensive information on it. The main difference is that the new tool will catalog well over 40,000 BDO artifacts (the scraper takes a solid 8 hours to run) and let you look them up.

The part I need help with is this:

The tool is driven by a file containing item IDs (or indexes as they are called in some databases). Older versions of the game like Archer stored this information in gamecommondata\datasheets.bexcel. Newer clients do not have this file embedded in their .PAZ files.

I have managed to find an items listing in .xlsx format in the Official nova server files and am currently using that to drive my scraper. But that only gets me items up to and including the Nova version.

My question to to you guys is...

Do any do you know how or where to find something similar to that datasheets.bexcel file or even group of files in the most up-to-date/official clients? I have no problem getting my hands dirty digging into into binary files with a hex editor or doing some homebrew translating.

Some nudges in the right direction would be much appreciated.

[MentaL]

[Kirito2105]

Hi, now you can't found file bexcel because pearls delete this file from client.
Now they using .bss/.dbss file how table excel but binary system.
Files .bss/.dbss you can find in paz to path "PAZ/gamecommondata/binary"

[givenzero]

Im not sure if you can actually read dbss files tho you can get corsair item list maybe reboot or drakania version if you know some people. Also as @Kirito2105 said you cant really get it from paz files. Seems like only way you pull data from bdo codex https://bdocodex.com/us/items/ in this link which is 200+ pages will take long time. Good luck

[Zexuce]

is there any way edit the bss/dbss files

[The Sourcerer]

is there any way edit the bss/dbss files

bss and dbss are structured binary files. Each file uses a specific structure for each type of data it contains. There is no single tool that can work with them all. Each requires a specialized editing tool to handle the extraction and repacking of the structured data structure it contains. This means you are on your own creating a tool for each file you want to edit.

[The Sourcerer]

Apparently there are ways to query the official BDO servers to get information in a roundabout way. The official BDO client lists a site in its service.ini for interacting with the Central Market. Using that site it may be possible to scrape its item information. Responses are in JSON format so it should be to pop it into a database, file or spreadsheet.

Here is the content of a live service.ini from my own current installation.

[SERVICE]
TYPE=NA
RES=_EN_
nationType=0

[NA]
isUseTotalAccount = 1
AUTHENTIC_DOMAIN=gameauth.na.playblackdesert.com
AUTHENTIC_PORT=8888
PATCH_URL=http://naeu-o-dn.playblackdesert.com/UploadData/
viewTradeMarketUrl=https://na-trade.naeu.playblackdesert.com/
gameTradeMarketUrl=https://na-game-trade.naeu.playblackdesert.com/

The url in blue is the one you query.
The green one is for actual in-game trades.
I posted a thread in the tutorials forum with information for the http request and expected responses that you can use with that site.

[givenzero]

That you can only wrap central market item list and item information, icons is stored in client so you probably will get only paths to icons. So this will only return item id and category sub category price values etc. OP asks for all item data so this not really an answer to OP’s ask. I think instead of trying to pull data from client easiest way would be making a bdo codex wrapper. Tho if you can figure oyt how to extract data from client that will be just a lot faster.

[The Sourcerer]

That you can only wrap central market item list and item information, icons is stored in client so you probably will get only paths to icons. So this will only return item id and category sub category price values etc. OP asks for all item data so this not really an answer to OP’s ask. I think instead of trying to pull data from client easiest way would be making a bdo codex wrapper. Tho if you can figure oyt how to extract data from client that will be just a lot faster.

Sadly clients newer than the Archer release have the data encoded in bss and dbss files, which are not so easy to crack. The older releases had a gamedata.bexcel which could be converted and loaded into a spreadsheet.

The tool that @PenguinJoe created does just that - calls bdocodex or bddatabase to get icons and descriptive information. My suggestion wouuld let you get the up-to-the-minute latest information out of the BDO official server without depending on the operators of the bdo database sites to update their information.

[Fyyre]

it's funny reading posts from people like you who don't know what you are doing.

I mean ... the leaked versions of Corsair, Nova, Sage... (that the retards at dreambd pissed all over the public sphere) - all come with a special DLL that (pro tip: zomg, I coded it!) .... converts the server side excel into dbss/bss ...

so, I don't know what kind of whack nonsense statements such as: "data encoded not easy to crack" .. when I made the client make the crap from scratch.

Granted, no one here will receive an ounce of help from me~

-fyyre

Sadly clients newer than the Archer release have the data encoded in bss and dbss files, which are not so easy to crack. The older releases had a gamedata.bexcel which could be converted and loaded into a spreadsheet.

The tool that @PenguinJoe created does just that - calls bdocodex or bddatabase to get icons and descriptive information. My suggestion wouuld let you get the up-to-the-minute latest information out of the BDO official server without depending on the operators of the bdo database sites to update their information.

[The Sourcerer]

it's funny reading posts from people like you who don't know what you are doing.

I mean ... the leaked versions of Corsair, Nova, Sage... (that the retards at dreambd pissed all over the public sphere) - all come with a special DLL that (pro tip: zomg, I coded it!) .... converts the server side excel into dbss/bss ...

so, I don't know what kind of whack nonsense statements such as: "data encoded not easy to crack" .. when I made the client make the crap from scratch.

Granted, no one here will receive an ounce of help from me~

-fyyre

Jeez, did somebody piss in your cornflakes this morning?

Good for you. You figured it out. Give yourself a pat on the back. Now go away.

[Fyyre]

Jeez, did somebody piss in your cornflakes this morning?

Good for you. You figured it out. Give yourself a pat on the back. Now go away.

It's one thing to say "you figured it out, no go away".

It's another thing to say that to the person who enabled you lamers to create dbss/bss to start with.

So here is my challenge to you: improve upon the process, and release it. Until you can do something other than parrot instructions or copy and paste from things other people created, keep your mouth shut.

[Vantuz]

It's one thing to say "you figured it out, no go away".

It's another thing to say that to the person who enabled you lamers to create dbss/bss to start with.

So here is my challenge to you: improve upon the process, and release it. Until you can do something other than parrot instructions or copy and paste from things other people created, keep your mouth shut.

Did you have the source codes of these clients or did you work only through the assembler?

[The Sourcerer]

It's one thing to say "you figured it out, no go away".

It's another thing to say that to the person who enabled you lamers to create dbss/bss to start with.

So here is my challenge to you: improve upon the process, and release it. Until you can do something other than parrot instructions or copy and paste from things other people created, keep your mouth shut.

Ok smartass. You coded it. Why? Out of the goodness of your heart? Did you publicize or even bother to document the thing or were you simply storing that bit of information in your head requiring the rest of us to use a crystal ball? Sorry my crystal balls are cracked. At least I am making the fkin effort.

It is one thing to pat yourself on the back saying "I'm the greatest coderr evar!", it is a whole other thing to support that great work.

Granted, no one here will receive an ounce of help from me~

That pretty much says it all dontcha think? So perhaps instead of bashing the efforts of others you perhaps keep your damn mouth shut instead huh?
Have a Merry Christmas. Perhaps you might get something other than a lump of coal in your sock.

[heuinmin]

Saw fyyre coming back online after a year of inactivity, usually with something to gift. Not this year. Looks like there is nothing more than past glory. Mmorpg is almost dead with all shared sources dated back 5 years at least. Perhaps fyyre should let go of all the remaining sources to complete loko last wish. Young lad being too harsh with the old reverse engineer, forgetting he took on companies infiltrated and brought out almost of the files you can get to download for free on ragezone, maybe you can take on the same journey as he did and bring us something new than fixing something that was already fixed.

[Katsu99]

Omg, noname clown asking Fyyre shut the mouth :D