Thanks everyone for the replies. I'll be starting a release thread soon in the appropriate section. This thread will just be for general discussion of the project, so keep any ideas coming!
Here is my final WIP video if anyone is interested:
You must be registered to see links
(I accidently recorded with music playing, so if you don't like it mute
)
I understand what you are saying.
The main point of the format is to share the "common" files between all private servers based on the same file set. If any server had a custom version for a file, then that file wouldn't be part of the "shared" setup.
For example, you download and install the base VSRO 1.188 client files. For any file that a private server does not modify, they simply don't redistribute that file. This means at least 90% of the files used in Silkroad won't actually be redistributed again and again, since they are always the same and never modified.
This isn't easily doable with archive setups since you'd have to modify the archive each time. In terms of the files that are distributed, it just depends on the system used. A "difference" file could be used to reduce the size of the downloads. The client would then duplicate the original file and apply the changes to the new file, leaving the old intact. Alternatively, they could just distribute the entire file. I've not really worked on a final system for that yet, so there's options if enough people were interested.
When it comes to the actual unique private server files though, they really would need to accept the fact that people will find a way to get access to the file no matter what, so they just distribute an unprotected file. A lot of people try to fight this, but they are wasting their time (in my opinion).
Consider all the private servers that have "custom files" and look at how many have had their blowfish key found, any modifications in their file format released, and so on. Are there even any long running servers that this hasn't happened on yet? What does it even gain them? In theory, exclusive content is needed to make a server stand out, but that only works when the files aren't public and you have very little competition. At this stage, exclusive content isn't going to be enough with how many servers there are.
The PK2 format has long been compromised, as pretty much any game file format has. If anyone really has something they want to keep private and protected, the
only solution is to not distribute it in the first place!
If someone really, really wants to try and keep their files protected, then the only way is to use this system and make a private implementation of it that implements compression, encryption, digital file signing, as well as run-time verification. That is a lot of work, and really hard to get right. All that for what? DDJ files people can steal through Direct3D detours? Txt files people can hook the client to rip? As soon as someone does reverse it and make tools, then you are back at square 1, which is why I'm not a huge fan of it in practice in this context.
Client side protection just isn't viable with Silkroad unless you had source code to the client. Without it, you need custom file handling, an obfuscated client with a lot of VM'ed code, and a run-time protection that doesn't actually suck (not cheap). All this is possible with enough time and resources, but it would take forever and not really make the server more desirable to play on than others.
I think ultimately, that is what matters the most. Content is still important, but people care about more than just that. The rates of a server, the management, the community, events, the "donations" or perception if it's "pay 2 win". These are all the main things that make and break a server. Focusing on content in this setup is just impractical because if you can do it, someone else can too.
Yea, that's how it works right now (I'm just using Win32 file i/o). I feel that compression is good for exchange (keep bandwidth usage low), but not really needed for on disk files with hard drives as large as they are nowadays. What I had in mind was lzma though, as that's what ISRO updated to a few years back because it offered better compression over zlib, at the expense of slightly more processing power on decompression.
Definitely, a plugin system is what I'd like to aim for to accomplish the custom security and features needed for each server. However, I'm starting out with the base project without any extras, so people are free to experiment as the want. It's quite a big change, and still experimental, so I don't see a lot of people rushing to try it out yet.
In that regards, by releasing everything as I'm about to, I'm conceding the fact most people would rather have private implementations for their servers rather than use a common shared system. Until people realize how DRM does not work, and if the client has your files, you cannot protect them, then I don't think that will ever change. So, I won't stand in their way, which is better for everyone since I'm not particularly committed to Silkroad. This project was just random because I needed something to do because of how bad Diablo 3 went.
I still like the idea of changing up how updates are done. Silkroad.exe is (still) horrible and the download server isn't too great overall. With the free standing file system, people should hopefully be able to start looking into better alternatives for handling updates and content delivery. However, it's understandable why people just use what works as it is.
I always liked the web based approach. What you can actually accomplish now with this project is stream files to the client as they are needed. Similarly to how a lot of games are streaming content to reduce download sizes, the same system could be implemented with Silkroad. This way, if you want to try out a new server, you don't need to download 2GB worth of files. You only might need a small portion to get into game and test out the server. The most obvious issue with this though is that it'd be hard to do 'smoothly', but I think it could possibly work out.
As for the network protocols, I'd not hold my breath on that. After you do so much dev with a game like Silkroad, and understand how it works and is setup, it'd probably just be better to invest the time and resources into your own game instead. That way, you don't have to worry about any legal issues and you have full control over everything.
At this point, we all aren't getting any younger. Considering how much there is to Silkroad packet wise, I know for sure I'm never going to sit down and try to reverse the entire protocol. That is, unless it was a well paying hourly job. Other than that, it's just not worth it to me at this stage, which is why I've not attempted to setup anything for it.
For regular development, I use the right click context menu to choose Notepad++, Hex Workshop, Excel, or whatever else I need. Out of security habits, I typically never left click to open files, as the extra step gives a brief second to think about the action before doing it. When I'm making a video, I try to keep my typical developer habits out, so that's why a lot of things might seem 'awkward'. Opening in notepad, only having windows on screen I use (I have dual monitors), going to File->Save, hiding desktop icons, are all intentional simplifications in the video that carry over from making tutorials. Rest assured though, I do make use of the good stuff normally though.