[C#] Skylight 3 - The Next Generation Emulator

[MentaL]

I've posted your topic on our social media pages.

 

[The General]

DMCA incoming 3... 2... 1... lol

 

[Caustik]

Hardware didn't not come so far that we can ignore architectural best practices and get lazy. Yes, your 10 user retro has no need for it nor does the largest 400 user retro. I'm speaking in terms of large servers with thousands of players consecutively at a minimal - this and any other Habbo "emulator" would never be able to handle it at scale and would require either shifting a ton of money into hardware, or having a terrible in game experience

I guess it really depends on if your intention is to learn and make a true best practices emulator, or just make another monolith copypasta that has some unit testing and the same exact architectural pitfalls as the 20 years of development before you.

PS: Clustering rooms is still a monolith.

I don't care how many users the largest retro has. Architectural best practices are the best for a reason.

Microservice architecture isn't considered a best practice, especially for a single developer. It's a useful approach in *some* scenarios
Unit tests on the other hand are considered a best practice unless you have a good reason for not unit testing

 

[TheOleg]

Microservice architecture isn't considered a best practice, especially for a single developer. It's a useful approach in *some* scenarios
Unit tests on the other hand are considered a best practice unless you have a good reason for not unit testing

I'm sorry, but neither is creating 50 classes to read 2 nested arrays.

And really not sure what is the point of unit tests here.
It's cool for something like gaming rooms with 50 wired triggers and conditions and stuff (like here ).
But I highly suspect that it will never be priority for solo developer.

Current unit tests check (if "abc".length == 3) and this nonsense:

To view the content, you need to sign in or register

Why did mods nuke yesterday posts? I hope I didn't miss any replies.

 

[The General]

Habbo is too simple to even worry about unit tests. Only thing I would consider covering is to do with player credit balance and trading.

Its just a waste of time for an emulator in my opinion. But if Joniz thinks its useful then he should do whatever suits him best.

 

[LeChris]

Microservice architecture isn't considered a best practice, especially for a single developer. It's a useful approach in *some* scenarios
Unit tests on the other hand are considered a best practice unless you have a good reason for not unit testing

Unit testing is one of the weakest forms of testing and misses out on a lot of things. Don’t even get me started on using mocks in testing

I’d rather have no testing, than 15% coverage of weak unit tests. Now if we had 70% coverage, integration tests on top of that and e2e testing in production I might feel fuzzy about it.

Current unit tests check (if "abc".length == 3) and this nonsense:

To view the content, you need to sign in or register

not even a true unit test, just a shallow test lol nice

Brb gonna stub out all of Arcturus for 100% coverage

 

[joniz]

Can we just cut the bs? Its very clear its in the early development stages.

 

[Caustik]

Unit testing is one of the weakest forms of testing and misses out on a lot of things. Don’t even get me started on using mocks in testing

I’d rather have no testing, than 15% coverage of weak unit tests. Now if we had 70% coverage, integration tests on top of that and e2e testing in production I might feel fuzzy about it.


not even a true unit test, just a shallow test lol nice

Brb gonna stub out all of Arcturus for 100% coverage

of course it is the weakest form of testing, it's in the name... *unit* testing
it's a core building block of a testing strategy, it's not the only thing you're supposed to use

stop trying so hard to appear clever, you don't need to do this

 

[joniz]

Is this not Skylight 3? Do I have to wait on Skylight 4 before I can critique?

You are not contributing any value here, you are commenting on the lack of code in a development thread. Is that not very obvious to you? I would suggest you figure out your own problems first and not act like a child.

If you can actually leave constructive criticism with relevant details on what problems they are targeting and how you would solve them, its very much welcome.

 

[Joopie]

Is this not Skylight 3? Do I have to wait on Skylight 4 before I can critique?


We need the sex command at 100% coverage pronto

Critique is fine when it doesn't come over as a personal attack. The choice of words in replies are offensive... to say the least. To name a few in replies: no constructive feedback - just destructive, only pointing out the wrongs, shouting buzzwords on what it should have been, ignoring given information regarding choices, making childish comments.

It's fine to have different opinions and ideas, but some just need to learn to put others into perspective and respect choices...

 

[Moogly]

Habbo is too simple to even worry about unit tests. Only thing I would consider covering is to do with player credit balance and trading.

Its just a waste of time for an emulator in my opinion. But if Joniz thinks its useful then he should do whatever suits him best.

After all the years of really garbage code, with rare exceptions like Martin's I think it should be a requirement to post projects in this section, if they don't have reasonable unit tested code, then we shouldn't even entertain your project.

 

[Zak©]

Now that we've had our fun on useless points and topics that you've all wasted time on, that is non refundable (enjoy).

How has development been going Joni, any updates on the public room stuff?

 

[joniz]

How has development been going Joni, any updates on the public room stuff?

Not a lot has happened since. The current plan is to fix the rough edges to a state where its semi-playable to allow some testing. But this project won't have my attention for the next month or two.

Room lifecycles were implemented, this allows proper room loading and unloading and is fully thread-safe. Rooms are kept alive by ref counting and after all references are disposed the room enters 30s grace period before it is being unloaded. First request launches the async task to load the room and sequential requests receive the reference to that, no locking. Unloading is also co-operative with this where the room first transitions to save state (very short period) and if any requests to the room were received, the existing room is promoted back. This prevent any possible race condition where the room could be loaded before it has had the chance to save its state.

Then some QoL changes like room heightmap, settings and some improvements to the catalog. And the usual bug fixes here and there

 

[Diddy]

It's been a while since I have thoroughly enjoyed following a development thread like this! Keep it up joniz

 

[joniz]

It has been quite quiet in the past few months. However, I recently got around to clean up some low hanging fruits in the bootstrapping logic and took some positive measurements from the after-match. The inner dev loop has been very important focus of the project and ensuring that productivity is not being wasted. Lets dive more in depth to this!

Bootstrapping
The bootstrapping logic is at the core of this, the longer we have to wait for the server to start accepting connections, the more time we end up wasting. I have few tricks here I use to get the most bang for my buck. The first thing that pops in to anyone's head when they want to make something faster is to run as much code in parallel as possible. The ILoadableService<T> concept plays a big part in this and I have more in depth explanation over here. In short, anything that needs to be loaded at the start is run in parallel while preserving the order of dependencies. While this does significantly improve the startup time, its not the only thing that contributes to it.

The other major contributing factor is the client. The sooner the client is able to start communicating with the server and bootstrap itself, the faster you are able to enter the room. I have two special config knobs here to improve the experience for development purposes.

The first one is a EarlyBind. With this option the listeners are bound as soon as possible to allow the client to establish the TCP connection to the server as fast as it can. The connection wont progress past authentication but it allows more snappy experience as the server can then let the clients in as soon as its ready, this can significantly reduce the roundtrip time. This can also be safely used in a production environment.

The second option is a EarlyAccept which is more fragile option and meant only for development purposes. This option lifts the restriction of not letting the connect progress past authentication before the server is in a steady state. This allows the server to process some parts of the authentication while still trying to ensure that no observable side-effects occur. For example, the catalog is not necessary for successful authentication and can be lazily initialized.

With all of these tricks, the client can be in a room in just couple seconds! How awesome is that!? Check out the video below for the demonstration!

For demonstration purposes, I am using the Nitro client as it can load much faster than the traditional Flash client. I also have some broken assets, so don't mind that, hah!

Hot Reload
While bootstrapping is one key component of the inner dev loop, its not the only part. The second part is not needing to restart the server at all and be able to observe the changes as you make them. While the C# runtime does support code hot reload (

To view the content, you need to sign in or register

), it does not mean that it works on all of the components out of the box. This could be due to some data being only initialized during bootstrapping and would not react to external components being added. Thankfully the runtime provides additional tools (

To view the content, you need to sign in or register

) for developers to integrate hot reload functionality more deeply into their code bases to allow refreshing these caches that are initialized during bootstrapping. This functionality now allows that the newly added components can be recognized and be made part of the existing components.

One great example of this is the packet handlers. As the developer adds a new packet handler, the server is able to pick up these changes and apply them accordingly. This now allows the developer to not close their client and immediately test the functionality that they just created. This type of change is more nuanced and requires refreshing of the whole network layer. This functionality is fully implemented in the server.

Another good example would be changing the packet definitions. These changes are more isolated as they live in their own assemblies and have only single entry-point. These can be more easily added or removed at the runtime and requires little extra handling. This functionality is fully implemented in the server.

Check out the video below for the demonstration!

 

[Z3r0j]

It has been quite quiet in the past few months. However, I recently got around to clean up some low hanging fruits in the bootstrapping logic and took some positive measurements from the after-match. The inner dev loop has been very important focus of the project and ensuring that productivity is not being wasted. Lets dive more in depth to this!

Bootstrapping
The bootstrapping logic is at the core of this, the longer we have to wait for the server to start accepting connections, the more time we end up wasting. I have few tricks here I use to get the most bang for my buck. The first thing that pops in to anyone's head when they want to make something faster is to run as much code in parallel as possible. The ILoadableService<T> concept plays a big part in this and I have more in depth explanation over here. In short, anything that needs to be loaded at the start is run in parallel while preserving the order of dependencies. While this does significantly improve the startup time, its not the only thing that contributes to it.

The other major contributing factor is the client. The sooner the client is able to start communicating with the server and bootstrap itself, the faster you are able to enter the room. I have two special config knobs here to improve the experience for development purposes.

The first one is a EarlyBind. With this option the listeners are bound as soon as possible to allow the client to establish the TCP connection to the server as fast as it can. The connection wont progress past authentication but it allows more snappy experience as the server can then let the clients in as soon as its ready, this can significantly reduce the roundtrip time. This can also be safely used in a production environment.

The second option is a EarlyAccept which is more fragile option and meant only for development purposes. This option lifts the restriction of not letting the connect progress past authentication before the server is in a steady state. This allows the server to process some parts of the authentication while still trying to ensure that no observable side-effects occur. For example, the catalog is not necessary for successful authentication and can be lazily initialized.

With all of these tricks, the client can be in a room in just couple seconds! How awesome is that!? Check out the video below for the demonstration!

For demonstration purposes, I am using the Nitro client as it can load much faster than the traditional Flash client. I also have some broken assets, so don't mind that, hah!

Hot Reload
While bootstrapping is one key component of the inner dev loop, its not the only part. The second part is not needing to restart the server at all and be able to observe the changes as you make them. While the C# runtime does support code hot reload (

To view the content, you need to sign in or register

), it does not mean that it works on all of the components out of the box. This could be due to some data being only initialized during bootstrapping and would not react to external components being added. Thankfully the runtime provides additional tools (

To view the content, you need to sign in or register

) for developers to integrate hot reload functionality more deeply into their code bases to allow refreshing these caches that are initialized during bootstrapping. This functionality now allows that the newly added components can be recognized and be made part of the existing components.

One great example of this is the packet handlers. As the developer adds a new packet handler, the server is able to pick up these changes and apply them accordingly. This now allows the developer to not close their client and immediately test the functionality that they just created. This type of change is more nuanced and requires refreshing of the whole network layer. This functionality is fully implemented in the server.

Another good example would be changing the packet definitions. These changes are more isolated as they live in their own assemblies and have only single entry-point. These can be more easily added or removed at the runtime and requires little extra handling. This functionality is fully implemented in the server.

Check out the video below for the demonstration!

Hello joniz, I was watching your project its look good, I would like to contribute to it, I have like 4 years outside of the Habbo hotel world and I would like to return, here is my github:

To view the content, you need to sign in or register

 

[Diddy]

Hello joniz, I was watching your project its look good, I would like to contribute to it, I have like 4 years outside of the Habbo hotel world and I would like to return, here is my github:

To view the content, you need to sign in or register

Its open source on github, start contributing! Joniz can always review your PR's!

 

[IDjinn]

Hey nice project, love c# and see people developing using it is awesome!

I saw some code, think its quite hard to unsderstand at first glance, if it has intuite of comtribuitors (e.g plus) should be nice have docs in roadmap

PS: some files has `lock(this)` instruction, its not a good pratice

 

[joniz]

if it has intuite of comtribuitors (e.g plus) should be nice have docs in roadmap

I have already tried to document as much as I can but most of it relies that you are already familiar with modern C# tooling, which I do understand that everyone might not be. I'm happy to clarify any pain points or direct to the Microsoft provided documentation on the relevant topics if needed.

PS: some files has `lock(this)` instruction, its not a good pratice

The few places that has it is because its not a big concern. Its not possible for external components to cause a deadlock in practice.

 

[Roper]

Is there any further progress or updates with Skylight?