The Hunger has moved to brand new servers and the huds have been updated to v4.5.
The main changes I've made involve HTTP-In. This is a feature of Second Life that allows you to assign a web address to objects and we use this in part to communicate with the huds.
A normal http request from a hud will send a message to our server and wait for a reply. These are very reliable and were not the issue. With HTTP-In we can also send a message from our game server to your hud and the hud sends a reply back. This is how we do things like update your hud info when clan changes are made via the website, and these are the connections that were causing issues. It would seem that occasionally these connections can be very slow for no apparent reason, even in a very healthy looking sim.
Here's an example... you arrive at a new region, your hud requests an http-in address and then sends that information to the game server. The game server adds your hud address to the database for http-in communication, then looks up other huds currently in the same region and contacts those huds letting them know you arrived so that you appear straight away on their scanner. Once it has done that, it sends the reply back to your hud to end the initial communication.
The problem was, when the server was attempting to contact other huds, if it hit too many delays, the original connection sent from your hud would time out after 60 seconds, and you would see a "Connection to server failed" message. In reality the initial connection to the server worked fine, but the attempted connections to other huds in the region were failing and the whole thing timed out. The same goes for when you make a kill... your kill attempt would be sent to the server from your hud, the server would try to inform the other huds in the region of the game changes and then reply back to your hud to end the communication. Again, if there were delays, your connection would time out and your hud would never receive the expected reply.
How have we fixed / avoided this? Several ways...
Since the last hud release, Second life has a new function called llGetAgentList() that will return a list of every avatar in the region. The scanner now uses this function and will instantly know about every avatar in the region, including non-playing humans. This means your hud no longer has to be informed when a new hunger player arrives, it will discover them for itself even if they are not wearing their hud and are thousands of meters away. It also means it no longer has to use llSensor() with it's limited 96m range and max 16 results per scan to find humans, so even in a very crowded area you will be able to see the 20 humans closest to you in correct distance order.
In other words, the scanner no longer relies on http-in connections to tell you what you need to know.
The scanner also uses another newer feature, PRIM_LINK_TARGET, which allows it to build a large list of parameters to use with a single llSetLinkPrimitiveParamsFast()... this means that instead of adding a single line of text at a time to the scan window, it can add them all in a single call. This will result in less transmissions from the second life server to your client and should make scan results appear faster on arrival at a new region.
The one problem with llGetAgentList() is it will always return every avatar in a region, so potentially 100 UUID's all at once. This can use a lot of memory and possibly crash a script. I think I've managed to squeeze it in there though and it has been tested working in a region with 81 agents. It's possible more than that could crash the script, but a region with more than 81 avatars is not really the place to be playing a fast paced hud game anyway :) If you find a region with more than 81 agents, please report to me if the scanner still works! Should the scanner crash, a hud reset will get it running again.
Fight / Bite Communication Improvements:
To avoid these issues all together I've done away with the need for HTTP-IN completely when it comes to fighting and biting. This means when you make a kill, rather than the game server trying to contact all the huds in the region to inform them of your victims death or the new blood level of a human you just bit, your hud will instead send a message locally within the region telling other huds of these details, which should prove much faster and more reliable and allow you to still play even when http-in is failing.
The huds still do use http-in for other features and this can't be avoided without a major re-build. Ground fighting and blood transfer for example were built from the ground up to use http-in. But I have at least been able to improve things / let you know when there is an issue...
Attempted http-in communications to your hud will now time out after a reasonable amount of time so that your hud will get a response to its requests. In the event that http-in failed, it will be added to a message cue so another server side script can try again using more relaxed time-out settings. It will first attempt communication again with a reasonable time-out. If that fails it will try again with a more relaxed time out. Any messages still failing after that will be re-tried with no time out in an attempt to find the issue (I've had proxy error message returned only after 110 seconds!!). This should allow the cue to quickly free up messages that are working at a reasonable speed, but still find out what's wrong with messages that just refuse to go though. In the event of total failure the message will be removed from the cue and you will receive an IM from a separate device (The Hunger Messenger that sends you IM's about things like promotions done via the website) letting you know that it failed and that your current region is having problems. You will also receive an IM if the message succeeds, but was particularly slow, warning you that the region could be having trouble and it might be better to play elsewhere.
- Human blood now increases by 1 every 12 hours instead of 2 every 24 hours (so don't panic when you start getting un-even amounts from bites). This was to spread out the load for the database operation.
- The scripts that bring you back to life are now scheduled server side (rather than using in-world devices) so even if all the Hunger regions go down you'll still come back to life on time.
- Links to help pages and profiles in the HUD have been fixed
What else is new?
All of this is now running on new servers. We've moved away from our old dedicated servers because it no longer makes any sense to use them. Dedicated servers, once rented, don't tend to be upgraded. As a result our servers were looking a bit outdated compared to more modern offerings. Instead we are now employing cloud servers that not only will be easier to upgrade in the future, but also give us a lot more bang for our buck. Before we were running on a dual core machine with 4 gigs of RAM, large HDD's and were paying a lot for features we really don't need such as "unlimited" bandwidth and vast storage. For just a little LESS money than before we can now have 4 cores, 16 gigs of RAM, and best of all, SSD storage that will see improvement in the most important area for our game, database speed. Every action in the game gets stored to the database and solid state drives will help shave off milliseconds where it counts. The new servers are in a San-Francisco data centre, moving them closer to the Second Life servers. I've seen an average ping of 22ms to Second life regions, 3 times faster than the old servers.
The old servers also came with a lot of things installed we really didn't need, such as resource hogging control panel software. The new servers have been built from the ground up with the bare essentials. We've moved away from clunky old memory hogging Apache and have instead adopted nginx, which has a much lower memory footprint than Apache, spits out information faster and scales up much more easily.
Finally, should we ever have the need to upgrade in the future, we can literally do so at the push of a button.
A well as running on nginx which should make the site feel noticeably more snappy, all game statistics are now pulled from a local database. Before, the website server would contact the game server for statistic information, using its resources, database and bandwidth. Database replication now means the website gets all changes as they happen replicated to its own local database, meaning no matter how hard people hit the statistics pages on the site, it will have no impact what-so-ever on the game server.