hello everybody

I decided to open a thread to track the servers in case of incidents, so this way in case something is going on with the servers you can always come here and know what's going on.

SUMARY (click on any entry to read)

[12-09-2012 12:00] HiszY CTF & HiszY II servers down (status: CLOSED)
[19-11-2011 00:00] network problems (status: CLOSED)
[31-08-2011 15:00] HiszY-APD CTF Server random error (status: CLOSED)
[04-08-2011 19:00] HiszY Servers company network maintenance (status: COMPLETE)
[01-03-2011 00:00] HiszY web and mail services rebuild (status: COMPLETE)
[10-02-2011 09:00] HiszY Servers company network upgrade (status: COMPLETE)
[19-12-2010 21:15] HiszY II & CTF Servers Temporaly Unaccesible (status: CLOSED)
[10-10-2010 13:00] Crash at the Freeze Server (status: SOLVED [recurrent, permanent fixing + further testing, planification])
[07-10-2010 19:00] Network Errors at the Freeze Server (status: CLOSED)
[25-09-2010 00:00] Maintenance of the Freeze Server (status: COMPLETE)
[22-09-2010 00:00] Maintenance of the CTF Server (status: COMPLETE)
[02-09-2010 01:30] CTF Server Temporaly Unaccesible (status: CLOSED)
[25-08-2010 20:00] Servers Temporaly Unaccesible (status: CLOSED)

Wednesday 25 of August, 2010, at 20:00 - Servers Temporaly Unaccesible (status: CLOSED)

hi there everybody

yes, I recently (around 21:00) verified that today, wednesday 25 of August of 2010, after 20:00 the servers are unaccessible, huge apologize, after make a investigation I found out that is because a problem that the servers' provider is suffering. they are actually on it.

as 22:30, as far I know and if I did not understand wrong, they did no indetify yet the source of the problem (what it was failing is clear, but not why and thus how to solve it) so it has been said that they will temporaly activate their backup infrastructure so the servers should be back in short, early this nigth I hope. so while they do not get the problem completely solved it's to expect that the servers will still keep occasionally failing or simply not performing so good.

as guessed, rigth after activate the backup infrastructure I can confirm now that the servers are back again, however as explained before this is just a temporal solution in this time they will keep working to get everything back as it was so some cuts are expected.

also around 23:45, as with the activation of the servers' company backup infrastructure this tiggered some problems with one of the servers wich leaded me to force a reboot that due these problems is lasting a bit more than usual (few mins), I guess that in few time it will be back, no more than one hour I hope.

finally as 00:45 (of the next day, 26 of August, 2010) the rebooted server recovered from long reboot (around but less than one hour, as expected) and now everything seems to be fine, so by now while they keep working to completelly solve the issue the servers should work with relatively normality.

and as thursday 26 of August, 2010, around 2:00 they claim to have solved the problem, so by now this incident will be marked as solved and it will pass to the verifying process, in my opinion it's still too soon to claim that the problem has completelly gone, I still would expect some more (minor, I hope) problems, so in next days I'll keep monitoring the servers and if nothing happen then I'll finally close this incident.

today around 13:30 another incident at the CTF Server, most likely related with this, has been detected, I'll force a reboot now at that server to see if that solves the problem, if not I'll have to undergo a complete reinstallation of that server in the next days.

ok, it seems that just after few mins (as usual) the CTF Server this time started properlly and everything is working again as normal however to make sure that everything is fine, I'll reissue a second control reboot now at that server.

once again just after few mins the secondary control reboot at the CTF Server went rigth so most likely it just was some kind of false start due the yesterday problem then by now both server should work normally.

honestly, I was almost forgot about this incident, so after more than a week and no more problem this incident is now marked as closed, this kind of problems are usual and they are recurrent over the time so it's possible that this appear again on the future however then it will be treat as another different independent issue.

to end I just want to sincerely apologize for this inconvenience and thank you all for your support and your patience.

regards

Thursday 2 of September, 2010, at 01:30 - CTF Server Temporaly Unaccesible (status: CLOSED)

hello

it seems that as a spin-off of the above closed incident today in the morning a similar problem arose at the CTF server (the freeze one is not affected), it seems that it did not start to affect the server performance until this morning after 8:00 and ended with the server unaccesible around noon, 12:00. I noticed this issue around 13:30 and after a quick maintenace at 14:00 the server was again operative, I believe that it was just a minor hardware/software failure and I don't expect any further problems, as much, some small cuts for maintenance, so by now, this is marked as solved and it enters in the verifying process, however, anyway, I'll scale this to the server's company so they'll can investigate further if needed.

as 17:45 the server's company has been properly informed of this incident so they should be aware of it, also note that while they work on this may be some small cuts are expected.

well, after 72 hours of the incident I don't know if the company did something, all I can say is that they were unable to give any sufessfull answer, but since it did no failed again I think that I can assumeno that it wasn't a relevant problem, just may be a temperol failure, so I'll mark as closed now this incident.

ok, as I'll keep updating this post as need with new information.

thanks for your support and your patience

best wishes

Wednesday 22 of September, 2010, at 00:00 - Maintenance of the CTF Server (status: COMPLETE)

hi there

recently, early this morning, Wednesday 22 of September, 2010, I found that there's a small "bug" affecting the server at initialization time, probably the problem is consequence of the silent, bigger maintenance, mainly a complete redesign of the servers, that I'm doing in the background that it's not advertised becasue it shouldn't affect the servers. I still have to determine the complete cause of the problem, this problem has been affecting the server during the last three days, however as you can see it's not a critical problem but just a minor incovenience, as you can see the server can perfectly works even because of this problem, so I won't inmediately interfire, I'll plan this for this or tomorrow's nigth, depending on how much players connected or what other tasks I have to do (many btw), during this time I'll reboot various times the server to determine and solve the problem then please be patience, thanks.

ok, as the afternoon of the saturday 25 of September of 2010 this maintenance has been completed. finally yesterday at very late night, today in the morning (around 3:30), I found the server nearly empty so I procceded with the maintenance task, by now, all the minor problems has been solved, and the mayor maintenance in the background maily finished, the number of improvements is large, among others the server now should perform better, has better tolerance to problems (in other words be more robust), improved security or be more easy to maintain for example, however this does not mean the end of nothing, as always this is just another step in teh way, I'll keep maintaining and improving the server, however the most relevant part should be done by now.

thanks for your support

best wishes

Saturday 25 of September, 2010, at 00:00 - Maintenance of the Freeze Server (status: COMPLETE)

hello all

following the past successfull maintenance of the ctf server, I'll now apply the same proccess to the freeze server. this maintenance is marked now as in progress however it has assigned bulk priority this means that since it's something that is causing no problems there will be other issues who will have higher priority and then I'll keep maintaning this silently in the background thus as consequence of this there won't be a end date for this, however I measure than if no mayor problems occur I should have complete this like in the next week or so. usually it shouldn't be necessary to inform about this, I reagulary maintain everyday the server, and even if this is a mayor maintenance, after being successfully applied previously at the ctf server everything should be fine, no mayor problems should occur, if no problems only some reboots at wierd hours are required, however I decided to post about to keep you informed and also as a start point in case I make some mistakes in the proccess.

ok, as of tuesday 5 of October, 2010, this maintienance has been completed, everything were fine, there was no relevant problems and in fact, there wasn't even any noticeable reboots.

thansk for your support, have fun

regards

Thursday 7 of October, 2010, at 19:00 - Network Errors at the Freeze Server (status: CLOSED)

hi

today, thursday 7 of October, 2010, at 19:00, I've just found that since this morning the server was suffering network errors, I'm still investigating the true source of the problem, I believe that it's just a random software problem, however I do not discard that it could even be a hardware problem (in simple words, that some of the server's parts were damaged, network cards, cpu, memory ...). by now I issued a reboot (that due these problems is lasting more than usual) if it's a software problem this should fix the issue if not I'll scale this to the company and I'll try to arrange this as soon as possible.

after 1 hour, by 20:00 the reboot still did not complete, I wouldn't expect this to last too much in few time the reboot should complete but by now I just can keep waiting to verify this, sorry.

ok finally just minutes after previous message the reboot operation sucessfully complete, in just quick check it seems that the problem gone, however I will procced now to a deeper testing.

ok, and after about four hours of the reboot everything seems fine so I guess that it was what I though just a random problem with some software related with the network. I wouldn't expect this problem to repeat (at least not often) so I'll keep checking it for the next 48 hour and if nothing changes then I'll close this incident.

and as of the noon of the Sunday 10 of October, 2010, after more than 48 hours after these errors no more problems occur, well, rigth now, another icindent that may be related occur, but that will be treat indepently, then by now I'll mark this incident as close, most likely, as I said, it would be just a random failure of the network related software.

sorry for the inconvenience and thanks for your patience and support

regards

Sunday 10 of October, 2010, at 13:00 - random server crashes (status: SOLVED - recurrent, permanent fixing + further testing, planification)

hi

as the morning of Sunday, 10 of October, 2010, around noon HiszY I crashed.

in addition this same problem affected the ctf server the past saturday 19 of february 2011 and today, thursday, 24 of february 2011, just a while ago it affected again HiszY I, both incidents also coincided at around the noon time and just few minutes later all server have been recovered from these incidents.

(more info soon, I hope :/ )

sorry for the inconvenience and thanks for your patience and support

regards

Sunday 19 of December, 2010, at 21:15 - HiszY II & CTF Server Temporaly Unaccesible (status: CLOSED)

hi

rigth now at around 21:15 of the Sunday, 19 of December, 2010, I found that the HiszY II & CTF servers are unaccesible. the reason is unknown, at my side I don't see any, so I believe that it would be just a routinare maintenance of network at the side of the Servers' company then I belive that in few time, let's say around midnight or so as much, they should be operative again.

ok, after 30 mins of the incident, 21:45, it seems that they are back again, I'm now performing usual maintenace at my side to make sure that everything is back again in place this should only last few minutes if no problems if not the operation could take up to few
hours.

in a first look it seems that it will be the case, this operation will take around one hour or so, let's say until midnight.

ok, and as around 23:30 the servers are operitave again, I've been looking and there wasn't anything however I applied a quick fix to increase the stability and the response in this kind of cases, so by now the incident will be marked as solved I don't expect that this perist however as always I'll keep closely monitoring this for the next 72 and then if nothing happen I'll defitively close this.

unfortunately a while ago today wednesday 22 of december 2010 around midnight this problem arose again, the problem has nothing to do with me or the servers (I mean at my side) is a issue external to the server so rigth now I'll inform the company about and then I'll wait for a week or so before close this to check this more deep.

ok, after near two weeks of the incident I think that it can be closed since there wasn't more problems and everything looks good if not better than before. on the company response they admited the problem and explained that it was just little problem because it coincided with a unexpectable load for them, I don't know if they made something to solve this load spikes or not (I want to believe that they did or are doing) but I don't expect that this arose nor at least frequently although the incident will recive recurrent character at least for some months more.

unfortunately today, sunday 9 of January 2011, in the morning at around 00:45 the problem arose again at that time there was a small, this time, interruption of the service of about 15 mins. the problem is solved now, it just arose at 00:45, at around 1:00 it gone and just after 15 mins or so at around 1:15 the system was rebooted and the problem solved so the servers should be working with absoletely normality by now. well, as expected, the problem remained recurrent, however contrarie to what the company explained initially it does not look like a overload issue (although the overload it was NOT related with the HiszY Servers, if not at his side, unless a attack or something yesterday in the nigth it didn't really look like a very high internet activity time) so in my opinion the problem looks more like some kind of stability (alhtough it could also be just maintenance, net reboot, or similar), well, I'll wait some days to see how this evo (initially it should be most of the times ok, more or less the frequency of the issue it doesn't seem to be very high, around a failure each 2 or 3 weeks, something that obviously it's not good but that fits inside a normal and understandable situation, at least, temporaly) and in case of further problems then I'll recontact them again to see what happen and if not then this will just be closed again and left as recurrent problem for at least some months more.

well, finally today thursday 10 of February 2011 the company really does something about this, today they upgraded their network (among other things but also suposedly becasue of this I guess) the upgrade is still in progress and it shouldn't be a major concern (look next entry for more details) it should complete in few hours or days as much and this should close this problem (or if not at least minimize it to the max).

ok, and finally as friday 18 of February 2011 the company completed the upgrade just few days ago and fortunately by now there haven't be more problems thus this has been been closed.

sorry for the incovenience and thanks for your support and patience.

regards

Thursday 10 of Frebruary, 2011, at 09:00 - HiszY Servers company network upgrade (status: COMPLETE)

hi

following the reported problem of the previous case (and obviously among other ones they would have too) finally the HiszY Servers company deciede to upgrade their network, this network upgrade (thing that it's absolutely external to HiszY, this is, to what belongs to me) took place today Thursday 10 of Frebruary 2011 and today at night it's still in progress status. it's expected to be completed in the next hours or as much tomorow in teh morning if no further problems arise.

during this time they claim that there shouldn't be mayor problems however obviously there always to be some and that's why today between the 18:15 and 18:45 the servers were inaccessible however surprisingly after this time they recover their operation with normality and altough some servers still experience problems from this situation they were tested and they are working with relatively normality without any further action. although this nigth I'll issue some reboots and checks to make sure that everything is back in place.

ok 3 in the morning and after the reboots and usual checks everything seems as normal as always now I just hope they complete the upgrade tomorrow and then these problems should disapear.

after they complete the upgrade I'll leave this open for some days to check that it doesn't introduce any new problem and if everything went right then it will be closed and with this I hope that this have solved the rare inaccesibility problems like the one described above.

ok, and finally as friday 18 of February 2011 the company has completed the upgrade and fortunately after few days there haven't be more problems and everything seems ok.

I'm sorry for the incoveniences that this situation could cause you, thanks for your support and patience.

regards

March of 2011 - HiszY web and mail services rebuild (status: COMPLETE)

hi

ok, first say that HiszY just really aims to provide a lineup of fresh and independent Quake III Arena game servers so it's not the real HiszY purpose to provide end user internet services like can be a website, http video or downloads just to put an example however at the same time we also understand that this kind of services also form part of the normal develope of a lineup of game servers and their users so in this way HiszY also secondary provide this kind of services but via third party places, like can be this gametracker, youtube or facebook sites for example instead and then HiszY only put them all together inside a HiszY Servers domain just to make them more accesible and representable.

ok, so said that, is important to note that then first HiszY never put many effort in these services and that consequently as being a enchacement all have free character and second that HiszY is only direct reponsible of their domains hiszy-servers.tk and hiszy.com (and also consequently of the pages/sites or other way of spaces at third places that belong to it but even so only limited).

so I didn't really think about make a incident of this however as commented above we also find this services important and since they started around last quarter of 2009 they were slowing deteriorating untill a point that they really needed a total rebuild.

when initially installed it took me about a month and at that time nearly everything worked fine and if not every single time almost, however with the time some free services involved were changing and getting problematic and therefore notabely decreasing the quality of the services. initially only the success rate decreased, this is, each time it was harder to access the site but it used to do most times (just by reloading), later it started to work intermitently and finaly it just didn't even work at all or directly redirected to external inapropiate places that nothing had to do with HiszY, like adult or scam sites and there is when a interventition was mandatory.

ok, about this problem I want to directly apologize for the inconveniences this situation could cause you and state that in no case and under no circunstances I (or anyother part at HiszY) had been doing this with any kind of lucrative or other way compensative purposes or just even intentionally at all, it was a problem with some of the free services involved absolutely external to HiszY, and I wasn't even aware until early this march when after some days of test and margin to see if the problem disapeared is when it was proceded with the total rebuild of these services.

initially it looked like a normal some days task however unfortunately it finally turnt out to be a total rebuild of the entire system and it took me about the last 3 week, nearly the whole 2011 march month. in addition these problems also affected the HiszY mail services that with the time also were losting reliability so a rebuild and improvement of the HiszY mail services were also performed, so in this time is possible that some mails were lost (although even with problems I kept reciving mails, an mail that I'll inmediately handle now rigth after this) then if you think that this were the case please resubmit them to me at the usual contact address admin@hiszy-servers.tk.

so as almost april of 2011 this rebuild of both the web and the mail services has been completed, it's suposed that theoricaly now they should be very reliable in the order of just around 1 percent of error and that they should be much more optimized and thus fast however tech aside I'll still continue to closely monitoring this and also if you find any kind of problem with please contact.

in addition with the rebuild also an automatic mail acknowledgement system has been added so this way each time you write me a mail you should recieve a confirmation if the message has been successfully recieve or not and then can know if it needs to be retransmited. the typical deliver time of a mail is about some minutes then if in ten minutes or so you didn't recieve any acknowleadgement try retransmiting.

ok, finally after more than four months of testing this as of august of 2011 this can be considered finally closed, while is still not perfect I don't really think that is actually possible to build a better system like this of free stuff. everything should be generaly working fine from now however is still to expect that some mails get lost and in additation the previously mentioned automatic confirmation response was removed because it caused problems related with spam at both sides so if you finally mail me and you don't receive a response to your issue in a days term then consider in resubmit your inquire becasue may be it got lost.

I'm sorry for the incoveniences that this situation could cause you, thanks for your support and patience.

regards