Wialon Hosting Outages caused by DDoS attacks

On March, 21^st, 2012 Wialon Data-center in the Netherlands underwent DDoS-attacks. Wialon Hosting outages started at 15:55 (Minsk time) and lasted till 20:00. By this time all hardware and communication channels were restored.

It’s important to admit that not the Wialon servers were attacked, but the whole Data-center of our provider. Its routers were overloaded with a huge amount of connections and requests that blocked data from GPS tracking devices and prevented users from Wialon Hosting connection.

Within 4 hours Gurtam specialists and their Netherlands colleagues neutralized the attack and provided safety of all data on our servers, using the built-in data backup. During the last hour we have been debugging procedures for automatic switching between primary and backup routers that would provide greater protection in case of recurrence of such incident.

By 20:00 (Minsk time) our customers were able to connect the system and use Wialon Hosting. It’s important to say that the majority of GPS devices registered in the system has kept their tracking data in their built-in memory and sent it to the server just after the connection had been restored.

All that is clear from the chart showing the number of incoming messages received after the connection had been restored. During 20 peak traffic minutes we managed to test the communication servers’ capacity and performance of other system elements. And they‘ve done their best! During this time they were receiving, processing and writing to the data-base over 15 000 messages per second. In total they had received 10.8 million messages which amounted to 1 GB of data.

During the accident all our servers and Wialon Hosting software continued working in normal mode and haven’t failed. But because of missing external communication all requests were not able to reach it, and that’s why all resources, including load processors (marked yellow on the graph) were close to zero. After routing recovery resource loading on servers reached 82% (while usual rate is 20%).

It’s important to admit that Wialon Hosting isn’t the only well-known web-service harmed by DDos-attacks lately. For example, last year popular service LiveJournal was out of reach for more than one day because of series of DDoS attacks.

Gurtam brings apologizes to all Wialon Hosting operators and users who were unable to log into the monitoring system. We strongly hope this accident won’t happen again.