Although comparatively rare, one of the most disruptive security events that can occur to an organisation is a denial of service (DOS) attack where network or computer resources are deliberately consumed to deny their legitimate use.
Janet’s high capacity means that attacks are unlikely to cause disruption to the core network. Successful attacks have been identified at the edges of the network where the resources of a single customer become a limiting factor. Unfortunately due to the disruption they have caused to high profile companies, DOS attacks gain a lot of
press coverage. DOS attacks are a favourite tool of the hacker collective “anonymous”.
There is a trend towards labeling any situation where symptoms similar to a DOS are labeled as an ?attack? without a complete analysis of the situation. The assumption that the situation must be adversarial can lead to panic and mistakes. Before any corrective controls are used to contain the situation, it is critical that the nature of the problem is fully understood. Here we use some examples from actual incidents illustrate some troubleshooting techniques that we have
found useful.
A key capability when investigating DOS attacks is the ability to inspect network traffic. Only through traffic analysis can you determine the properties that will allow you to filter the attack from legitimate traffic. In one incident a customer was receiving huge volumes of traffic that suddenly overwhelmed a service they were offering, and asked for our help in filtering this traffic. Unfortunately our netflow data provided no coverage of the problem. By helping the customer to configure a mirror port on their switch they were able to capture a brief sample of the traffic to their service. Analysis revealed that the problem traffic was arriving from another Janet customer who was a legitimate user of the system. Their client software had suffered from a bug where it was failing to connect to the service but immediately and aggressively trying to interrogate the service, causing the flood of traffic. Isolating this one user from the service restored the service immediately.
A second site called us with similar symptoms of a DOS attack. They had lost all external connectivity and their border router was so overloaded that even the console was unresponsive. One of the first tools we use to investigate a DOS is Janet’s Netsight. At a glance we can view a customer’s current and historical bandwidth usage and get a high level view of the situation.
It was clear from the graphs that their connection was suffering problems. Pings were not being returned indicating that there was an issue somewhere, but use of their connection was almost nothing. This combination is not typical of an external DOS attack on a network where we would expect higher traffic levels. The customer was unable to configure a mirror port on their switch and needed to get their network up and running quickly as the problem was disrupting services across the entire campus.
The seemingly drastic but effective way to trace and isolate the traffic with no sophisticated monitoring tools available was simply to disconnect cables from the interfaces on the router until network performance returned to normal. The last cable to be disconnected indicates the origin of the problem. By following these steps on other devices where necessary until more subtle tools are available, the problem can be traced to it’s source. Disconnecting cables will
disrupt network traffic, but if your network is already down there’s little to lose.
These incidents illustrate why having visibility of network traffic before something goes wrong is critical in quickly resolving DOS attacks. When you can’t see the cause of a problem, conclusions are jumped to. Once you’ve found the cause of a security event it becomes a lot easier to defend against it.