How to prevent DOS (denial of service) attack?

Let us assume you are hosting a web-application using a Apache Server on a Linux machine. With this configuration you can use one of these to prevent such an attack:

(i) IP Tables – this works at very low network level.

(ii) Mod Security module (with apache) – this ofcourse works at web-server level.

(iii) Mod Evasive (with apache) – this ofcourse works at web-server level. This is closest to what you may need.

(iv) Dos Deflate – this is a script that runs regularly to find offenders and block them via IP Tables.

IP Tables

IP Tables is a Linux module which is normally available by default on Linux machines, if not; you can always install it. It is capable of analyzing (for source IP Address, destination IP Address, destination Port etc.) incoming packets and doing various things with them, like letting them in, rejecting them etc.

Here are some useful commands.

To log all incoming requests

iptables -I INPUT 1 -j LOG –log-level debug –log-prefix “IPTablesLog: ”
iptables -I OUTPUT 1 -j LOG –log-level debug –log-prefix “IPTablesLog: ”
iptables -I FORWARD 1 -j LOG –log-level debug –log-prefix “IPTablesLog: “

Configure the system log (you need to do this to see logs):

Add this line to /etc/syslog.conf (NOTE: there is a tab between ‘*.debug’ and ‘/var/log/iptables.log’ – and not a space.):
*.debug /var/log/iptables.log

Restart log service:

Restart Syslog
service syslog restart

Flush IP Tables (resets IP tables such that no rules apply):

# iptables -F

Restart IP Tables (any rules added from command line will be lost):

# service iptables restart

Log packets from/to a particular IP:
iptables -I INPUT -s <TheIPHere> -j LOG –log-level debug –log-prefix “IPTablesLog: ”
iptables -I OUTPUT -d <TheIPHere> -j LOG –log-level debug –log-prefix “IPTablesLog: ”
iptables -I FORWARD d <TheIPHere> LOG –log-level debug –log-prefix “IPTablesLog: “

Log Packets for a particular destination port (always use this as the last statement):
iptables -I INPUT -p tcp –dport 80 -j LOG –log-level debug –log-prefix “IPTablesLog: “

Reject Packets from a particular source for a particular destination port:
iptables -I INPUT -s <TheIPHere> -p tcp  –dport 80 -j REJECT –reject-with icmp-port-unreachable

iptables -A INPUT -s 192.168.0.121 -p tcp  –dport 80 -j DROP

Limit Packets for a particular destination port:
iptables -I INPUT -p tcp –dport 80 -m limit –limit 10/sec –limit-burst 10 -j ACCEPT

If you want to permanently add any of the above rules you can add them to the file:/etc/sysconfig/iptables. And then restart iptables. Note that order of the commands could be important.

Mod Security

This is an module that allows you to configure Apache in a variety of useful ways. For example, preventing uploads of a certain kind, preventing access to pages with a certain extension, and so on…

URL: http://www.modsecurity.org
Download: http://www.modsecurity.org/download/direct.html

Most of the information regarding that is available on its website.
In brief the install instructions are:

a.) Download Mod Security gzip .
b.) Unzip it in some dir.
c.) Configure it by issuing command: # ./configure –with-apxs=<APACHE_HOME>/bin/apxs
d.) Compile: # make

e.) Install: # make install

f.) In <APACHE_HOME>/conf/httpd.conf add following lines (the second line is optional):

LoadFile /usr/lib/libxml2.so
#Next line is optional.
#LoadFile /usr/lib/liblua5.1.so

LoadModule security2_module modules/mod_security2.so

g.) Download the Core Rules (these are example config files for mod security)
unzip them such that they land up inside <APACHE_HOME>/conf/modsecurity/

h.) In <APACHE_HOME>/conf/httpd.conf add following lines:
Include conf/modsecurity/*.conf

The most important conf file in that dirs are:
modsecurity_crs_10_config.conf
modsecurity_crs_30_http_policy.conf

You can edit them as you see fit. And rest of the config files you can move to some other sub-dir, if you do not want them loaded.

i.) Now you can restart Apache, and you should be all set.

Now, that you have mod security, you will wonder where is the option for preventing DOS attacks? Infact, Mod Security by itself cannot prevent DOS; but it can do so in conjunction with another tool called ‘httpd guardian’.  See the SecGuardianLog option on this page: http://www.modsecurity.org/documentation/modsecurity-apache/2.1.0/modsecurity2-apache-reference.html .

This particular httpd guardian tool can also be used independently (in which case it analyzes apache logs) or in conjunction with Mod Security in which case it receives information from Mod Security; either way it analyzes input information to determine action to take. The download location of this particular tool is this:

It is available at: http://www.apachesecurity.net/tools/index.html and the download location of httpd-guardian is: http://apache-tools.cvs.sourceforge.net/viewvc/apache-tools/apache-tools/httpd-guardian?view=markup .

Mod Security is probably a great tool, but it does not seem right for the purpose of preventing DOS attacks – mainly because that is really not built into the tool, and it uses a yet another tool (httpd-guardian) to achieve it. So you get the picture, too many pieces involved etc. I have used Mod Security, but not in conjunction with httpd-guardian, so i may not be the best person to pass judgement in this regard.

Mod Evasive

This is the solution that i finally zeroed on and found to be the best solution. It is easy to configure and test.  And preventing DOS attacks is the primary purpose of this tool.

Install Instructions: http://wiki.leenox.in/index.php/Installing_mod_evasive

Download from: http://www.zdziarski.com/projects/mod_evasive/mod_evasive_1.10.1.tar.gz
Compile and Install: #<APACHE_HOME>/bin/apxs -cia mod_evasive20.c

Add to httpd.conf:
<IfModule mod_evasive.c>
DOSHashTableSize 3097
DOSPageCount 2
DOSSiteCount 50
DOSPageInterval 1
DOSSiteInterval 1
DOSBlockingPeriod 300
DOSLogDir “<PathToYourApacheLogDirHere>”
DOSEmailNotify myemal@email.com
</IfModule>

NOTE: Do not forget that even a single web page can have many images and such things, so loading a single page can also cause many requests to be sent to the server(to load images etc). So, you will need to arriveat an appropriate values for the above params yourself.
Explanation of params (this link is useful http://www.directadmin.com/forum/showthread.php?s=&threadid=10957)

DOSHashTableSize
Size of the hash table. The greater this setting, the more memory is required for the look up table, but also the faster the look ups are processed. This option will automatically round up to the nearest prime number.

DOSPageCount
Number of requests for the same page within the ‘DOSPageInterval’ interval that will get an IP address added to the blocking list.

DOSSiteCount
Same as ‘DOSPageCount’, but corresponds to the number of requests for a given site, and uses the ‘DOSSiteInterval’ interval.

DOSPageInterval
Interval for the ‘DOSPageCount’ threshold in second intervals.

DOSSiteInterval
Interval for the ‘DOSSiteCount’ threshold in second intervals.

DOSBlockingPeriod
Blocking period in seconds if any of the thresholds are met. The user will recieve a 403 (Forbidden) when blocked, and the timer will be reset each time the site gets hit when the user is still blocked.

Logs:
You can still see the log messages here whenever a IP Address is blocked:
– <APACHE_HOME>/logs/error_log ( eg: [error] [client <IPAddressHere>] client denied by server configuration: )

– <APACHE_HOME>/logs/<dos_<IP>>( A file with IP Address in its name is created here if an IP Address is blocked)

– /var/log/messages – ( eg: mod_evasive[6480]: Blacklisting address <IPAddressHere>: possible DoS attack)

You can easily test Mod Evasive. Just keep F5 pressed on a web-page of your site (the browser will load the page again and again very fast), and when mod evasive kicks in it will return 403 error.

NOTE: One thing that did not work for me was that, even when a IP Address was correctly identified and blocked, i could still continue to randomly access the web-application.

DOS Deflate

This is a shell script that runs via cron every minute, and checks the processes on that machine. If there are too many http processes initiated by a particular IP Address, it blocks that IP Address using IPTables, and will unblock the IP Address again after some time (based on configuration i.e in ddos.conf file – see the link below).

http://deflate.medialayer.com/ – this is main URL you want to see.

After looking at it, i thought it required one improvement, i.e it should not just check for all processes but should check for httpd processes (assuming u r running Apache ofcourse). For this change the following line in instrall.sh from:

netstat -ntu | awk ‘{print $5}’ | cut -d: -f1 | sort | uniq -c | sort -n

to:

netstat -aptn | grep httpd | awk ‘{print $5}’ | cut -d: -f4 | sort | uniq -c | sort -n

Remember that the concept is quite striaght forward, and you can even write your own java program (if you are unconformatable with shell scripts) to achieve the same.

How do i find that i am under DOS attack?

If you are using Mod Evasive, you can see its logs or you may receive a mail from it if that is configured correctly. But, if you are not using any such thing, read on…

The most obvious thing is that your application will become ‘unexpectedly’ overloaded (and therefore unresponsive). Remember, you could be genuinely overloaded too, because your web-application is so popular! So, one key thing is that it would be ‘unexpected’, another that, most of the requests would seem to come from some specific IPs (which would further confirm that this is an auotmated attack).

On Linux, find out if there are a huge number of http processes:

Find if you have huge number of http processes (>100 is not good):
#ps -aux|grep HTTP|wc -l

#ps -aux|grep HTTP

You have more than 30 connection from a single ip. Under normal cases there is no need for that many number of connection requests from a single IP. Try to identify such ips/networks from the list you get.
#netstat -lpn|grep :80 |awk ‘{print $5}’|sort

If you identify suspicious IPs, you can block them using iptables.

NOTE: In case of DDOS (distributed denial of service) there will be more than one IP that would be suspicious, and it would be much more difficult to stop it too, as the source IP may keep changing.

If you are running an Application Server (say tomcat) – all the threads will be occupied (busy), causing the application to become unresponsive.

Is there a even better and robust solution?

Yes ofcourse, the most robust solution to this problem is to be found in expensive Hardware Load Balancers, or FireWalls.

Advertisements

6 responses

  1. […] server from same ip IPTABLES – Block DoS Attack (UDP) A Comparison of iptables Automation Tools How to prevent DOS (denial of service) attack? Anti-DDos with […]

  2. Very good article, What I think it is better to block the DDOS attack using CSF or APF. Some times if the web site has more no of images and external files included in it then the no of request might be high In such scenario the normal request may also be considered as a fake one.

  3. I wish for once that the world was free of hackers, and spammers. Worthless, shameless son of bitches who don’t deserve to inhabit this beautiful place.

    mod_evasive don’t work so well for DDOS. The request URL is different every time, and it comes from compromised computers all over the place. I can’t even set a good value for DOSSiteCount and DOSSiteInterval because I serve a lot of static content, particularly small images.

    In the end I wrote a ultra-strict web server using select and nonblocking sockets. If a request is invalid or resource doesn’t exists, the server (which is named Infallible) doesn’t even bother to respond. It works real well for a year. Now they start attacking my Red5 server and I’m frustrated. I think I might as well figure out how to protect Tomcat also, and deal with this once for all.

  4. I am new to all of this, so excuse the dumb question:
    I am setting up mod_security and I want to use httpd-guardian. Where do I put the httpd-guardian script? I am using an Amazon Linux (EC2 instance) build – similar to RH – Apache 2.2. I know to use the SecGuardian directive, just stuck on where it goes! Thanks for any help.

  5. Great post! I was a victim of hackers a few months back. Thanks for the advice, it will help greatly!

    1. Good to know this was of help.. Thanks.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: