There are few commands I usually use to track if my server is possibly under a Denial of Service attack or under Distributed Denial of Service
Sys Admins who still have not experienced the terrible times of being under a DoS attack are happy people for sure …
1. How to Detect a TCP/IP Denial of Service Attack This are the commands I use to find out if a loaded Linux server is under a heavy DoS attack, one of the most essential one is of course netstat.
To check if a server is under a DoS attack with netstat, it’s common to use:
linux:~# netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n|wc -l
If the output of below command returns a result like 2000 or 3000 connections!, then obviously it’s very likely the server is under a DoS attack.
To check all the IPS currently connected to the Apache Webserver and get a very brief statistics on the number of times each of the IPs connected to my server, I use the cmd:
linux:~# netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n
221 80.143.207.107 233 145.53.103.70 540 82.176.164.36
As you could see from the above command output the IP 80.143.207.107 is either connected 221 times to the server or is in state of connecting or disconnecting to the node.
Another possible way to check, if a Linux or BSD server is under a Distributed DoS is with the list open files command lsof
Here is how lsof can be used to list the approximate number of ESTABLISHED connections to port 80.
linux:~# lsof -i TCP:80
litespeed 241931 nobody 17u IPv4 18372655 TCP server.www.pc-freak.net:http (LISTEN)
litespeed 241931 nobody 25u IPv4 18372659 TCP 85.17.159.89:http (LISTEN)
litespeed 241931 nobody 30u IPv4 29149647 TCP server.www.pc-freak.net:http->83.101.6.41:54565 (ESTABLISHED)
litespeed 241931 nobody 33u IPv4 18372647 TCP 85.17.159.93:http (LISTEN)
litespeed 241931 nobody 34u IPv4 29137514 TCP server.www.pc-freak.net:http->83.101.6.41:50885 (ESTABLISHED)
litespeed 241931 nobody 35u IPv4 29137831 TCP server.www.pc-freak.net:http->83.101.6.41:52312 (ESTABLISHED)
litespeed 241931 nobody 37w IPv4 29132085 TCP server.www.pc-freak.net:http->83.101.6.41:50000 (ESTABLISHED)
Another way to get an approximate number of established connections to let’s say Apache or LiteSpeed webserver with lsof can be achieved like so:
linux:~# lsof -i TCP:80 |wc -l
2100
I find it handy to keep track of above lsof command output every few secs with gnu watch , like so:
linux:~# watch "lsof -i TCP:80"
2. How to Detect if a Linux server is under an ICMP SMURF attack
ICMP attack is still heavily used, even though it’s already old fashioned and there are plenty of other Denial of Service attack types, one of the quickest way to find out if a server is under an ICMP attack is through the command:
server:~# while :; do netstat -s| grep -i icmp | egrep 'received|sent' ; sleep 1; done
120026 ICMP messages received
1769507 ICMP messages sent
120026 ICMP messages received
1769507 ICMP messages sent
As you can see the above one liner in a loop would check for sent and recieved ICMP packets every few seconds, if there are big difference between in the output returned every few secs by above command, then obviously the server is under an ICMP attack and needs to hardened.
3. How to detect a SYN flood with netstat
linux:~# netstat -nap | grep SYN | wc -l
1032
1032 SYNs per second is quite a high number and except if the server is not serving let’s say 5000 user requests per second, therefore as the above output reveals it’s very likely the server is under attack, if however I get results like 100/200 SYNs, then obviously there is no SYN flood targetting the machine 😉
Another two netstat command application, which helps determining if a server is under a Denial of Service attacks are:
server:~# netstat -tuna |wc -l
10012
and
server:~# netstat -tun |wc -l
9606
Of course there also some other ways to check the count the IPs who sent SYN to the webserver, for example:
server:~# netstat -n | grep :80 | grep SYN |wc -l
In many cases of course the top or htop can be useful to find, if many processes of a certain type are hanging around.
4. Checking if UDP Denial of Service is targetting the server
server:~# netstat -nap | grep 'udp' | awk '{print $5}' | cut -d: -f1 | sort |uniq -c |sort -n
The above command will list information concerning possible UDP DoS.
The command can easily be accustomed also to check for both possible TCP and UDP denial of service, like so:
server:~# netstat -nap | grep 'tcp|udp' | awk '{print $5}' | cut -d: -f1 | sort |uniq -c |sort -n
104 109.161.198.86
115 112.197.147.216
129 212.10.160.148
227 201.13.27.137
3148 91.121.85.220
If after getting an IP that has too many connections to the server and is almost certainly a DoS host you would like to filter this IP.
You can use the /sbin/route command to filter it out, using route will probably be a better choice instead of iptables, as iptables would load up the CPU more than simply cutting the route to the server.
Here is how I remove hosts to not be able to route packets to my server:
route add 110.92.0.55 reject
The above command would null route the access of IP 110.92.0.55 to my server.
Later on to look up for a null routed IP to my host, I use:
route -n |grep -i 110.92.0.55
Well hopefully this should be enough to give a brief overview on how, one can dig in his server and find if he is under a Distributed Denial of Service, hope it’s helpful to somebody out there.
Cheers 😉
How to install and configure Jabber Server (Ejabberd) on Debian Lenny GNU / Linux
Wednesday, December 28th, 2011I've recently installed a jabber server on one Debian Lenny server and hence decided to describe my installations steps hoping this would help ppl who would like to run their own jabber server on Debian . After some research of the jabber server softwares available, I decided to install Ejabberd
The reasons I choose Ejabberd is has rich documentation, good community around the project and the project in general looks like one of the best free software jabber servers available presently. Besides that ejabberd doesn't need Apache or MySQL and only depends on erlang programming language.
Here is the exact steps I followed to have installed and configured a running XMPP jabber server.
1. Install Ejabberd with apt
The installation of Ejabberd is standard, e.g.:
debian:~# apt-get --yes install ejabberd
Now as ejabberd is installed, some minor configuration is necessery before the server can be launched:
2. Edit /etc/ejabberd/ejabberd.cfg
Inside I changed the default settings for:
a) Uncomment%%override_acls.. Changed:
%%%% Remove the Access Control Lists before new ones are added.%%%%override_acls.to
%%
%% Remove the Access Control Lists before new ones are added.
%%
override_acls.
b) Admin User from:
%% Admin user
{acl, admin, {user, "", "example.com"}}.
to
%% Admin user
{acl, admin, {user, "admin", "jabber.myserver-host.com"}}.
c) default %% Hostname of example.com to my real hostname:
%% Hostname
{hosts, ["jabber.myserver-host.com"]}.
The rest of the configurations in /etc/ejabberd/ejabberd.cfg can stay like it is, though it is interesting to read it carefully before continuing as, there are some config timings which might prevent the XMPP server from user brute force attacks as well as few other goodies like for example (ICQ, MSN , Yahoo etc.) protocol transports.
3. Add iptables ACCEPT traffic (allow) rules for ports which are used by Ejabberd
The minimum ACCEPT rules to add are:
/sbin/iptables -A INPUT -p tcp -m tcp --dport 22 -j ACCEPT
/sbin/iptables -A INPUT -p tcp -m tcp --dport 5222 -j ACCEPT
/sbin/iptables -A INPUT -p udp -m udp --dport 5222 -j ACCEPT
/sbin/iptables -A INPUT -p tcp -m tcp --dport 5223 -j ACCEPT
/sbin/iptables -A INPUT -p udp -m udp --dport 5223 -j ACCEPT
/sbin/iptables -A INPUT -p tcp -m tcp --dport 5269 -j ACCEPT
/sbin/iptables -A INPUT -p udp -m udp --dport 5269 -j ACCEPT
/sbin/iptables -A INPUT -p tcp -m tcp --dport 5280 -j ACCEPT
/sbin/iptables -A INPUT -p udp -m udp --dport 5280 -j ACCEPT
/sbin/iptables -A INPUT -p tcp -m tcp --dport 4369 -j ACCEPT
/sbin/iptables -A INPUT -p udp -m udp --dport 4369 -j ACCEPT
/sbin/iptables -A INPUT -p tcp -m tcp --dport 53873 -j ACCEPT
Of course if there is some specific file which stores iptables rules or some custom firewall these rules has to be added / modified to fit appropriate place or chain.
4. Restart ejabberd via init.d script
debian:~# /etc/init.d/ejabberd restart
Restarting jabber server: ejabberd is not running. Starting ejabberd.
5. Create ejabberd necessery new user accounts
debian:~# /usr/sbin/ejabberdctl register admin jabber.myserver-host.com mypasswd1
etc.debian:~# /usr/sbin/ejabberdctl register hipo jabber.myserver-host.com mypasswd2
debian:~# /usr/sbin/ejabberdctl register newuser jabber.myserver-host.com mypasswd3
debian:~# /usr/sbin/ejabberdctl register newuser1 jabber.myserver-host.com mypasswd4
...
ejabberdctl ejabberd server client (frontend) has multiple other options and the manual is a good reading.
One helpful use of ejabberdctl is:
debian:~# /usr/sbin/ejabberdctl status
Node ejabberd@debian is started. Status: started
ejabberd is running
ejabberctl can be used also to delete some existent users, for example to delete the newuser1 just added above:
debian:~# /usr/sbin/ejabberdctl unregister newuser jabber.myserver-host.com
6. Post install web configurations
ejabberd server offers a web interface listening on port 5280, to access the web interface right after it is installed I used URL: http://jabber.myserver-host.com:5280/admin/
To login to http://jabber.myserver-host.com:5280/admin/ you will need to use the admin username previously added in this case:
admin@jabber.myserver-host.com mypasswd1
Anyways in the web interface there is not much of configuration options available for change.
7. Set dns SRV records
I'm using Godaddy 's DNS for my domain so here is a screenshot on the SRV records that needs to be configured on Godaddy:
In the screenshto Target is the Fually qualified domain hostname for the jabber server.
Setting the SRV records for the domain using Godaddy's DNS could take from 24 to 48 hours to propagate the changes among all the global DNS records so be patient.
If instead you use own custom BIND DNS server the records that needs to be added to the respective domain zone file are:
_xmpp-client._tcp 900 IN SRV 5 0 5222 jabber.myserver-host.com.
_xmpp-server._tcp 900 IN SRV 5 0 5269 jabber.myserver-host.com.
_jabber._tcp 900 IN SRV 5 0 5269 jabber.myserver-host.com.
8. Testing if the SRV dns records for domain are correct
debian:~$ nslookup
> set type=SRV
> jabber.myserver-host.com
...
> myserver-host.com
If all is fine above nslookup request should return the requested domain SRV records.
You might be wondering what is the purpose of setting DNS SRV records at all, well if your jabber server has to communicate with the other jabber servers on the internet using the DNS SRV record is the way your server will found the other ones and vice versa.
DNS records can also be checked with dig for example
$ dig SRV _xmpp-server._tcp.mydomain.net
[…]
;; QUESTION SECTION:
;_xmpp-server._tcp.mydomain.net. IN SRV
;; ANSWER SECTION:
_xmpp-server._tcp.mydomain.net. 259200 IN SRV 5 0 5269 jabber.mydomain.net.
;; ADDITIONAL SECTION:
jabber.mydomain.net. 259200 IN A 11.22.33.44
;; Query time: 109 msec
;; SERVER: 212.27.40.241#53(212.27.40.241)
;; WHEN: Sat Aug 14 14:14:22 2010
;; MSG SIZE rcvd: 111
9. Debugging issues with ejabberd
Ejabberd log files are located in /var/log/ejabberd , you will have to check the logs in case of any issues with the jabber XMPP server. Here is the three files which log messages from ejabberd:
debian:~$ ls -1 /var/log/ejabberd/
ejabberd.log
erl_crash.dump
sasl.log
I will not get into details on the logs as the best way to find out about them is to read them 😉
10. Testing ejabberd server with Pidgin
To test if my Jabber server works properly I used Pidgin universal chat client . However there are plenty of other multiplatform jabber clients out there e.g.: Psi , Spark , Gajim etc.
Here is a screenshot of my (Accounts -> Manage Accounts -> Add) XMPP protocol configuration
Tags: admin, apache, best free software, brute force, cfg, com, configure, custom, default hostname, default settings, DNS, doesn, dport, ejabberd, ejabberdctl, exact steps, file, GNU, goodies, hostname, hosts, init, INPUT, Install, installation, Jabber, Linux, Lists, mypasswd, necessery, override, ports, ppl, programming language, Protocol, quot, quot quot, rich documentation, server, servers, software, tcp, transports, uncomment, User, xmpp
Posted in Linux, System Administration, Various | 8 Comments »