Posts Tagged ‘authentication’

How to automate open xen Hypervisor Virtual Machines backups shell script

Tuesday, June 22nd, 2021

openxen-backup-logo As a sysadmin that have my own Open Xen Debian Hypervisor running on a Lenovo ThinkServer few months ago due to a human error I managed to mess up one of my virtual machines and rebuild the Operating System from scratch and restore files service and MySQl data from backup that really pissed me of and this brought the need for having a decent Virtual Machine OpenXen backup solution I can implement on the Debian ( Buster) 10.10 running the free community Open Xen version 4.11.4+107-gef32c7afa2-1. The Hypervisor is a relative small one holding just 7 VM s:

HypervisorHost:~#  xl list
Name                                        ID   Mem VCPUs      State   Time(s)
Domain-0                                     0 11102    24     r—–  214176.4
pcfrxenweb                                  11 12288     4     -b—-  247425.5
pcfrxen                                     12 16384    10     -b—-  1371621.4
windows7                                    20  4096     2     -b—-   97887.2
haproxy2                                    21  4096     2     -b—-   11806.9
jitsi-meet                                  22  2048     2     -b—-   12843.9
zabbix                                      23  2048     2     -b—-   20275.1
centos7                                     24  2040     2     -b—-   10898.2

HypervisorHost:~# xl list|grep -v 'Name ' |grep  -v 'Domain-0'  |wc -l
7


The backup strategy of the script is very simple to shutdown the running VM machine, make a copy with rsync to a backup location the image of each of the Virtual Machines in a bash shell loop for each virtual machine shown in output of xl command and backup to a preset local directory in my case this is /backups/ the backup of each virtual machine is produced within a separate backup directory with a respective timestamp. Backup VM .img files are produced in my case to mounted 2x external attached hard drives each of which is a 4 Terabyte Seagate Plus Backup (Storage). The original version of the script was made to be a slightly different by Zhiqiang Ma whose script I used for a template to come up with my xen VM backup solution. To prevent the Hypervisor's load the script is made to do it with a nice of (nice -n 10) this might be not required or you might want to modify it to better suit your needs. Below is the script itself you can fetch a copy of it /usr/sbin/xen_vm_backups.sh :

#!/bin/bash

# Author: Zhiqiang Ma (http://www.ericzma.com/)
# Modified to work with xl and OpenXen by Georgi Georgiev – https://pc-freak.net
# Original creation dateDec. 27, 2010
# Script takes all defined vms under xen_name_list and prepares backup of each
# after shutting down the machine prepares archive and copies archive in externally attached mounted /backup/disk1 HDD
# Latest update: 08.06.2021 G. Georgiev – hipo@pc-freak.net

mark_file=/backups/disk1/tmp/xen-bak-marker
log_file=/var/log/xen/backups/bak-$(date +%Y_%m_%d).log
err_log_file=/var/log/xen/backups/bak_err-$(date +%H_%M_%Y_%m_%d).log
xen_dir=/xen/domains
xen_vmconfig_dir=/etc/xen/
local_bak_dir=/backups/disk1/tmp
#bak_dir=xenbak@backup_host1:/lhome/xenbak
bak_dir=/backups/disk1/xen-backups/xen_images/$(date +%Y_%m_%d)/xen/domains
#xen_name_list="haproxy2 pcfrxenweb jitsi-meet zabbix windows7 centos7 pcfrxenweb pcfrxen"
xen_name_list="windows7 haproxy2 jitsi-meet zabbix centos7"

if [ ! -d /var/log/xen/backups ]; then
echo mkdir -p /var/log/xen/backups
 mkdir -p /var/log/xen/backups
fi

if [ ! -d $bak_dir ]; then
echo mkdir -p $bak_dir
 mkdir -p $bak_dir

fi


# check whether bak runned last week
if [ -e $mark_file ] ; then
        echo  rm -f $mark_file
 rm -f $mark_file
else
        echo  touch $mark_file
 touch $mark_file
  # exit 0
fi

# set std and stderr to log file
        echo mv $log_file $log_file.old
       mv $log_file $log_file.old
        echo mv $err_log_file $err_log_file.old
       mv $err_log_file $err_log_file.old
        echo "exec 2> $err_log_file"
       exec 2> $err_log_file
        echo "exec > $log_file"
       exec > $log_file


# check whether the VM is running
# We only backup running VMs

echo "*** Check alive VMs"

xen_name_list_tmp=""

for i in $xen_name_list
do
        echo "/usr/sbin/xl list > /tmp/tmp-xen-list"
        /usr/sbin/xl list > /tmp/tmp-xen-list
  grepinlist=`grep $i" " /tmp/tmp-xen-list`;
  if [[ “$grepinlist” == “” ]]
  then
    echo $i is not alive.
  else
    echo $i is alive.
    xen_name_list_tmp=$xen_name_list_tmp" "$i
  fi
done

xen_name_list=$xen_name_list_tmp

echo "Alive VM list:"

for i in $xen_name_list
do
   echo $i
done

echo "End alive VM list."

###############################
date
echo "*** Backup starts"

###############################
date
echo "*** Copy VMs to local disk"

for i in $xen_name_list
do
  date
  echo "Shutdown $i"
        echo  /usr/sbin/xl shutdown $i
        /usr/sbin/xl shutdown $i
        if [ $? != ‘0’ ]; then
                echo 'Not Xen Disk image destroying …';
                /usr/sbin/xl destroy $i
        fi
  sleep 30

  echo "Copy $i"
  echo "Copy to local_bak_dir: $local_bak_dir"
      echo /usr/bin/rsync -avhW –no-compress –progress $xen_dir/$i/{disk.img,swap.img} $local_bak_dir/$i/
     time /usr/bin/rsync -avhW –no-compress –progress $xen_dir/$i/{disk.img,swap.img} $local_bak_dir/$i/
      echo /usr/bin/rsync -avhW –no-compress –progress $xen_vmconfig_dir/$i.cfg $local_bak_dir/$i.cfg
     time /usr/bin/rsync -avhW –no-compress –progress $xen_vmconfig_dir/$i.cfg $local_bak_dir/$i.cfg
  date
  echo "Create $i"
  # with vmmem=1024"
  # /usr/sbin/xm create $xen_dir/vm.run vmid=$i vmmem=1024
          echo /usr/sbin/xl create $xen_vmconfig_dir/$i.cfg
          /usr/sbin/xl create $xen_vmconfig_dir/$i.cfg
## Uncomment if you need to copy with scp somewhere
###       echo scp $log_file $bak_dir/xen-bak-111.log
###      echo  /usr/bin/rsync -avhW –no-compress –progress $log_file $local_bak_dir/xen-bak-111.log
done

####################
date
echo "*** Compress local bak vmdisks"

for i in $xen_name_list
do
  date
  echo "Compress $i"
      echo tar -z -cfv $bak_dir/$i-$(date +%Y_%m_%d).tar.gz $local_bak_dir/$i-$(date +%Y_%m_%d) $local_bak_dir/$i.cfg
     time nice -n 10 tar -z -cvf $bak_dir/$i-$(date +%Y_%m_%d).tar.gz $local_bak_dir/$i/ $local_bak_dir/$i.cfg
    echo rm -vf $local_bak_dir/$i/ $local_bak_dir/$i.cfg
    rm -vrf $local_bak_dir/$i/{disk.img,swap.img}  $local_bak_dir/$i.cfg
done

####################
date
echo "*** Copy local bak vmdisks to remote machines"

copy_remote () {
for i in $xen_name_list
do
  date
  echo "Copy to remote: vm$i"
        echo  scp $local_bak_dir/vmdisk0-$i.tar.gz $bak_dir/vmdisk0-$i.tar.gz
done

#####################
date
echo "Backup finishes"
        echo scp $log_file $bak_dir/bak-111.log

}

date
echo "Backup finished"

 

Things to configure before start using using the script to prepare backups for you is the xen_name_list variable

#  directory skele where to store already prepared backups
bak_dir=/backups/disk1/xen-backups/xen_images/$(date +%Y_%m_%d)/xen/domains

# The configurations of the running Xen Virtual Machines
xen_vmconfig_dir=/etc/xen/
# a local directory that will be used for backup creation ( I prefer this directory to be on the backup storage location )
local_bak_dir=/backups/disk1/tmp
#bak_dir=xenbak@backup_host1:/lhome/xenbak
# the structure on the backup location where daily .img backups with be produced with rsync and tar archived with bzip2
bak_dir=/backups/disk1/xen-backups/xen_images/$(date +%Y_%m_%d)/xen/domains

# list here all the Virtual Machines you want the script to create backups of
xen_name_list="windows7 haproxy2 jitsi-meet zabbix centos7"

If you need the script to copy the backup of Virtual Machine images to external Backup server via Local Lan or to a remote Internet located encrypted connection with a passwordless ssh authentication (once you have prepared the Machines to automatically login without pass over ssh with specific user), you can uncomment the script commented section to adapt it to copy to remote host.

Once you have placed at place /usr/sbin/xen_vm_backups.sh use a cronjob to prepare backups on a regular basis, for example I use the following cron to produce a working copy of the Virtual Machine backups everyday.
 

# crontab -u root -l 

# create windows7 haproxy2 jitsi-meet centos7 zabbix VMs backup once a month
00 06 1,2,3,4,5,6,7,8,9,10,11,12 * * /usr/sbin/xen_vm_backups.sh 2>&1 >/dev/null


I do clean up virtual machines Images that are older than 95 days with another cron job

# crontab -u root -l

# Delete xen image files older than 95 days to clear up space from backup HDD
45 06 17 * * find /backups/disk1/xen-backups/xen_images* -type f -mtime +95 -exec rm {} \; 2>&1 >/dev/null

#### Delete xen config backups older than 1 year+3 days (368 days)
45 06 17 * * find /backups/disk1/xen-backups/xen_config* -type f -mtime +368 -exec rm {} \; 2>&1 >/dev/null

 

# Delete xen image files older than 95 days to clear up space from backup HDD
45 06 17 * * find /backups/disk1/xen-backups/xen_images* -type f -mtime +95 -exec rm {} \; 2>&1 >/dev/null

#### Delete xen config backups older than 1 year+3 days (368 days)
45 06 17 * * find /backups/disk1/xen-backups/xen_config* -type f -mtime +368 -exec rm {} \; 2>&1 >/dev/null

Improve wordpress admin password encryption authentication keys security with WordPress Unique Authentication Keys and Salts

Friday, October 9th, 2020

wordpress-improve-security-logo-linux

Having a wordpress blog or website with an admistrator and access via a Secured SSL channel is common nowadays. However there are plenty of SSL encryption leaks already out there and many of which are either slow to be patched or the hosting companies does not care enough to patch on time the libssl Linux libraries / webserver level. Taking that in consideration many websites hosted on some unmaintained one-time run not-frequently updated Linux servers are still vulneable and it might happen that, if you paid for some shared hosting in the past and someone else besides you hosted the website and forget you even your wordpress installation is still living on one of this SSL vulnerable hosts. In situations like that malicious hackers could break up the SSL security up to some level or even if the SSL is secured use MITM (MAN IN THE MIDDLE) attack to simulate your well secured and trusted SSID Name WIFi network to  redirects the network traffic you use (via an SSL transparent Proxy) to connect to WordPress Administrator Dashbiard via https://your-domain.com/wp-admin. Once your traffic is going through the malicious hax0r even if you haven't used the password to authenticate every time, e.g. you have saved the password in browser and WordPress Admin Panel authentication is achieved via a Cookie the cookies generated and used one time by Woddpress site could be easily stealed one time and later from the vicious 1337 h4x0r and reverse the hash with an interceptor Tool and login to your wordpress …

Therefore to improve the wordpress site security it very important to have configured WordPress Unique Authentication Keys and Salts (known also as the WordPress security keys).

They're used by WordPress installation to have a uniquely generated different key and Salt from the default one to the opened WordPress Blog / Site Admin session every time.

So what are the Authentication Unique Keys and Salts and why they are Used?

Like with almost any other web application, when PHP session is opened to WordPress, the code creates a number of Cookies stored locally on your computer.

Two of the cookies created are called:

 wordpress_[hash]
wordpress_logged_in_[hash]

First  cookie is used only in the admin pages (WordPress dashboard), while the second cookie is used throughout WordPress to determine if you are logged in to WordPress or not. Note: [hash] is a random hashed value typically assigned to your session, therefore in reality the cookies name would be named something like wordpress_ffc02f68bc9926448e9222893b6c29a9.

WordPress session stores your authentication details (i.e. WordPress username and password) in both of the above mentioned cookies.

The authentication details are hashed, hence it is almost impossible for anyone to reverse the hash and guess your password through a cookie should it be stolen. By almost impossible it also means that with today’s computers it is practically unfeasible to do so.

WordPress security keys are made up of four authentication keys and four hashing salts (random generated data) that when used together they add an extra layer to your cookies and passwords. 

The authentication details in these cookies are hashed using the random pattern specified in the WordPress security keys. I will not get into too much details but as you might have heard in Cryptography Salts and Keys are important – an indepth explanation on Salts Cryptography (here). A good reading for those who want to know more on how does the authentication based and salts work is on stackexchange.

How to Set up Salt and Key Authentication on WordPress
 

To be used by WP Salts and Key should be configured under wp-config.php usually they look like so:

wordpress-website-blog-salts-keys-wp-config-screenshot-linux

!!! Note !!!  that generating (manually or generated via a random generator program), the definition strings you have to use a random string value of more than 60 characters to prevent predictability 

The default on any newly installed WordPress Website is to have the 4 definitions with _KEY and the four _SALTs to be unconfigured strings looks something like:

default-WordPress-security-keys-and-salts-entries-in-wordPress-wp-config-php-file

Most people never ever take a look at wp-config.php as only the Web GUI Is used for any maintainance, tasks so there is a great chance that if you never heard specifically by some WordPress Security Expert forum or some Security plugin (such as WP Titan Anti Spam & Security) installed to report the WP KEY / SALT you might have never noticed it in the config.

There are 8 WordPress security keys in current WP Installs, but not all of them have been introduced at the same time.
Historically they were introduced in WP versions in below order:

WordPress 2.6: AUTH_KEY, SECURE_AUTH_KEY, LOGGED_IN_KEY
WordPress 2.7: NONCE_KEY
WordPress 3.0: AUTH_SALT, SECURE_AUTH_SALT, LOGGED_IN_SALT, NONCE_SALT

Setting a custom random generated values is an easy task as there is already online Wordpress Security key Random generator.
You can visit above address and you will get an automatic randomly generated values which could be straight copy / pasted to your wp-config.php.

Howeever if you're a paranoic on the guessability of the random generator algorithm, I would advice you use the generator and change some random values yourself on each of the 8 line, the end result in the configuration should be something similar to:

 

define('AUTH_KEY',         '|w+=W(od$V|^hy$F5w)g6O-:e[WI=NHY/!Ez@grd5=##!;jHle_vFPqz}D5|+87Q');
define('SECURE_AUTH_KEY',  'rGReh.<%QBJ{DP )p=BfYmp6fHmIG~ePeHC[MtDxZiZD;;_OMp`sVcKH:JAqe$dA');
define('LOGGED_IN_KEY',    '%v8mQ!)jYvzG(eCt>)bdr+Rpy5@t fTm5fb:o?@aVzDQw8T[w+aoQ{g0ZW`7F-44');
define('NONCE_KEY',        '$o9FfF{S@Z-(/F-.6fC/}+K 6-?V.XG#MU^s?4Z,4vQ)/~-[D.X0<+ly0W9L3,Pj');
define('AUTH_SALT',        ':]/2K1j(4I:DPJ`(,rK!qYt_~n8uSf>=4`{?LC]%%KWm6@j|aht@R.i*ZfgS4lsj');
define('SECURE_AUTH_SALT', 'XY{~:{P&P0Vw6^i44Op*nDeXd.Ec+|c=S~BYcH!^j39VNr#&FK~wq.3wZle_?oq-');
define('LOGGED_IN_SALT',   '8D|2+uKX;F!v~8-Va20=*d3nb#4|-fv0$ND~s=7>N|/-2]rk@F`DKVoh5Y5i,w*K');
define('NONCE_SALT',       'ho[<2C~z/:{ocwD{T-w+!+r2394xasz*N-V;_>AWDUaPEh`V4KO1,h&+c>c?jC$H');

 


Wordpress-auth-key-secure-auth-salt-Linux-wordpress-admin-security-hardening

Once above defines are set, do not forget to comment or remove old AUTH_KEY / SECURE_AUTH_KEY / LOGGED_IN_KEY / AUTH_SALT / SECURE_AUTH_SALT / LOGGED_IN_SALT /NONCE_SALT keys.

The values are configured one time and never have to be changed, WordPress installation automatic updates or Installed WP Plugins will not tamper the value with time.
You should never expand or show your private generated keys to anyone otherwise this could be used to hack your website site.
It is also a good security practice to change this keys, especially if you have some suspects someone has somehow stolen your wp-onfig keys. 
 

Closure

Having AUTH KEYs and Properly configured is essential step to improve your WordPress site security. Anytime having any doubt for a browser hijacked session (or if you have logged in) to your /wp-admin via unsecured public Computer with a chance of a stolen site cookies you should reset keys / salts to a new random values. Setting the auth keys is not a panacea and frequent WP site core updates and plugins should be made to secure your install. Always do frequent audits to WP owned websites with a tool such as WPScan is essential to keep your WP Website unhacked.

 

 

Stop SSH Bruteforce authentication attempt Attacks with fail2ban

Monday, July 6th, 2020

Fail2ban stop restrict ssh bruteforce authentication attempt attacks

Most of webmasters today have some kind of SSH console remote access to the server and the OpenSSH Secure Shell service is usually not filtered for specific Networks but fully accessbible on the internet. This is especially true for home brew Linux Web servers as well as small to mid sized websites and blogs hosted on a cheap dedicated servers hosted in UK2 / Contabo RackSpace etc.

Brute force password guess attack tools such as Hydra and a distributed password dictionary files have been circulating quite for a while and if the attacker has enough time as well as a solid dictionary base, as well as some kind of relatively weak password you can expect that sooner or later some of the local UNIX accounts can be breaked and the script kiddie can get access to your server and make quickly a havoc, if he is lucky enough to be able to exploit some local vulnerability and get root access …

If you're a sysadmin that has to manage the Linux server and you do a routine log reading on the machine, you will soon get annoyed of the ever growing amount of different users, that are trying to login unsucessfully to the SSH (TCP port 22) service filling up the logs with junk and filling up disk space for nothing as well as consuming some CPU and Memory resources for nothing, you will need some easy  solution to make brute force attacks from an IP get filtered after few unsuccessful login attempts.
 

The common way to protect SSH would then be to ban an IP address from logging in if there are too many failed login attempts based on an automatic firewall inclusion of any IP that tried to unsuccessfully login lets say 5 or 10 reoccuring times.

In Linux there is a toll called “fail2ban” (F2B) used to limit brute force authentication attempts.

F2B works with minimal configuration and besides being capable of protecting the SSH service, it can be set to protect a lot of other Server applications like;

 Apache / NGINX Web Servers with PHP / Mail Servers (Exim, Postfix, Qmail, Sendmail), POP3 IMAP / AUTH services (Dovecot, Courier, Cyrus), DNS Bind servers, MySQL DBs, Monitoring tools such as Nagios, FTP servers (ProFTPD / PureFTP), Proxy servers (Squid), WordPress sites (wp-login) brute force attacks, Web Mail services (Horde / Roundcube / OpenWebmail), jabber servers etc.

To get a better overview below is F2B package description:
 

linux:~# apt-cache show fail2ban|grep -i description-en -A 21
Description-en: ban hosts that cause multiple authentication errors
 Fail2ban monitors log files (e.g. /var/log/auth.log,
 /var/log/apache/access.log) and temporarily or persistently bans
 failure-prone addresses by updating existing firewall rules.  Fail2ban
 allows easy specification of different actions to be taken such as to ban
 an IP using iptables or hostsdeny rules, or simply to send a notification
 email.
 .
 By default, it comes with filter expressions for various services
 (sshd, apache, qmail, proftpd, sasl etc.) but configuration can be
 easily extended for monitoring any other text file.  All filters and
 actions are given in the config files, thus fail2ban can be adopted
 to be used with a variety of files and firewalls.  Following recommends
 are listed:
 .
  – iptables/nftables — default installation uses iptables for banning.
    nftables is also suported. You most probably need it
  – whois — used by a number of *mail-whois* actions to send notification
    emails with whois information about attacker hosts. Unless you will use
    those you don't need whois
  – python3-pyinotify — unless you monitor services logs via systemd, you
    need pyinotify for efficient monitoring for log files changes

 

Using fail2ban is easy as there is a multitude of preexisting filters and actions (that gets triggered on a filter match) already written by different people usually found in /etc/fail2ban/filter.d an /etc/fail2ban/action.d.
as well as custom action / filter / jails is easy to do.
Fail2ban is available as a standard distro RPM / DEB package
on most modern versions of Ubuntu (16.04 and later), Debian, Mint and CentOS 7, OpenSuSE, Fedora etc.
 

1. Install Fail2ban package


– On Deb based distros do the usual:

 

linux:~# apt update
linux:~# apt install –yes fail2ban


On RPM distros Fedora / CentOS / SuSE etc.
 

linux:~# sudo yum -y install epel-release
linux:~# sudo yum -y install fail2ban

 

2. Enable fail2ban ban rules for SSH failed authentication filtering

 

Create the file /etc/fail2ban/jail.local :
 

# cat > /etc/fail2ban/jail.local


Paste below content:
 

[DEFAULT]
# Ban hosts for one hour:
bantime = 360
# Override /etc/fail2ban/jail.d/00-firewalld.conf:
banaction = iptables-multiport

 

[sshd]
enabled = true
maxretry = 10
findtime = 43200
# ban for 1 day
bantime = 3600
# ban for 1 day
#bantime = 86400

 

This config will ban for 1 day any IP that tries to access more than 10 unsuccesful times sshd daemon.
This works through IPTABLES indicated by config banaction = iptables-multiport config making fail2ban to automatically add a new iptables block rule valid for 1 day.

To close the file press CTRL + D simulanesly

  • maxretry controls the maximum number of allowed retries.
  • findtime specifies the time window (in seconds) which should be considered for banning an IP. (43200 seconds is 12 hours)
  • bantime specifies the time window (in seconds) for which the IP address will be banned (86400 seconds is 24 hours).

If SSH listens to a different port from 22 on the machine, you can specify the port number with port = <port_number> in this file.

 

3. Start fail2ban service


Either use the /etc/init.d/fail2ban start script or systemd systemctl
 

linux:~# systemctl enable fail2ban
linux:~# systemctl restart fail2ban

 

4. Fail2ban operational principle in short


Fail2ban uses Filters, Actions And Jails:

Filtersspecify certain patterns of text that Fail2ban should recognize in log files.
Actionsare things Fail2ban can do once a filter is matched.
Jailstell Fail2ban to match a filter on some logs. When the number of matches goes beyond a certain limit specified in the jail, Fail2ban takes an action specified in the jail.

If still wonder what is Fail2ban jail? Each configured jail tells fail2ban to look at system logs and take actions against attacks on a configured service, in our case OpenSSH service.
 

5. Blocking repeated unsuccessful password authentication attempts for longer periods


If more than number of failed ssh logins happen to occur (lets say 35 reoccuring ones in /var/log/auth.log (the debian failed ssh login file) or /var/log/secure (redhat distros failed ssh log file).
You will perhaps want to permanently block this IP for 3 days or so, here is how:
 

banaction = iptables-multiport
maxretry  = 35
findtime  = 259200
bantime   = 608400
enabled   = true
filter    = sshd

 

Add this to [sshlongterm] section to make it finally look like this:
 

[sshlongterm]
port      = ssh
logpath   = %(sshd_log)s
banaction = iptables-multiport
maxretry  = 35
findtime  = 259200
bantime   = 608400
enabled   = true
filter    = sshd


Of course to load new config restart fail2ban
 

# /etc/init.d/fail2ban restart


Default jail as well as the sshlongterm jail should now work together. Short term attacks will be handled by the default jail under [sshd], and the long term attacks handled by our own second jail [sshlongterm].
 

6. Checking which intruders were blocked by fail2ban


Fail2ban creates a separate chain f2b-sshd to which it adds each blocked IP for the period of time preset in the config, to list it:

linux:~# /sbin/iptables -L f2b-sshd
Chain f2b-sshd (1 references)
target     prot opt source               destination         
REJECT     all  —  165.22.2.95          anywhere             reject-with icmp-port-unreachable
REJECT     all  —  ec2-13-250-58-46.ap-southeast-1.compute.amazonaws.com  anywhere             reject-with icmp-port-unreachable
REJECT     all  —  cable200-116-3-133.epm.net.co  anywhere             reject-with icmp-port-unreachable
REJECT     all  —  mail.antracite.org   anywhere             reject-with icmp-port-unreachable
REJECT     all  —  139.59.57.2          anywhere             reject-with icmp-port-unreachable
REJECT     all  —  111.68.98.152.pern.pk  anywhere             reject-with icmp-port-unreachable
REJECT     all  —  216.126.59.61        anywhere             reject-with icmp-port-unreachable
REJECT     all  —  104.248.4.138        anywhere             reject-with icmp-port-unreachable
REJECT     all  —  210.211.119.10       anywhere             reject-with icmp-port-unreachable
REJECT     all  —  85.204.118.13        anywhere             reject-with icmp-port-unreachable
REJECT     all  —  broadband.actcorp.in  anywhere             reject-with icmp-port-unreachable
REJECT     all  —  104.236.33.155       anywhere             reject-with icmp-port-unreachable
REJECT     all  —  114.ip-167-114-114.net  anywhere             reject-with icmp-port-unreachable
REJECT     all  —  142.162.194.168.rfc6598.dynamic.copelfibra.com.br  anywhere             reject-with icmp-port-unreachable
REJECT     all  —  server.domain.cl     anywhere             reject-with icmp-port-unreachable
REJECT     all  —  ip202.ip-51-68-251.eu  anywhere             reject-with icmp-port-unreachable
REJECT     all  —  188.166.185.157      anywhere             reject-with icmp-port-unreachable
REJECT     all  —  93.49.11.206         anywhere             reject-with icmp-port-unreachable
RETURN     all  —  anywhere             anywhere            

 

Conclusion


What we have seen here is how to make fail2ban protect Internet firewall unrestricted SSHD Service to filter out 1337 skript kiddie 'hackers' out of your machine. With a bit of tuning could not only break the occasional SSH Brute Force bot scanners that craw the new but could even mitigate massive big botnet initiated brute force attacks to servers.
Of course fail2ban is not a panacea and to make sure you won't get hacked one days better make sure to only allow access to SSH service only for a certain IP addresses or IP address ranges that are of your own PCs.

How to SSH client Login to server with password provided from command line as a script argument – Running same commands to many Linux servers

Friday, March 6th, 2015

ssh-how-to-login-with-password-provided-from-command-line-use-sshpass-to-run-same-command-to-forest-of-linux-servers

Usually admins like me who casuanlly need to administer "forests" (thousands of identicallyconfigured services Linux servers) are generating and using RSA / DSA key authentication for passwordless login, however this is not always possible as some client environments does prohibit the use of RSA / DSA non-pass authentication, thus in such environments to make routine server basic package rpm / deb upgrades or do other maintanance patching its necessery to use normal ssh user / pass login but as ssh client doesn't allow password to be provided from prompt for security reasons and therefore using some custom bash loop to issue single command to many servers (such as explained in my previous article) requires you to copy / paste password on password prompt multiple times. This works its pretty annoying so if you want to run single command on all your 500 servers with specifying the password from password prompt use sshpass tool (for non-interactive ssh password auth).

SSHPASS official site description:
 

sshpass is a utility designed for running ssh using the mode referred to as "keyboard-interactive" password authentication, but in non-interactive mode.

 

Install sshpass on Debian / Ubuntu (deb based) Linux

sshpass is installable right out of regular repositories so to install run:
 

apt-get install —yes sshpass


Install sshpass on CentOS / Fedora (RPM based) Linux


sshpass is available also across most RPM based distros too so just use yum package manager

 

yum -y install sshpass


If its not available across standard RPM distro provided repositories, there should be RPM on the net for distro just download latest one and use wget and rpm to install:

 wget -q http://dl.fedoraproject.org/pub/epel/6/x86_64/sshpass-1.05-1.el6.x86_64.rpm

 rpm -ivh sshpass-1.05-1.el6.x86_64.rpm

 

How Does SshPass Works?

 

Normally openssh (ssh) client binary uses direct TTY (/dev/tty)= an abbreviation for PhyTeleTYpewriter or (the admin jargon call Physical Console access)  instead of standard remotely defined /dev/ptsVirtual PTY.
To get around this Sshpass runs ssh in a dedicated TTY to emulate the password is indeed issues by interactive keyboard user thus  fooling remote sshd server to thinking password
is provided by interactive user.


SSHPass use

Very basic standard use which allows you to pass the password from command line is like this:
 

sshpass -p 'Your_Password_Goes_here123' ssh username@server.your-server.com


Note that the server you're working is shared with other developers they might be able to steal your username / password by using a simple process list command such as:
 

 ps auxwwef


In my case security is not a hot issue, as I'm the only user on the server (and only concern might be if someone hacks into the server 🙂 

 

Then assuming that you have a plain text file with all your administered servers, you can easily use sshpass in a Bash Script loop in order to run, lets say a package upgrade across all identical Linux version machines:
 

while read line; do
sshpass -p 'Your_Password_Goes_here123' ssh username@$line "apt-get update && apt-get upgrade && apt-get dist-upgrade" < /dev/null;
done < all_servers_list.txt

Change the command you like to issue across all machines with the string "apt-get …"
Above command can be used to keep up2date all Debian stable server packages. What you will do on servers is up to your imaginations, very common use of above line would be if you want to see uptime /netstat command output across all your network servers.

 

while read line; do
sshpass -p 'Your_Password_Goes_here123' ssh username@$line "uptime; who; netstat -tunlp; " < /dev/null;
done < all_servers_list.txt

 


As you can guess SshPass is swiss army knife tool for admins whoneed to automate things with scripts simultaneously across number of servers.
 

Happy SSH-ing 🙂

 

 

 

Preserve Session IDs of Tomcat cluster behind Apache reverse proxy / Sticky sessions with mod_proxy and Tomcat

Wednesday, February 26th, 2014

apache_and_tomcat_merged_logo_prevent_sticky_sessions
Having a combination of Apache webservice Reverse Proxy to redirect invisibly traffic to a number of Tomcat server positioned in a DMZ is a classic task in big companies Corporate world.
Hence if you work for company like IBM or HP sooner or later you will need to configure Apache Webserver cluster with few running Jakarta Tomcat Application servers behind. Scenario with necessity to access a java based application via Tomcat which requires logging (authentication) relaying on establishing and keeping a session ID is probably one of the most common ones and if you do it for first time you will probably end up with Session ID issues.  Session ID issues are hard to capture at first as on first glimpse application will seem to be working but users will have to re-login all the time even though the programmers might have coded for a session to expiry in 30 minutes or so.

… I mean not having configured Session ID prevention to Tomcats will cause random authentication session expiries and users using the Tomcat app will be unable to normally access below application with authenticated credentials. The solution to these is known under term "Sticky sessions"
To configure Sticky sessions you need to already have configured Apache/s with following minimum configuration:

  • enabled mod_proxy, proxy_balancer_module, proxy_http_module and or mod_proxy_ajp (in Apache config)

  LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_balancer_module modules/mod_proxy_balancer.so
LoadModule proxy_http_module modules/mod_proxy_http.so

  • And configured and tested Tomcats running an Application reachable via AJP protocol

Below example assumes there is Reverse Proxy Load Balancer Apache which has to forward all traffic to 2 tomcats. The config can easily be extended for as many as necessary by adding more BalancerMembers.

In Apache webserver (apache2.conf / httpd.conf) you need to have JSESSIONID configured. These JSESSIONID is going to be appended to each client request from Reverse Proxy to each of Tomcat servers with value opened once on authentication to first Tomcat node to each of the other ones.

<Proxy balancer://mycluster>
BalancerMember ajp://10.16.166.53:11010/ route=delivery1
BalancerMember ajp://10.16.166.66:11010/ route=delivery2
</Proxy>

ProxyRequests Off
ProxyPass / balancer://mycluster/ stickysession=JSESSIONID
ProxyPassReverse / balancer://mycluster/

The two variables route=delivery1 and route=delivery2 are routed to hosts identificators that also has to be present in Tomcat server configurations
In Tomcat App server First Node (server.xml)

<Engine name="Catalina" defaultHost="localhost" jvmRoute="delivery1">

In Tomcat App server Second Node (server.xml)

<Engine name="Catalina" defaultHost="localhost" jvmRoute="delivery2">

Once Sticky Sessions are configured it is useful to be able to track they work fine this is possible through logging each of established JESSSIONIDs, to do so add in httpd.conf

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"\"%{JSESSIONID}C\"" combined

After modifications restart Apache and Tomcat to load new configs. In Apache access.log the proof should be the proof that sessions are preserved via JSESSIONID, there should be logs like:
 

127.0.0.1 - - [18/Sep/2013:10:02:02 +0800] "POST /examples/servlets/servlet/RequestParamExample HTTP/1.1" 200 662 "http://localhost/examples/servlets/servlet/RequestParamExample" "Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130807 Firefox/17.0""B80557A1D9B48EC1D73CF8C7482B7D46.server2"

127.0.0.1 - - [18/Sep/2013:10:02:06 +0800] "GET /examples/servlets/servlet/RequestInfoExample HTTP/1.1" 200 693 "http://localhost/examples/servlets/" "Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130807 Firefox/17.0""B80557A1D9B48EC1D73CF8C7482B7D46.server2"

That should solve problems with mysterious session expiries 🙂

phpMyAdmin No activity within 1440 seconds; please log in again Fix

Friday, July 5th, 2013

phpmyadmin no activity within 1440 seconds please log in again screenshot Debian Gnu Linux
I had some complains from Web Developers who constantly was working on a Testing Web Development server. That their opened PhpMyadmin in browser is often closing opened session (auto logging out) with an error:
 

No activity within 1440 seconds; please log in again

This message was driving crazy people, as often they code something in PHP and design a new table or something and refreshing in browser blocked their work flow process with this annoying error …

Thanksfully there is an easy fix to that, just raise the time limit via /etc/phpmyadmin/config.inc.php

First its necessary to enable cookies authentication (by default it is commented):

Line:

//$cfg['Servers'][$i]['auth_type'] = 'cookie';

should be:

$cfg['Servers'][$i]['auth_type'] = 'cookie';

PHPMyAdmin 1140 seconds (24 minutes) timeout behavior behavior is controlled through variable: cfg['LoginCookieValidity']
Also it is necessary to increase timeout from server php.ini  (in Debian and Ubuntu via /etc/php5/apache2/php.ini or in CentOS / RHEL / Fedora Linux by editting /etc/php.ini and changing 1h session expiry setting:

session.gc_maxlifetime = 3600

to

(60*60*8  = 28800 – 8 hrs)

session.gc_maxlifetime = 28800

By default cfg['LoginCookieValidity'] is omitted from config.inc.php so you have to insert it at end of file.

A reasonable timeout value is 8 hours. To change PhPMyadmin Login TimeOut to 8 hours:

$cfg['LoginCookieValidity'] = 60 * 60 * 8; // in seconds (8 hours)

If you want to make Timeout Expire almost never (and you don't care about security) set it to some extra high timeout like 1 year  🙂

$cfg['LoginCookieValidity'] = 3600 * 24 * 365; // 1 year