Posts Tagged ‘root root’

Create and Configure SSL bundle file for GoGetSSL issued certificate in Apache Webserver on Linux

Saturday, November 3rd, 2018

gogetssl-install-certificate-on-linux-howto-sslcertificatechainfile-obsolete

I had a small task to configure a new WildCard SSL for domains on a Debian GNU / Linux Jessie running Apache 2.4.25.

The official documentation on how to install the SSL certificate on Linux given by GoGetSSL (which is by COMODO was obsolete as of time of writting this article and suggested as install instructions:
 

SSLEngine on
SSLCertificateKeyFile /etc/ssl/ssl.key/server.key
SSLCertificateFile /etc/ssl/ssl.crt/yourDomainName.crt
SSLCertificateChainFile /etc/ssl/ssl.crt/yourDomainName.ca-bundle


Adding such configuration to domain Vhost and testing with apache2ctl spits an error like:

 

root@webserver:~# apache2ctl configtest
AH02559: The SSLCertificateChainFile directive (/etc/apache2/sites-enabled/the-domain-name-ssl.conf:17) is deprecated, SSLCertificateFile should be used instead
Syntax OK

 


To make issued GoGetSSL work with Debian Linux, hence, here is the few things done:

The files issued by Gogetssl.COM were the following:

 

AddTrust_External_CA_Root.crt
COMODO_RSA_Certification_Authority.crt
the-domain-name.crt


The webserver had already SSL support via mod_ssl Apache module, e.g.:

 

root@webserver:~# ls -al /etc/apache2/mods-available/*ssl*
-rw-r–r– 1 root root 3112 окт 21  2017 /etc/apache2/mods-available/ssl.conf
-rw-r–r– 1 root root   97 сеп 19  2017 /etc/apache2/mods-available/ssl.load
root@webserver:~# ls -al /etc/apache2/mods-enabled/*ssl*
lrwxrwxrwx 1 root root 26 окт 19  2017 /etc/apache2/mods-enabled/ssl.conf -> ../mods-available/ssl.conf
lrwxrwxrwx 1 root root 26 окт 19  2017 /etc/apache2/mods-enabled/ssl.load -> ../mods-available/ssl.load


For those who doesn't have mod_ssl enabled, to enable it quickly run:

 

# a2enmod ssl


The VirtualHost used for the domains had Apache config as below:

 

 

 

NameVirtualHost *:443

<VirtualHost *:443>
    ServerAdmin support@the-domain-name.com
    ServerName the-domain-name.com
    ServerAlias *.the-domain-name.com the-domain-name.com

    DocumentRoot /home/the-domain-namecom/www
    SSLEngine On
#    <Directory />
#        Options FollowSymLinks
#        AllowOverride None
#    </Directory>
    <Directory /home/the-domain-namecom/www>
        Options Indexes FollowSymLinks MultiViews
        AllowOverride None
        Include /home/the-domain-namecom/www/htaccess_new.txt
        Order allow,deny
        allow from all
    </Directory>

    ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
    <Directory "/usr/lib/cgi-bin">
        AllowOverride None
        Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
        Order allow,deny
        Allow from all
    </Directory>

    ErrorLog ${APACHE_LOG_DIR}/error.log

    # Possible values include: debug, info, notice, warn, error, crit,
    # alert, emerg.
    LogLevel warn

    CustomLog ${APACHE_LOG_DIR}/access.log combined

#    Alias /doc/ "/usr/share/doc/"
#   <Directory "/usr/share/doc/">
#       Options Indexes MultiViews FollowSymLinks
#       AllowOverride None
#       Order deny,allow
#       Deny from all
#       Allow from 127.0.0.0/255.0.0.0 ::1/128
#   </Directory>
SSLCertificateKeyFile /etc/apache2/ssl/the-domain-name.com.key
SSLCertificateFile /etc/apache2/ssl/chain.crt

 

</VirtualHost>

The config directives enabling and making the SSL actually work are:
 

SSLEngine On
SSLCertificateKeyFile /etc/apache2/ssl/the-domain-name.com.key
SSLCertificateFile /etc/apache2/ssl/chain.crt

 

The chain.crt file is actually a bundle file containing a bundle of the gogetssl CA_ROOT and RSA_Certification_Authority 3 files, to prepare that file, I've used bundle.sh small script found on serverfault.com here I've made a mirror of bundle.sh on pc-freak.net here   the script content is as follows:

To prepare the chain.crt  bundle, I ran:

 

sh create-ssl-bundle.sh _iq-test_cc.crt chain.crt
sh create-ssl-bundle.sh _iq-test_cc.crt >chain.crt
sh create-ssl-bundle.sh COMODO_RSA_Certification_Authority.crt >> chain.crt
sh create-ssl-bundle.sh bundle.sh AddTrust_External_CA_Root.crt >> chain.crt


Then I copied the file to /etc/apache2/ssl together with the-domain-name.com.key file earlier generated using openssl command earlier explained in my article how to install RapidSSL certificate on Linux

/etc/apache2/ssl was not previously existing (on Debian Linux), so to create it:

 

root@webserver:~# mkdir /etc/apache2/ssl
root@webserver:~# ls -al /etc/apache2/ssl/chain.crt
-rw-r–r– 1 root root 20641 Nov  2 12:27 /etc/apache2/ssl/chain.crt
root@webserver:~# ls -al /etc/apache2/ssl/the-domain-name.com.key
-rw-r–r– 1 root root 6352 Nov  2 20:35 /etc/apache2/ssl/the-domain-name.com.key

 

As I needed to add the SSL HTTPS configuration for multiple domains, further on I've wrote and used a tiny shell script add_new_vhost.sh which accepts as argument the domain name I want to add. The script works with a sample Skele (Template) file, which is included in the script itself and can be easily modified for the desired vhost config.
To add my multiple domains, I've used the script as follows:
 

sh add_new_vhost.sh add-new-site-domain.com
sh add_new_vhost.sh add-new-site-domain1.com


etc.

Here is the complete script as well:

 

#!/bin/sh
# Shell script to add easily new domains for virtual hosting on Debian machines
# arg1 should be a domain name
# This script takes the domain name which you type as arg1 uses it and creates
# Docroot / cgi-bin directory for the domain, create seperate site's apache log directory
# then takes a skele.com file and substitutes a skele.com with your domain name and directories
# This script's aim is to easily enable sysadmin to add new domains in Debian
sites_base_dir=/var/www/jail/home/www-data/sites/;
# the directory where the skele.com file is
skele_dir=/etc/apache2/sites-available;
# base directory where site log dir to be created
cr_sep_log_file_d=/var/log/apache2/sites;
# owner of the directories
username='www-data';
# read arg0 and arg1
arg0=$0;
arg1=$1;
if [[ -z $arg1 ]]; then
echo "Missing domain name";
exit 1;
fi

 

# skele template
echo "#
#  Example.com (/etc/apache2/sites-available/www.skele.com)
#
<VirtualHost *>
        ServerAdmin admin@design.bg
        ServerName  skele.com
        ServerAlias www.skele.com


        # Indexes + Directory Root.
        DirectoryIndex index.php index.htm index.html index.pl index.cgi index.phtml index.jsp index.py index.asp

        DocumentRoot /var/www/jail/home/www-data/sites/skelecom/www/docs
        ScriptAlias /cgi-bin "/var/www/jail/home/www-data/sites/skelecom/cgi-bin"
        
        # Logfiles
        ErrorLog  /var/log/apache2/sites/skelecom/error.log
        CustomLog /var/log/apache2/sites/skelecom/access.log combined
#       CustomLog /dev/null combined
      <Directory /var/www/jail/home/www-data/sites/skelecom/www/docs/>
                Options FollowSymLinks MultiViews -Includes
                AllowOverride None
                Order allow,deny
                allow from all
                # This directive allows us to have apache2's default start page
                # in /apache2-default/, but still have / go to the right place
#               RedirectMatch ^/$ /apache2-default/
        </Directory>

        <Directory /var/www/jail/home/www-data/sites/skelecom/www/docs/>
                Options FollowSymLinks ExecCGI -Includes
                AllowOverride None
                Order allow,deny
                allow from all
        </Directory>

</VirtualHost>
" > $skele_dir/skele.com;

domain_dir=$(echo $arg1 | sed -e 's/\.//g');
new_site_dir=$sites_base_dir/$domain_dir/www/docs;
echo "Creating $new_site_dir";
mkdir -p $new_site_dir;
mkdir -p $sites_base_dir/cgi-bin;
echo "Creating sites's Docroot and CGI directory";
chown -R $username:$username $new_site_dir;
chown -R $username:$username $sites_base_dir/cgi-bin;
echo "Creating site's Log files Directory";
mkdir -p $cr_sep_log_file_d/$domain_dir;
echo "Creating sites's VirtualHost file and adding it for startup";
sed -e "s#skele.com#$arg1#g" -e "s#skelecom#$domain_dir#g" $skele_dir/skele.com >> $skele_dir/$arg1;
ln -sf $skele_dir/$arg1 /etc/apache2/sites-enabled/;
echo "All Completed please restart apache /etc/init.d/apache restart to Load the new virtual domain";

# Date Fri Jan 11 16:27:38 EET 2008


Using the script saves a lot of time to manually, copy vhost file and then edit it to change ServerName directive, for vhosts whose configuration is identical and only the ServerName listener has to change, it is perfect to create all necessery domains, I've created a simple text file with each of the domains and run it in a loop:
 

while :; do sh add_new_vhost.sh $i; done < domain_list.txt
 

 

How to install KVM Kernel-based Virtual Machine Virtualization on Linux

Sunday, October 14th, 2018

install-KVM-Kernel-based-Virtual-Machine-virtualization-on-Linux

If you want to run multiple virtual machines on GNU / Linux server or your Linux powered Desktop you have the possibility to use a couple of Virtual Machines just to name a few VirtualBox and VMWare are the option the native way to do it is using the Linux kernel via a loadable kernel module called KVM (Kernel-based Virtual Machine).
Though Oracle's Virtualbox generally works and you could add new test beds virtual machines (install multiple Linux / *BSD OS) it is not fully Free Software and not even fully open source licensed, VMWare even though superior as a Virtualization product is proprietary and its application costs a lot of money which not each develpoper or small / mid-sized company could afford.

Once the kvm.ko module is loaded your Linux kernel turns into a full-featured Virtual Machine Hypervisor.
Starting with Linux kernel 2.6.X the KVM Hypervisor is available and easy to install virtually all modern Linux distributions Redhat / CentOS Debian / Ubuntu etc. support it and its up to running few commands to install and start using the Power of Kernel embedded Virtualization.

KVM could be used to run in parallel multiple Operating Systems such as Windows / Linux / FreeBSD and others of BSDs family,  each running under a separate virtual machine with its private dedicated (isolated), disc, graphic card, network card etc.

To start up I assume you have already installed some kind of Linux distribution either locally or on a remote dedicated server.
 

1. Installing KVM on Debian GNU / Linux / Ubuntu / Mint and other deb based distros

 

Using APT tool install below packages:

 

root@jeremiah:~# apt install –yes qemu-kvm libvirt-clients libvirt-daemon-system bridge-utils libguestfs-tools genisoimage virtinst libosinfo-bin

 

2. Installing virt-manager GUI to manage Virtual servers

 

root@jeremiah:~# apt-cache show virt-manager|grep -i desc -A 1
Description-en: desktop application for managing virtual machines
 It presents a summary view of running domains and their live performance &

Description-md5: 9f7f584744b77cdacc2291f2a8ac220e
Homepage: http://virt-manager.et.redhat.com/

 

root@jeremiah:~# apt install –yes virt-manager

 


virtual-manager-kvm-gnu-linux-virtual-machines-cpu-hdd-load-statistics-screenshot

 

 

virtual-manager-fedora-28-linux-virtual-machine-settings-screenshot


3. Configure bridged networking to allow access to newly configured VMs

Bridging has to be added via /etc/network/interfaces therefore it is a good idea to create a backup of it before modifying:

 

# cp -rpf /etc/network/interfaces /etc/network/interfaces.bakup-$(echo $(date '+%Y-%m-%d-%H'))

 

# vim /etc/network/interfaces

auto br0
 iface br0 inet static
         address 10.15.44.26
         netmask 255.255.255.192
         broadcast 10.15.44.63
         dns-nameservers 10.0.80.11 10.0.80.12
         # set static route for LAN
      post-up route add -net 10.0.0.0 netmask 255.0.0.0 gw 10.18.44.1
      post-up route add -net 161.26.0.0 netmask 255.255.0.0 gw 10.18.44.1
         bridge_ports eth0
         bridge_stp off
         bridge_fd 0
         bridge_maxwait 0
 
 # br1 setup with static wan IPv4 with ISP router as a default gateway
 auto br1
 iface br1 inet static
         address 192.168.222.51
         netmask 255.255.255.248
         broadcast 192.168.222.55
         gateway 192.168.222.49
         bridge_ports eth1
         bridge_stp off
         bridge_fd 0
         bridge_maxwait 0

 

Once file is saved in vim editor restart the networking.

 

# systemctl restart network.manager

 

To verify whether the bridge has been succesfully upped.

 

root@jeremiah:/home/hipo/kvm# brctl show
bridge name    bridge id        STP enabled    interfaces
virbr0        8000.525400cb1cd1    yes        virbr0-nic

 

4. List all installable Virtual OS images
 

root@jeremiah:/home/hipo/kvm# virt-builder -list
centos-6                 x86_64     CentOS 6.6
centos-7.0               x86_64     CentOS 7.0
centos-7.1               x86_64     CentOS 7.1
centos-7.2               aarch64    CentOS 7.2 (aarch64)
centos-7.2               x86_64     CentOS 7.2
centos-7.3               x86_64     CentOS 7.3
centos-7.4               x86_64     CentOS 7.4
centos-7.5               x86_64     CentOS 7.5
cirros-0.3.1             x86_64     CirrOS 0.3.1
cirros-0.3.5             x86_64     CirrOS 0.3.5
debian-6                 x86_64     Debian 6 (Squeeze)
debian-7                 sparc64    Debian 7 (Wheezy) (sparc64)
debian-7                 x86_64     Debian 7 (wheezy)
debian-8                 x86_64     Debian 8 (jessie)
debian-9                 x86_64     Debian 9 (stretch)
fedora-18                x86_64     Fedora® 18
fedora-19                x86_64     Fedora® 19
fedora-20                x86_64     Fedora® 20
fedora-21                aarch64    Fedora® 21 Server (aarch64)
fedora-21                armv7l     Fedora® 21 Server (armv7l)
fedora-21                ppc64      Fedora® 21 Server (ppc64)
fedora-21                ppc64le    Fedora® 21 Server (ppc64le)
fedora-21                x86_64     Fedora® 21 Server
fedora-22                aarch64    Fedora® 22 Server (aarch64)
fedora-22                armv7l     Fedora® 22 Server (armv7l)
fedora-22                i686       Fedora® 22 Server (i686)
fedora-22                x86_64     Fedora® 22 Server
fedora-23                aarch64    Fedora® 23 Server (aarch64)
fedora-23                armv7l     Fedora® 23 Server (armv7l)
fedora-23                i686       Fedora® 23 Server (i686)
fedora-23                ppc64      Fedora® 23 Server (ppc64)
fedora-23                ppc64le    Fedora® 23 Server (ppc64le)
fedora-23                x86_64     Fedora® 23 Server
fedora-24                aarch64    Fedora® 24 Server (aarch64)
fedora-24                armv7l     Fedora® 24 Server (armv7l)
fedora-24                i686       Fedora® 24 Server (i686)
fedora-24                x86_64     Fedora® 24 Server
fedora-25                aarch64    Fedora® 25 Server (aarch64)
fedora-25                armv7l     Fedora® 25 Server (armv7l)
fedora-25                i686       Fedora® 25 Server (i686)
fedora-25                ppc64      Fedora® 25 Server (ppc64)
fedora-25                ppc64le    Fedora® 25 Server (ppc64le)
fedora-25                x86_64     Fedora® 25 Server
fedora-26                aarch64    Fedora® 26 Server (aarch64)
fedora-26                armv7l     Fedora® 26 Server (armv7l)
fedora-26                i686       Fedora® 26 Server (i686)
fedora-26                ppc64      Fedora® 26 Server (ppc64)
fedora-26                ppc64le    Fedora® 26 Server (ppc64le)
fedora-26                x86_64     Fedora® 26 Server
fedora-27                aarch64    Fedora® 27 Server (aarch64)
fedora-27                armv7l     Fedora® 27 Server (armv7l)
fedora-27                i686       Fedora® 27 Server (i686)
fedora-27                ppc64      Fedora® 27 Server (ppc64)
fedora-27                ppc64le    Fedora® 27 Server (ppc64le)
fedora-27                x86_64     Fedora® 27 Server
fedora-28                i686       Fedora® 28 Server (i686)
fedora-28                x86_64     Fedora® 28 Server
freebsd-11.1             x86_64     FreeBSD 11.1
scientificlinux-6        x86_64     Scientific Linux 6.5
ubuntu-10.04             x86_64     Ubuntu 10.04 (Lucid)
ubuntu-12.04             x86_64     Ubuntu 12.04 (Precise)
ubuntu-14.04             x86_64     Ubuntu 14.04 (Trusty)
ubuntu-16.04             x86_64     Ubuntu 16.04 (Xenial)
ubuntu-18.04             x86_64     Ubuntu 18.04 (bionic)
opensuse-13.1            x86_64     openSUSE 13.1
opensuse-13.2            x86_64     openSUSE 13.2
opensuse-42.1            x86_64     openSUSE Leap 42.1
opensuse-tumbleweed      x86_64     openSUSE Tumbleweed


5. Create Virtual Machine OS-es from scratch with virt-builder

Below we'll create two images one for Fedora 28 and 1 for Debian 9 using the virt-builder (a tool to build virtual images quickly), the images that could be used are shown through below virt-builder –list command.
 

# iso='fedora-28';
# iso1='debian-9';

 

# sudo virt-builder $iso \
     –size=10G \
     –format qcow2 -o /var/lib/libvirt/images/$iso-vm1.qcow2 \
     –hostname $iso-vm1 \
     –network \
     –timezone Europe/Sofia

 

[   3.3] Downloading: http://libguestfs.org/download/builder/fedora-28.xz
[   5.2] Planning how to build this image
[   5.2] Uncompressing
[  20.8] Resizing (using virt-resize) to expand the disk to 10.0G
[  50.8] Opening the new disk
[  53.7] Setting a random seed
[  53.7] Setting the hostname: fedora-28-vm1
[  53.7] Setting the timezone: Europe/Sofia
[  53.7] Setting passwords
virt-builder: Setting random password of root to YMTkxaJIkEU24Ytf

[  54.7] Finishing off
                   Output file: /var/lib/libvirt/images/fedora-28-vm1.qcow2
                   Output size: 10.0G
                 Output format: qcow2
            Total usable space: 9.3G
                    Free space: 8.2G (87%)

 

# sudo virt-builder $iso1 \
     –size=10G \
     –format qcow2 -o /var/lib/libvirt/images/$iso-vm1.qcow2 \
     –hostname $iso1-vm1 \
     –network \
     –timezone Europe/Sofia

 

[   3.2] Downloading: http://libguestfs.org/download/builder/debian-9.xz
[   4.1] Planning how to build this image
[   4.1] Uncompressing
[  16.9] Resizing (using virt-resize) to expand the disk to 10.0G
[  40.1] Opening the new disk
[  42.9] Setting a random seed
virt-builder: warning: random seed could not be set for this type of guest
[  42.9] Setting the hostname: debian-9-vm1
[  43.6] Setting the timezone: Europe/Sofia
[  43.6] Setting passwords
virt-builder: Setting random password of root to JtzEYGff9KxL5jCR
[  44.3] Finishing off
                   Output file: /var/lib/libvirt/images/debian-9-vm1.qcow2
                   Output size: 10.0G
                 Output format: qcow2
            Total usable space: 9.8G
                    Free space: 9.0G (91%)


vim bridged.xml

<network>
  <name>br0</name>
  <forward mode="bridge"/>
  <bridge name="br0"/>
</network>

 

# sudo virsh net-define –file bridged.xml
# sudo virsh net-autostart br0
# sudo virsh net-start br0

 

Above two commands will download pre-packaged KVM isos and store them inside /var/lib/libvirt/images/ you see also the root (administrator) password for both ISOs printed out.

 

root@jeremiah:/home/hipo/kvm# ls -ld /var/lib/libvirt/images/*
-rw-r–r– 1 root         root         10739318784 Oct 12 23:45 /var/lib/libvirt/images/debian-9-vm1.qcow2
-rw-r–r– 1 root         root         10739318784 Oct 12 23:46 /var/lib/libvirt/images/fedora-28-vm1.qcow2

 

To access directly the new created VMs as we have specified the –vnc option it is possible to directly vnc to the new host with VNC client (in linux I use vncviewer), on Windows you can use something like TightVNC.
 

6. Use official Linux distributions ISO boot files to install into KVM VM


Those who would like to run inside KVM VM Linux could do it directly using installable ISO files and install the set of Linux with the required packages, just like installing a fresh new Linux on a bare-metal machine.
To do so download your ISO image from the net (either from official distro website or a mirror website, in case if you need to spin an older version) and use virt-install to run the installer inside KVM.

 

root@jeremiah:~# cd /var/lib/libvirt/boot/;
root@jeremiah:~# wget http://mirrors.netix.net/centos/7.5.1804/isos/x86_64/CentOS-7-x86_64-DVD-1804.iso

 

# sudo virt-install \
–virt-type=kvm \
–name centos7 \
–ram 2048 \
–vcpus=2 \
–os-variant=centos7.0 \
–virt-type=kvm \
–hvm \
–cdrom=/var/lib/libvirt/boot/CentOS-7-x86_64-DVD-1804.iso \
–network=bridge=br0,model=virtio \
–network=bridge=br1,model=virtio \
–graphics vnc \
–disk path=/var/lib/libvirt/images/centos7.qcow2,size=40,bus=virtio,format=qcow2


7. List newly created VMs with Virsh command

 

root@jeremiah:/home/hipo/kvm# virsh list –all
 Id    Name                           State
—————————————————-
 3     fedora-28                      running
 –     debian9                        shut off

 

The –all parameter lists all available VMs ready to spin, if you want to check what are the VMs that are only running use instead:

 

root@jeremiah:/home/hipo/kvm# virsh list
 Id    Name                           State
—————————————————-
 3     fedora-28                      running

 

8. Install Virtual Machine OS-es

Below lines will install 2 Virtual machines one Fedora 28 and Debian 9

 

 os='fedora-28';
virt-install –import –name $os \
    –ram 2048 \
    –vcpu 2 \
    –disk path=/var/lib/libvirt/images/$os-vm1.qcow2,format=qcow2 \
    –os-variant fedora-unknown \
    –network=bridge=br0,model=virtio \
    –noautoconsole \
  –hvm \
  –graphics vnc

os='debian9';
virt-install –import –name $os     \
–ram 2048     \
–vcpu 2     \
–disk path=/var/lib/libvirt/images/$os-vm1.qcow2,format=qcow2     \
–os-variant debian9     –network=bridge=br0,model=virtio     \
–noautoconsole \
–hvm \
–graphics vnc


To deploy more just change the virtual machine type in os variable and modify the –os-variant variable to match the distribution name, to get the correct –os-variant variables that can be passed use osinfo-query below is output of the cmd:

 

root@jeremiah:/home/hipo/kvm# osinfo-query os
 Short ID             | Name                                               | Version  | ID                                      
———————-+—————————————————-+———-+—————————————–
 altlinux1.0          | Mandrake RE Spring 2001                            | 1.0      | http://altlinux.org/altlinux/1.0        
 altlinux2.0          | ALT Linux 2.0                                      | 2.0      | http://altlinux.org/altlinux/2.0        
 altlinux2.2          | ALT Linux 2.2                                      | 2.2      | http://altlinux.org/altlinux/2.2        
 altlinux2.4          | ALT Linux 2.4                                      | 2.4      | http://altlinux.org/altlinux/2.4        
 altlinux3.0          | ALT Linux 3.0                                      | 3.0      | http://altlinux.org/altlinux/3.0        
 altlinux4.0          | ALT Linux 4.0                                      | 4.0      | http://altlinux.org/altlinux/4.0        
 altlinux4.1          | ALT Linux 4.1                                      | 4.1      | http://altlinux.org/altlinux/4.1        
 altlinux5.0          | ALT Linux 5.0                                      | 5.0      | http://altlinux.org/altlinux/5.0        
 altlinux6.0          | ALT Linux 6.0                                      | 6.0      | http://altlinux.org/altlinux/6.0        
 altlinux7.0          | ALT Linux 7.0                                      | 7.0      | http://altlinux.org/altlinux/7.0        
 centos6.0            | CentOS 6.0                                         | 6.0      | http://centos.org/centos/6.0            
 centos6.1            | CentOS 6.1                                         | 6.1      | http://centos.org/centos/6.1            
 centos6.2            | CentOS 6.2                                         | 6.2      | http://centos.org/centos/6.2            
 centos6.3            | CentOS 6.3                                         | 6.3      | http://centos.org/centos/6.3            
 centos6.4            | CentOS 6.4                                         | 6.4      | http://centos.org/centos/6.4            
 centos6.5            | CentOS 6.5                                         | 6.5      | http://centos.org/centos/6.5            
 centos6.6            | CentOS 6.6                                         | 6.6      | http://centos.org/centos/6.6            
 centos6.7            | CentOS 6.7                                         | 6.7      | http://centos.org/centos/6.7            
 centos6.8            | CentOS 6.8                                         | 6.8      | http://centos.org/centos/6.8            
 centos6.9            | CentOS 6.9                                         | 6.9      | http://centos.org/centos/6.9            
 centos7.0            | CentOS 7.0                                         | 7.0      | http://centos.org/centos/7.0            
 debian1.1            | Debian Buzz                                        | 1.1      | http://debian.org/debian/1.1            
 debian1.2            | Debian Rex                                         | 1.2      | http://debian.org/debian/1.2            
 debian1.3            | Debian Bo                                          | 1.3      | http://debian.org/debian/1.3            
 debian2.0            | Debian Hamm                                        | 2.0      | http://debian.org/debian/2.0            
 debian2.1            | Debian Slink                                       | 2.1      | http://debian.org/debian/2.1            
 debian2.2            | Debian Potato                                      | 2.2      | http://debian.org/debian/2.2            
 debian3              | Debian Woody                                       | 3        | http://debian.org/debian/3              
 debian3.1            | Debian Sarge                                       | 3.1      | http://debian.org/debian/3.1            
 debian4              | Debian Etch                                        | 4        | http://debian.org/debian/4              
 debian5              | Debian Lenny                                       | 5        | http://debian.org/debian/5              
 debian6              | Debian Squeeze                                     | 6        | http://debian.org/debian/6              
 debian7              | Debian Wheezy                                      | 7        | http://debian.org/debian/7              
 debian8              | Debian Jessie                                      | 8        | http://debian.org/debian/8              
 debian9              | Debian Stretch                                     | 9        | http://debian.org/debian/9              
 debiantesting        | Debian Testing                                     | testing  | http://debian.org/debian/testing        
 fedora-unknown       | Fedora                                             | unknown  | http://fedoraproject.org/fedora/unknown
 fedora1              | Fedora Core 1                                      | 1        | http://fedoraproject.org/fedora/1       
 fedora10             | Fedora 10                                          | 10       | http://fedoraproject.org/fedora/10      
 fedora11             | Fedora 11                                          | 11       | http://fedoraproject.org/fedora/11      
 fedora12             | Fedora 12                                          | 12       | http://fedoraproject.org/fedora/12      
 fedora13             | Fedora 13                                          | 13       | http://fedoraproject.org/fedora/13      
 fedora14             | Fedora 14                                          | 14       | http://fedoraproject.org/fedora/14      
 fedora15             | Fedora 15                                          | 15       | http://fedoraproject.org/fedora/15      
 fedora16             | Fedora 16                                          | 16       | http://fedoraproject.org/fedora/16      
 fedora17             | Fedora 17                                          | 17       | http://fedoraproject.org/fedora/17      
 fedora18             | Fedora 18                                          | 18       | http://fedoraproject.org/fedora/18      
 fedora19             | Fedora 19                                          | 19       | http://fedoraproject.org/fedora/19      
 fedora2              | Fedora Core 2                                      | 2        | http://fedoraproject.org/fedora/2       
 fedora20             | Fedora 20                                          | 20       | http://fedoraproject.org/fedora/20      
 fedora21             | Fedora 21                                          | 21       | http://fedoraproject.org/fedora/21      
 fedora22             | Fedora 22                                          | 22       | http://fedoraproject.org/fedora/22      
 fedora23             | Fedora 23                                          | 23       | http://fedoraproject.org/fedora/23      
 fedora24             | Fedora 24                                          | 24       | http://fedoraproject.org/fedora/24      
 fedora25             | Fedora 25                                          | 25       | http://fedoraproject.org/fedora/25      
 fedora26             | Fedora 26                                          | 26       | http://fedoraproject.org/fedora/26      
 fedora3              | Fedora Core 3                                      | 3        | http://fedoraproject.org/fedora/3       
 fedora4              | Fedora Core 4                                      | 4        | http://fedoraproject.org/fedora/4       
 fedora5              | Fedora Core 5                                      | 5        | http://fedoraproject.org/fedora/5       
 fedora6              | Fedora Core 6                                      | 6        | http://fedoraproject.org/fedora/6       
 fedora7              | Fedora 7                                           | 7        | http://fedoraproject.org/fedora/7       
 fedora8              | Fedora 8                                           | 8        | http://fedoraproject.org/fedora/8       
 fedora9              | Fedora 9                                           | 9        | http://fedoraproject.org/fedora/9       
 freebsd1.0           | FreeBSD 1.0                                        | 1.0      | http://freebsd.org/freebsd/1.0          
 freebsd10.0          | FreeBSD 10.0                                       | 10.0     | http://freebsd.org/freebsd/10.0         
 freebsd10.1          | FreeBSD 10.1                                       | 10.1     | http://freebsd.org/freebsd/10.1         
 freebsd10.2          | FreeBSD 10.2                                       | 10.2     | http://freebsd.org/freebsd/10.2         
 freebsd10.3          | FreeBSD 10.3                                       | 10.3     | http://freebsd.org/freebsd/10.3         
 freebsd10.4          | FreeBSD 10.4                                       | 10.4     | http://freebsd.org/freebsd/10.4         
 freebsd11.0          | FreeBSD 11.0                                       | 11.0     | http://freebsd.org/freebsd/11.0         
 freebsd11.1          | FreeBSD 11.1                                       | 11.1     | http://freebsd.org/freebsd/11.1         
 freebsd2.0           | FreeBSD 2.0                                        | 2.0      | http://freebsd.org/freebsd/2.0          
 freebsd2.0.5         | FreeBSD 2.0.5                                      | 2.0.5    | http://freebsd.org/freebsd/2.0.5        
 freebsd2.2.8         | FreeBSD 2.2.8                                      | 2.2.8    | http://freebsd.org/freebsd/2.2.8        
 freebsd2.2.9         | FreeBSD 2.2.9                                      | 2.2.9    | http://freebsd.org/freebsd/2.2.9        
 freebsd3.0           | FreeBSD 3.0                                        | 3.0      | http://freebsd.org/freebsd/3.0          
 freebsd3.2           | FreeBSD 3.2                                        | 3.2      | http://freebsd.org/freebsd/3.2          
 freebsd4.0           | FreeBSD 4.0                                        | 4.0      | http://freebsd.org/freebsd/4.0          
 freebsd4.1           | FreeBSD 4.1                                        | 4.1      | http://freebsd.org/freebsd/4.1          
 freebsd4.10          | FreeBSD 4.10                                       | 4.10     | http://freebsd.org/freebsd/4.10         
 freebsd4.11          | FreeBSD 4.11                                       | 4.11     | http://freebsd.org/freebsd/4.11         
 freebsd4.2           | FreeBSD 4.2                                        | 4.2      | http://freebsd.org/freebsd/4.2          
 freebsd4.3           | FreeBSD 4.3                                        | 4.3      | http://freebsd.org/freebsd/4.3          
 freebsd4.4           | FreeBSD 4.4                                        | 4.4      | http://freebsd.org/freebsd/4.4          
 freebsd4.5           | FreeBSD 4.5                                        | 4.5      | http://freebsd.org/freebsd/4.5          
 freebsd4.6           | FreeBSD 4.6                                        | 4.6      | http://freebsd.org/freebsd/4.6          
 freebsd4.7           | FreeBSD 4.7                                        | 4.7      | http://freebsd.org/freebsd/4.7          
 freebsd4.8           | FreeBSD 4.8                                        | 4.8      | http://freebsd.org/freebsd/4.8          
 freebsd4.9           | FreeBSD 4.9                                        | 4.9      | http://freebsd.org/freebsd/4.9          
 freebsd5.0           | FreeBSD 5.0                                        | 5.0      | http://freebsd.org/freebsd/5.0          
 freebsd5.1           | FreeBSD 5.1                                        | 5.1      | http://freebsd.org/freebsd/5.1          
 freebsd5.2           | FreeBSD 5.2                                        | 5.2      | http://freebsd.org/freebsd/5.2          
 freebsd5.2.1         | FreeBSD 5.2.1                                      | 5.2.1    | http://freebsd.org/freebsd/5.2.1        
 freebsd5.3           | FreeBSD 5.3                                        | 5.3      | http://freebsd.org/freebsd/5.3          
 freebsd5.4           | FreeBSD 5.4                                        | 5.4      | http://freebsd.org/freebsd/5.4          
 freebsd5.5           | FreeBSD 5.5                                        | 5.5      | http://freebsd.org/freebsd/5.5          
 freebsd6.0           | FreeBSD 6.0                                        | 6.0      | http://freebsd.org/freebsd/6.0          
 freebsd6.1           | FreeBSD 6.1                                        | 6.1      | http://freebsd.org/freebsd/6.1          
 freebsd6.2           | FreeBSD 6.2                                        | 6.2      | http://freebsd.org/freebsd/6.2          
 freebsd6.3           | FreeBSD 6.3                                        | 6.3      | http://freebsd.org/freebsd/6.3          
 freebsd6.4           | FreeBSD 6.4                                        | 6.4      | http://freebsd.org/freebsd/6.4          
 freebsd7.0           | FreeBSD 7.0                                        | 7.0      | http://freebsd.org/freebsd/7.0          
 freebsd7.1           | FreeBSD 7.1                                        | 7.1      | http://freebsd.org/freebsd/7.1          
 freebsd7.2           | FreeBSD 7.2                                        | 7.2      | http://freebsd.org/freebsd/7.2          
 freebsd7.3           | FreeBSD 7.3                                        | 7.3      | http://freebsd.org/freebsd/7.3          
 freebsd7.4           | FreeBSD 7.4                                        | 7.4      | http://freebsd.org/freebsd/7.4          
 freebsd8.0           | FreeBSD 8.0                                        | 8.0      | http://freebsd.org/freebsd/8.0          
 freebsd8.1           | FreeBSD 8.1                                        | 8.1      | http://freebsd.org/freebsd/8.1          
 freebsd8.2           | FreeBSD 8.2                                        | 8.2      | http://freebsd.org/freebsd/8.2          
 freebsd8.3           | FreeBSD 8.3                                        | 8.3      | http://freebsd.org/freebsd/8.3          
 freebsd8.4           | FreeBSD 8.4                                        | 8.4      | http://freebsd.org/freebsd/8.4          
 freebsd9.0           | FreeBSD 9.0                                        | 9.0      | http://freebsd.org/freebsd/9.0          
 freebsd9.1           | FreeBSD 9.1                                        | 9.1      | http://freebsd.org/freebsd/9.1          
 freebsd9.2           | FreeBSD 9.2                                        | 9.2      | http://freebsd.org/freebsd/9.2          
 freebsd9.3           | FreeBSD 9.3                                        | 9.3      | http://freebsd.org/freebsd/9.3          
 freedos1.2           | FreeDOS 1.2                                        | 1.2      | http://freedos.org/freedos/1.2          
 gnome-continuous-3.10 | GNOME 3.10                                         | 3.10     | http://gnome.org/gnome-continuous/3.10  
 gnome-continuous-3.12 | GNOME 3.12                                         | 3.12     | http://gnome.org/gnome-continuous/3.12  
 gnome-continuous-3.14 | GNOME 3.14                                         | 3.14     | http://gnome.org/gnome-continuous/3.14  
 gnome3.6             | GNOME 3.6                                          | 3.6      | http://gnome.org/gnome/3.6              
 gnome3.8             | GNOME 3.8                                          | 3.8      | http://gnome.org/gnome/3.8              
 macosx10.0           | MacOS X Cheetah                                    | 10.0     | http://apple.com/macosx/10.0            
 macosx10.1           | MacOS X Puma                                       | 10.1     | http://apple.com/macosx/10.1            
 macosx10.2           | MacOS X Jaguar                                     | 10.2     | http://apple.com/macosx/10.2            
 macosx10.3           | MacOS X Panther                                    | 10.3     | http://apple.com/macosx/10.3            
 macosx10.4           | MacOS X Tiger                                      | 10.4     | http://apple.com/macosx/10.4            
 macosx10.5           | MacOS X Leopard                                    | 10.5     | http://apple.com/macosx/10.5            
 macosx10.6           | MacOS X Snow Leopard                               | 10.6     | http://apple.com/macosx/10.6            
 macosx10.7           | MacOS X Lion                                       | 10.7     | http://apple.com/macosx/10.7            
 mageia1              | Mageia 1                                           | 1        | http://mageia.org/mageia/1              
 mageia2              | Mageia 2                                           | 2        | http://mageia.org/mageia/2              
 mageia3              | Mageia 3                                           | 3        | http://mageia.org/mageia/3              
 mageia4              | Mageia 4                                           | 4        | http://mageia.org/mageia/4              
 mageia5              | Mageia 5                                           | 5        | http://mageia.org/mageia/5              
 mageia6              | Mageia 6                                           | 6        | http://mageia.org/mageia/6              
 mandrake10.0         | Mandrake Linux 10.0                                | 10.0     | http://mandriva.com/mandrake/10.0       
 mandrake10.1         | Mandrake Linux 10.1                                | 10.1     | http://mandriva.com/mandrake/10.1       
 mandrake10.2         | Mandrake Linux 10.2                                | 10.2     | http://mandriva.com/mandrake/10.2       
 mandrake5.1          | Mandrake Linux 5.1                                 | 5.1      | http://mandriva.com/mandrake/5.1        
 mandrake5.2          | Mandrake Linux 5.2                                 | 5.2      | http://mandriva.com/mandrake/5.2        
 mandrake5.3          | Mandrake Linux 5.3                                 | 5.3      | http://mandriva.com/mandrake/5.3        
 mandrake6.0          | Mandrake Linux 6.0                                 | 6.0      | http://mandriva.com/mandrake/6.0        
 mandrake6.1          | Mandrake Linux 6.1                                 | 6.1      | http://mandriva.com/mandrake/6.1        
 mandrake7.0          | Mandrake Linux 7.0                                 | 7.0      | http://mandriva.com/mandrake/7.0        
 mandrake7.1          | Mandrake Linux 7.1                                 | 7.1      | http://mandriva.com/mandrake/7.1        
 mandrake7.2          | Mandrake Linux 7.2                                 | 7.2      | http://mandriva.com/mandrake/7.2        
 mandrake8.0          | Mandrake Linux 8.0                                 | 8.0      | http://mandriva.com/mandrake/8.0        
 mandrake8.1          | Mandrake Linux 8.1                                 | 8.1      | http://mandriva.com/mandrake/8.1        
 mandrake8.2          | Mandrake Linux 8.2                                 | 8.2      | http://mandriva.com/mandrake/8.2        
 mandrake9.0          | Mandrake Linux 9.0                                 | 9.0      | http://mandriva.com/mandrake/9.0        
 mandrake9.1          | Mandrake Linux 9.1                                 | 9.1      | http://mandriva.com/mandrake/9.1        
 mandrake9.2          | Mandrake Linux 9.2                                 | 9.2      | http://mandriva.com/mandrake/9.2        
 mandriva2006.0       | Mandriva Linux 2006.0                              | 2006.0   | http://mandriva.com/mandriva/2006.0     
 mandriva2007         | Mandriva Linux 2007                                | 2007     | http://mandriva.com/mandriva/2007       
 mandriva2007.1       | Mandriva Linux 2007 Spring                         | 2007.1   | http://mandriva.com/mandriva/2007.1     
 mandriva2008.0       | Mandriva Linux 2008                                | 2008.0   | http://mandriva.com/mandriva/2008.0     
 mandriva2008.1       | Mandriva Linux 2008 Spring                         | 2008.1   | http://mandriva.com/mandriva/2008.1     
 mandriva2009.0       | Mandriva Linux 2009                                | 2009.0   | http://mandriva.com/mandriva/2009.0     
 mandriva2009.1       | Mandriva Linux 2009 Spring                         | 2009.1   | http://mandriva.com/mandriva/2009.1     
 mandriva2010.0       | Mandriva Linux 2010                                | 2010.0   | http://mandriva.com/mandriva/2010.0     
 mandriva2010.1       | Mandriva Linux 2010 Spring                         | 2010.1   | http://mandriva.com/mandriva/2010.1     
 mandriva2010.2       | Mandriva Linux 2010.2                              | 2010.2   | http://mandriva.com/mandriva/2010.2     
 mandriva2011         | Mandriva Linux 2011                                | 2011     | http://mandriva.com/mandriva/2011       
 mbs1.0               | Mandriva Business Server 1.0                       | 1.0      | http://mandriva.com/mbs/1.0             
 mes5                 | Mandriva Enterprise Server 5.0                     | 5.0      | http://mandriva.com/mes/5.0             
 mes5.1               | Mandriva Enterprise Server 5.1                     | 5.1      | http://mandriva.com/mes/5.1             
 msdos6.22            | Microsoft MS-DOS 6.22                              | 6.22     | http://microsoft.com/msdos/6.22         
 netbsd0.8            | NetBSD 0.8                                         | 0.8      | http://netbsd.org/netbsd/0.8            
 netbsd0.9            | NetBSD 0.9                                         | 0.9      | http://netbsd.org/netbsd/0.9            
 netbsd1.0            | NetBSD 1.0                                         | 1.0      | http://netbsd.org/netbsd/1.0            
 netbsd1.1            | NetBSD 1.1                                         | 1.1      | http://netbsd.org/netbsd/1.1            
 netbsd1.2            | NetBSD 1.2                                         | 1.2      | http://netbsd.org/netbsd/1.2            
 netbsd1.3            | NetBSD 1.3                                         | 1.3      | http://netbsd.org/netbsd/1.3            
 netbsd1.4            | NetBSD 1.4                                         | 1.4      | http://netbsd.org/netbsd/1.4            
 netbsd1.5            | NetBSD 1.5                                         | 1.5      | http://netbsd.org/netbsd/1.5            
 netbsd1.6            | NetBSD 1.6                                         | 1.6      | http://netbsd.org/netbsd/1.6            
 netbsd2.0            | NetBSD 2.0                                         | 2.0      | http://netbsd.org/netbsd/2.0            
 netbsd3.0            | NetBSD 3.0                                         | 3.0      | http://netbsd.org/netbsd/3.0            
 netbsd4.0            | NetBSD 4.0                                         | 4.0      | http://netbsd.org/netbsd/4.0            
 netbsd5.0            | NetBSD 5.0                                         | 5.0      | http://netbsd.org/netbsd/5.0            
 netbsd5.1            | NetBSD 5.1                                         | 5.1      | http://netbsd.org/netbsd/5.1            
 netbsd6.0            | NetBSD 6.0                                         | 6.0      | http://netbsd.org/netbsd/6.0            
 netbsd6.1            | NetBSD 6.1                                         | 6.1      | http://netbsd.org/netbsd/6.1            
 netbsd7.0            | NetBSD 7.0                                         | 7.0      | http://netbsd.org/netbsd/7.0            
 netbsd7.1            | NetBSD 7.1                                         | 7.1      | http://netbsd.org/netbsd/7.1            
 netbsd7.1.1          | NetBSD 7.1.1                                       | 7.1.1    | http://netbsd.org/netbsd/7.1.1          
 netware4             | Novell Netware 4                                   | 4        | http://novell.com/netware/4             
 netware5             | Novell Netware 5                                   | 5        | http://novell.com/netware/5             
 netware6             | Novell Netware 6                                   | 6        | http://novell.com/netware/6             
 openbsd4.2           | OpenBSD 4.2                                        | 4.2      | http://openbsd.org/openbsd/4.2          
 openbsd4.3           | OpenBSD 4.3                                        | 4.3      | http://openbsd.org/openbsd/4.3          
 openbsd4.4           | OpenBSD 4.4                                        | 4.4      | http://openbsd.org/openbsd/4.4          
 openbsd4.5           | OpenBSD 4.5                                        | 4.5      | http://openbsd.org/openbsd/4.5          
 openbsd4.8           | OpenBSD 4.8                                        | 4.8      | http://openbsd.org/openbsd/4.8          
 openbsd4.9           | OpenBSD 4.9                                        | 4.9      | http://openbsd.org/openbsd/4.9          
 openbsd5.0           | OpenBSD 5.0                                        | 5.0      | http://openbsd.org/openbsd/5.0          
 openbsd5.1           | OpenBSD 5.1                                        | 5.1      | http://openbsd.org/openbsd/5.1          
 openbsd5.2           | OpenBSD 5.2                                        | 5.2      | http://openbsd.org/openbsd/5.2          
 openbsd5.3           | OpenBSD 5.3                                        | 5.3      | http://openbsd.org/openbsd/5.3          
 openbsd5.4           | OpenBSD 5.4                                        | 5.4      | http://openbsd.org/openbsd/5.4          
 openbsd5.5           | OpenBSD 5.5                                        | 5.5      | http://openbsd.org/openbsd/5.5          
 openbsd5.6           | OpenBSD 5.6                                        | 5.6      | http://openbsd.org/openbsd/5.6          
 openbsd5.7           | OpenBSD 5.7                                        | 5.7      | http://openbsd.org/openbsd/5.7          
 openbsd5.8           | OpenBSD 5.8                                        | 5.8      | http://openbsd.org/openbsd/5.8          
 openbsd5.9           | OpenBSD 5.9                                        | 5.9      | http://openbsd.org/openbsd/5.9          
 openbsd6.0           | OpenBSD 6.0                                        | 6.0      | http://openbsd.org/openbsd/6.0          
 openbsd6.1           | OpenBSD 6.1                                        | 6.1      | http://openbsd.org/openbsd/6.1          
 openbsd6.2           | OpenBSD 6.2                                        | 6.2      | http://openbsd.org/openbsd/6.2          
 opensolaris2009.06   | OpenSolaris 2009.06                                | 2009.06  | http://sun.com/opensolaris/2009.06      
 opensuse-factory     | openSUSE                                           | factory  | http://opensuse.org/opensuse/factory    
 opensuse-unknown     | openSUSE                                           | unknown  | http://opensuse.org/opensuse/unknown    
 opensuse10.2         | openSUSE 10.2                                      | 10.2     | http://opensuse.org/opensuse/10.2       
 opensuse10.3         | openSUSE 10.3                                      | 10.3     | http://opensuse.org/opensuse/10.3       
 opensuse11.0         | openSUSE 11.0                                      | 11.0     | http://opensuse.org/opensuse/11.0       
 opensuse11.1         | openSUSE 11.1                                      | 11.1     | http://opensuse.org/opensuse/11.1       
 opensuse11.2         | openSUSE 11.2                                      | 11.2     | http://opensuse.org/opensuse/11.2       
 opensuse11.3         | openSUSE 11.3                                      | 11.3     | http://opensuse.org/opensuse/11.3       
 opensuse11.4         | openSUSE 11.4                                      | 11.4     | http://opensuse.org/opensuse/11.4       
 opensuse12.1         | openSUSE 12.1                                      | 12.1     | http://opensuse.org/opensuse/12.1       
 opensuse12.2         | openSUSE 12.2                                      | 12.2     | http://opensuse.org/opensuse/12.2       
 opensuse12.3         | openSUSE 12.3                                      | 12.3     | http://opensuse.org/opensuse/12.3       
 opensuse13.1         | openSUSE 13.1                                      | 13.1     | http://opensuse.org/opensuse/13.1       
 opensuse13.2         | openSUSE 13.2                                      | 13.2     | http://opensuse.org/opensuse/13.2       
 opensuse42.1         | openSUSE Leap 42.1                                 | 42.1     | http://opensuse.org/opensuse/42.1       
 opensuse42.2         | openSUSE Leap 42.2                                 | 42.2     | http://opensuse.org/opensuse/42.2       
 opensuse42.3         | openSUSE Leap 42.3                                 | 42.3     | http://opensuse.org/opensuse/42.3       
 opensusetumbleweed   | openSUSE Tumbleweed                                | tumbleweed | http://opensuse.org/opensuse/tumbleweed
 rhel-atomic-7.0      | Red Hat Enterprise Linux Atomic Host 7.0           | 7.0      | http://redhat.com/rhel-atomic/7.0       
 rhel-atomic-7.1      | Red Hat Enterprise Linux Atomic Host 7.1           | 7.1      | http://redhat.com/rhel-atomic/7.1       
 rhel-atomic-7.2      | Red Hat Enterprise Linux Atomic Host 7.2           | 7.2      | http://redhat.com/rhel-atomic/7.2       
 rhel2.1              | Red Hat Enterprise Linux 2.1                       | 2.1      | http://redhat.com/rhel/2.1              
 rhel2.1.1            | Red Hat Enterprise Linux 2.1 Update 1  
/etc/bind/masters/elinvent.com            | 2.1.1    | http://redhat.com/rhel/2.1.1            
 rhel2.1.2            | Red Hat Enterprise Linux 2.1 Update 2              | 2.1.2    | http://redhat.com/rhel/2.1.2            
 rhel2.1.3            | Red Hat Enterprise Linux 2.1 Update 3              | 2.1.3    | http://redhat.com/rhel/2.1.3            
 rhel2.1.4            | Red Hat Enterprise Linux 2.1 Update 4              | 2.1.4    | http://redhat.com/rhel/2.1.4            
 rhel2.1.5            | Red Hat Enterprise Linux 2.1 Update 5              | 2.1.5    | http://redhat.com/rhel/2.1.5            
 rhel2.1.6            | Red Hat Enterprise Linux 2.1 Update 6              | 2.1.6    | http://redhat.com/rhel/2.1.6            
 rhel2.1.7            | Red Hat Enterprise Linux 2.1 Update 7              | 2.1.7    | http://redhat.com/rhel/2.1.7            
 rhel3                | Red Hat Enterprise Linux 3                         | 3        | http://redhat.com/rhel/3                
 rhel3.1              | Red Hat Enterprise Linux 3 Update 1                | 3.1      | http://redhat.com/rhel/3.1              
 rhel3.2              | Red Hat Enterprise Linux 3 Update 2                | 3.2      | http://redhat.com/rhel/3.2              
 rhel3.3              | Red Hat Enterprise Linux 3 Update 3                | 3.3      | http://redhat.com/rhel/3.3              
 rhel3.4              | Red Hat Enterprise Linux 3 Update 4                | 3.4      | http://redhat.com/rhel/3.4              
 rhel3.5              | Red Hat Enterprise Linux 3 Update 5                | 3.5      | http://redhat.com/rhel/3.5              
 rhel3.6              | Red Hat Enterprise Linux 3 Update 6                | 3.6      | http://redhat.com/rhel/3.6              
 rhel3.7              | Red Hat Enterprise Linux 3 Update 7                | 3.7      | http://redhat.com/rhel/3.7              
 rhel3.8              | Red Hat Enterprise Linux 3 Update 8                | 3.8      | http://redhat.com/rhel/3.8              
 rhel3.9              | Red Hat Enterprise Linux 3 Update 9                | 3.9      | http://redhat.com/rhel/3.9              
 rhel4.0              | Red Hat Enterprise Linux 4.0                       | 4.0      | http://redhat.com/rhel/4.0              
 rhel4.1              | Red Hat Enterprise Linux 4.1                       | 4.1      | http://redhat.com/rhel/4.1              
 rhel4.2              | Red Hat Enterprise Linux 4.2                       | 4.2      | http://redhat.com/rhel/4.2              
 rhel4.3              | Red Hat Enterprise Linux 4.3                       | 4.3      | http://redhat.com/rhel/4.3              
 rhel4.4              | Red Hat Enterprise Linux 4.4                       | 4.4      | http://redhat.com/rhel/4.4              
 rhel4.5              | Red Hat Enterprise Linux 4.5                       | 4.5      | http://redhat.com/rhel/4.5              
 rhel4.6              | Red Hat Enterprise Linux 4.6                       | 4.6      | http://redhat.com/rhel/4.6              
 rhel4.7              | Red Hat Enterprise Linux 4.7                       | 4.7      | http://redhat.com/rhel/4.7              
 rhel4.8              | Red Hat Enterprise Linux 4.8                       | 4.8      | http://redhat.com/rhel/4.8              
 rhel4.9              | Red Hat Enterprise Linux 4.9                       | 4.9      | http://redhat.com/rhel/4.9              
 rhel5.0              | Red Hat Enterprise Linux 5.0                       | 5.0      | http://redhat.com/rhel/5.0              
 rhel5.1              | Red Hat Enterprise Linux 5.1                       | 5.1      | http://redhat.com/rhel/5.1              
 rhel5.10             | Red Hat Enterprise Linux 5.10                      | 5.10     | http://redhat.com/rhel/5.10             
 rhel5.11             | Red Hat Enterprise Linux 5.11                      | 5.11     | http://redhat.com/rhel/5.11             
 rhel5.2              | Red Hat Enterprise Linux 5.2                       | 5.2      | http://redhat.com/rhel/5.2              
 rhel5.3              | Red Hat Enterprise Linux 5.3                       | 5.3      | http://redhat.com/rhel/5.3              
 rhel5.4              | Red Hat Enterprise Linux 5.4                       | 5.4      | http://redhat.com/rhel/5.4              
 rhel5.5              | Red Hat Enterprise Linux 5.5                       | 5.5      | http://redhat.com/rhel/5.5              
 rhel5.6              | Red Hat Enterprise Linux 5.6                       | 5.6      | http://redhat.com/rhel/5.6              
 rhel5.7              | Red Hat Enterprise Linux 5.7                       | 5.7      | http://redhat.com/rhel/5.7              
 rhel5.8              | Red Hat Enterprise Linux 5.8                       | 5.8      | http://redhat.com/rhel/5.8              
 rhel5.9              | Red Hat Enterprise Linux 5.9                       | 5.9      | http://redhat.com/rhel/5.9              
 rhel6.0              | Red Hat Enterprise Linux 6.0                       | 6.0      | http://redhat.com/rhel/6.0              
 rhel6.1              | Red Hat Enterprise Linux 6.1                       | 6.1      | http://redhat.com/rhel/6.1              
 rhel6.2              | Red Hat Enterprise Linux 6.2                       | 6.2      | http://redhat.com/rhel/6.2              
 rhel6.3              | Red Hat Enterprise Linux 6.3                       | 6.3      | http://redhat.com/rhel/6.3              
 rhel6.4              | Red Hat Enterprise Linux 6.4                       | 6.4      | http://redhat.com/rhel/6.4              
 rhel6.5              | Red Hat Enterprise Linux 6.5                       | 6.5      | http://redhat.com/rhel/6.5              
 rhel6.6              | Red Hat Enterprise Linux 6.6                       | 6.6      | http://redhat.com/rhel/6.6              
 rhel6.7              | Red Hat Enterprise Linux 6.7                       | 6.7      | http://redhat.com/rhel/6.7              
 rhel6.8              | Red Hat Enterprise Linux 6.8                       | 6.8      | http://redhat.com/rhel/6.8              
 rhel6.9              | Red Hat Enterprise Linux 6.9                       | 6.9      | http://redhat.com/rhel/6.9              
 rhel7.0              | Red Hat Enterprise Linux 7.0                       | 7.0      | http://redhat.com/rhel/7.0              
 rhel7.1              | Red Hat Enterprise Linux 7.1                       | 7.1      | http://redhat.com/rhel/7.1              
 rhel7.2              | Red Hat Enterprise Linux 7.2                       | 7.2      | http://redhat.com/rhel/7.2              
 rhel7.3              | Red Hat Enterprise Linux 7.3                       | 7.3      | http://redhat.com/rhel/7.3              
 rhel7.4              | Red Hat Enterprise Linux 7.4                       | 7.4      | http://redhat.com/rhel/7.4              
 rhl1.0               | Red Hat Linux 1.0                                  | 1.0      | http://redhat.com/rhl/1.0               
 rhl1.1               | Red Hat Linux 1.1                                  | 1.1      | http://redhat.com/rhl/1.1               
 rhl2.0               | Red Hat Linux 2.0                                  | 2.0      | http://redhat.com/rhl/2.0               
 rhl2.1               | Red Hat Linux 2.1                                  | 2.1      | http://redhat.com/rhl/2.1               
 rhl3.0.3             | Red Hat Linux 3.0.3                                | 3.0.3    | http://redhat.com/rhl/3.0.3             
 rhl4.0               | Red Hat Linux 4.0                                  | 4.0      | http://redhat.com/rhl/4.0               
 rhl4.1               | Red Hat Linux 4.1                                  | 4.1      | http://redhat.com/rhl/4.1               
 rhl4.2               | Red Hat Linux 4.2                                  | 4.2      | http://redhat.com/rhl/4.2               
 rhl5.0               | Red Hat Linux 5.0                                  | 5.0      | http://redhat.com/rhl/5.0               
 rhl5.1               | Red Hat Linux 5.1                                  | 5.1      | http://redhat.com/rhl/5.1               
 rhl5.2               | Red Hat Linux 5.2                                  | 5.2      | http://redhat.com/rhl/5.2               
 rhl6.0               | Red Hat Linux 6.0                                  | 6.0      | http://redhat.com/rhl/6.0               
 rhl6.1               | Red Hat Linux 6.1                                  | 6.1      | http://redhat.com/rhl/6.1               
 rhl6.2               | Red Hat Linux 6.2                                  | 6.2      | http://redhat.com/rhl/6.2               
 rhl7                 | Red Hat Linux 7                                    | 7        | http://redhat.com/rhl/7                 
 rhl7.1               | Red Hat Linux 7.1                                  | 7.1      | http://redhat.com/rhl/7.1               
 rhl7.2               | Red Hat Linux 7.2                                  | 7.2      | http://redhat.com/rhl/7.2               
 rhl7.3               | Red Hat Linux 7.3                                  | 7.3      | http://redhat.com/rhl/7.3               
 rhl8.0               | Red Hat Linux 8.0                                  | 8.0      | http://redhat.com/rhl/8.0               
 rhl9                 | Red Hat Linux 9                                    | 9        | http://redhat.com/rhl/9                 
 sled10               | SUSE Linux Enterprise Desktop 10                   | 10       | http://suse.com/sled/10                 
 sled10sp1            | SUSE Linux Enterprise Desktop 10 SP1               | 10.1     | http://suse.com/sled/10.1               
 sled10sp2            | SUSE Linux Enterprise Desktop 10 SP2               | 10.2     | http://suse.com/sled/10.2               
 sled10sp3            | SUSE Linux Enterprise Desktop 10 SP3               | 10.3     | http://suse.com/sled/10.3               
 sled10sp4            | SUSE Linux Enterprise Desktop 10 SP4               | 10.4     | http://suse.com/sled/10.4               
 sled11               | SUSE Linux Enterprise Desktop 11                   | 11       | http://suse.com/sled/11                 
 sled11sp1            | SUSE Linux Enterprise Desktop 11 SP1               | 11.1     | http://suse.com/sled/11.1               
 sled11sp2            | SUSE Linux Enterprise Desktop 11 SP2               | 11.2     | http://suse.com/sled/11.2               
 sled11sp3            | SUSE Linux Enterprise Desktop 11 SP3               | 11.3     | http://suse.com/sled/11.3               
 sled11sp4            | SUSE Linux Enterprise Desktop 11 SP4               | 11.4     | http://suse.com/sled/11.4               
 sled12               | SUSE Linux Enterprise Desktop 12                   | 12       | http://suse.com/sled/12                 
 sled12sp1            | SUSE Linux Enterprise Desktop 12 SP1               | 12.1     | http://suse.com/sled/12.1               
 sled12sp2            | SUSE Linux Enterprise Desktop 12 SP2               | 12.2     | http://suse.com/sled/12.2               
 sled9                | SUSE Linux Enterprise Desktop 9                    | 9        | http://suse.com/sled/9                  
 sles10               | SUSE Linux Enterprise Server 10            
/etc/bind/masters/elinvent.com        | 10       | http://suse.com/sles/10                 
 sles10sp1            | SUSE Linux Enterprise Server 10 SP1                | 10.1     | http://suse.com/sles/10.1               
 sles10sp2            | SUSE Linux Enterprise Server 10 SP2                | 10.2     | http://suse.com/sles/10.2               
 sles10sp3            | SUSE Linux Enterprise Server 10 SP3                | 10.3     | http://suse.com/sles/10.3               
 sles10sp4            | SUSE Linux Enterprise Server 10 SP4                | 10.4     | http://suse.com/sles/10.4               
 sles11               | SUSE Linux Enterprise Server 11                    | 11       | http://suse.com/sles/11                 
 sles11sp1            | SUSE Linux Enterprise Server 11 SP1                | 11.1     | http://suse.com/sles/11.1               
 sles11sp2            | SUSE Linux Enterprise Server 11 SP2                | 11.2     | http://suse.com/sles/11.2               
 sles11sp3            | SUSE Linux Enterprise Server 11 SP3                | 11.3     | http://suse.com/sles/11.3               
 sles11sp4            | SUSE Linux Enterprise Server 11 SP4                | 11.4     | http://suse.com/sles/11.4               
 sles12               | SUSE Linux Enterprise Server 12                    | 12       | http://suse.com/sles/12                 
 sles12sp1            | SUSE Linux Enterprise Server 12 SP1                | 12.1     | http://suse.com/sles/12.1               
 sles12sp2            | SUSE Linux Enterprise Server 12 SP2                | 12.2     | http://suse.com/sles/12.2               
 sles9                | SUSE Linux Enterprise Server 9                     | 9        | http://suse.com/sles/9                  
 solaris10            | Solaris 10                                         | 10       | http://sun.com/solaris/10               
 solaris11            | Oracle Solaris 11                                  | 11       | http://oracle.com/solaris/11            
 solaris9             | Solaris 9                                          | 9        | http://sun.com/solaris/9                
 ubuntu10.04          | Ubuntu 10.04 LTS                                   | 10.04    | http://ubuntu.com/ubuntu/10.04          
 ubuntu10.10          | Ubuntu 10.10                                       | 10.10    | http://ubuntu.com/ubuntu/10.10          
 ubuntu11.04          | Ubuntu 11.04                                       | 11.04    | http://ubuntu.com/ubuntu/11.04          
 ubuntu11.10          | Ubuntu 11.10                                       | 11.10    | http://ubuntu.com/ubuntu/11.10          
 ubuntu12.04          | Ubuntu 12.04 LTS                                   | 12.04    | http://ubuntu.com/ubuntu/12.04          
 ubuntu12.10          | Ubuntu 12.10                                       | 12.10    | http://ubuntu.com/ubuntu/12.10          
 ubuntu13.04          | Ubuntu 13.04                                       | 13.04    | http://ubuntu.com/ubuntu/13.04          
 ubuntu13.10          | Ubuntu 13.10                                       | 13.10    | http://ubuntu.com/ubuntu/13.10          
 ubuntu14.04          | Ubuntu 14.04 LTS                                   | 14.04    | http://ubuntu.com/ubuntu/14.04          
 ubuntu14.10          | Ubuntu 14.10                                       | 14.10    | http://ubuntu.com/ubuntu/14.10          
 ubuntu15.04          | Ubuntu 15.04                                       | 15.04    | http://ubuntu.com/ubuntu/15.04          
 ubuntu15.10          | Ubuntu 15.10                                       | 15.10    | http://ubuntu.com/ubuntu/15.10          
 ubuntu16.04          | Ubuntu 16.04                                       | 16.04    | http://ubuntu.com/ubuntu/16.04          
 ubuntu16.10          | Ubuntu 16.10                                       | 16.10    | http://ubuntu.com/ubuntu/16.10          
 ubuntu17.04          | Ubuntu 17.04                                       | 17.04    | http://ubuntu.com/ubuntu/17.04          
 ubuntu17.10          | Ubuntu 17.10                                       | 17.10    | http://ubuntu.com/ubuntu/17.10          
 ubuntu4.10           | Ubuntu 4.10                                        | 4.10     | http://ubuntu.com/ubuntu/4.10           
 ubuntu5.04           | Ubuntu 5.04                                        | 5.04     | http://ubuntu.com/ubuntu/5.04           
 ubuntu5.10           | Ubuntu 5.10                                        | 5.10     | http://ubuntu.com/ubuntu/5.10           
 ubuntu6.06           | Ubuntu 6.06 LTS                                    | 6.06     | http://ubuntu.com/ubuntu/6.06           
 ubuntu6.10           | Ubuntu 6.10                                        | 6.10     | http://ubuntu.com/ubuntu/6.10           
 ubuntu7.04           | Ubuntu 7.04                                        | 7.04     | http://ubuntu.com/ubuntu/7.04           
 ubuntu7.10           | Ubuntu 7.10                                        | 7.10     | http://ubuntu.com/ubuntu/7.10           
 ubuntu8.04           | Ubuntu 8.04 LTS                                    | 8.04     | http://ubuntu.com/ubuntu/8.04           
 ubuntu8.10           | Ubuntu 8.10                                        | 8.10     | http://ubuntu.com/ubuntu/8.10           
 ubuntu9.04           | Ubuntu 9.04                                        | 9.04     | http://ubuntu.com/ubuntu/9.04           
 ubuntu9.10           | Ubuntu 9.10                                        | 9.10     | http://ubuntu.com/ubuntu/9.10           
 win1.0               | Microsoft Windows 1.0                              | 1.0      | http://microsoft.com/win/1.0            
 win10                | Microsoft Windows 10                               | 10.0     | http://microsoft.com/win/10             
 win2.0               | Microsoft Windows 2.0                              | 2.0      | http://microsoft.com/win/2.0            
 win2.1               | Microsoft Windows 2.1                              | 2.1      | http://microsoft.com/win/2.1            
 win2k                | Microsoft Windows 2000                             | 5.0      | http://microsoft.com/win/2k             
 win2k12              | Microsoft Windows Server 2012                      | 6.3      | http://microsoft.com/win/2k12           
 win2k12r2            | Microsoft Windows Server 2012 R2                   | 6.3      | http://microsoft.com/win/2k12r2         
 win2k3               | Microsoft Windows Server 2003                      | 5.2      | http://microsoft.com/win/2k3            
 win2k3r2             | Microsoft Windows Server 2003 R2                   | 5.2      | http://microsoft.com/win/2k3r2          
 win2k8               | Microsoft Windows Server 2008                      | 6.0      | http://microsoft.com/win/2k8            
 win2k8r2             | Microsoft Windows Server 2008 R2                   | 6.1      | http://microsoft.com/win/2k8r2          
 win3.1               | Microsoft Windows 3.1                              | 3.1      | http://microsoft.com/win/3.1            
 win7                 | Microsoft Windows 7                                | 6.1      | http://microsoft.com/win/7              
 win8                 | Microsoft Windows 8                                | 6.2      | http://microsoft.com/win/8              
 win8.1               | Microsoft Windows 8.1                              | 6.3      | http://microsoft.com/win/8.1            
 win95                | Microsoft Windows 95                               | 4.0      | http://microsoft.com/win/95             
 win98                | Microsoft Windows 98                               | 4.1      | http://microsoft.com/win/98             
 winme                | Microsoft Windows Millennium Edition               | 4.9      | http://microsoft.com/win/me             
 winnt3.1             | Microsoft Windows NT Server 3.1                    | 3.1      | http://microsoft.com/winnt/3.1          
 winnt3.5             | Microsoft Windows NT Server 3.5                    | 3.5      | http://microsoft.com/winnt/3.5          
 winnt3.51            | Microsoft Windows NT Server 3.51                   | 3.51     | http://microsoft.com/winnt/3.51         
 winnt4.0             | Microsoft Windows NT Server 4.0                    | 4.0      | http://microsoft.com/winnt/4.0          
 winvista             | Microsoft Windows Vista                            | 6.0      | http://microsoft.com/win/vista          
 winxp                | Microsoft Windows XP                               | 5.1      | http://microsoft.com/win/xp  

 

9. Start / Stop listed KVM Virtual Machine

 

root@jeremiah:~# virsh list –all
 Id    Name                           State
—————————————————-
 3     fedora-28                      running
 –     debian9                        shut off

 

To start debian9 linux virtual machine that is currently off

 

root@jeremiah:~# virsh start fedora-28
Domain fedora-28 started

 

root@jeremiah:/home/hipo# virsh start debian9
error: Failed to start domain debian9
error: Requested operation is not valid: network 'default' is not active

root@jeremiah:/home/hipo# virsh net-list –all
Name                 State      Autostart     Persistent
———————————————————-
br0                  active     yes           yes
default              inactive   no            yes

 

root@jeremiah:/home/hipo# virsh net-start default
Network default started

root@jeremiah:/home/hipo# virsh start debian9
Domain debian9 started

 

10. Attach to running VM with virsh or virt-manager

 

root@jeremiah:~# virsh list
 Id    Name                           State
—————————————————-
 1     fedora-28                      running
 3     debian9                        running

root@jeremiah:~# virsh connect debian9

 


Note that to make the login prompt appear you have to press enter once after the ^] connection string appears


kvm-connect-to-virtual-machine-with-virsh-command-screenshot-howto

An alternative way is to use virt-manager GUI KVM desktop management interface and click over the Virtual Machine Guest name, in same fashion like in VirtualBox.

virtual-manager-virt-manager-screenshot-with-Virtual-Machines-inside-on-Debian-Linux

virt-manager-gui-interface-connect-to-fedora-28-virtual-machine

If you have KVM running on your Linux desktop PC / notebook you can also connect via VNC with virsh command.

 

root@jericho:~# virsh vncdisplay centos7


Another handy thing is to expose the Virtualized Guest OS with VNC in order to be able to connect and manage installation or further Linux configuration via VNC using an SSH Tunnel with port forwarding:

 

$ ssh hipo@pc-freak.net -L 5901:127.0.0.1:5901

 

11.  Start / Shutdown / Suspend / Reboot (safe reboot) a VM guest machine domain

 

 

root@jericho:~# virsh shutdown debian9
root@jericho:~# virsh start fedora-28
root@jericho:~# virsh suspend debian9
root@jericho:~# virsh reboot fedora-28

 

12. Remove / Delete KVM Virtual Machines domain

 

root@jeremiah:~# virsh undefine fedora-28
root@jeremiah:~# virsh destroy fedora-28


Closing words


Using KVM to experiment with different OS distributions is really fun just like you can easily run a number of the major most popular Linux Distributions and a set of different versions. It takes few minutes to have a fully functional Linux to play with and it saves a lot of hassles when dealing with GNU / Linux and FreeBSD, doing so in Virtualbox for me prooved to be much more complicated (not to mention that often Virtualbox had an ugly bugs so even Importing an Appliance as a Guest VM with an official distro OS-es failed with weird errors.
One other very practical use of Kerkel-based Virtualization is if you want to run your servers using own Micro-Services architecture (e.g. run multiple Linux OS-es each running a separate Apache / Nginx / MySQL / PostGreSQL / Backup / Storage) etc. all of it running on a single dedicated server or a self-hosted bare-metal
There are plenty of Web Interfaces for Management KVM (proprietary and free software) that could even futher simplify the use and deploy / destory of KVM VMs.
All that makes possible running your own Linux or Web hosting provider a relatively easy task and seriously could cut business expenses and operational (maintenance) costs.

If you plan to run youw own hosting company, I can help you establish your infrastructure and advise you on the right technologies to use.

 

Qmail redirect mail to another one and keep local Mailbox copy with .qmail file – Easy Set up email forwarding Qmail

Saturday, August 11th, 2018

Qmail redirect mail box to another one with .Qmail file dolphin artistic logo

QMail (Considered to be the most secure Mail server out there whose modified version is running on Google – Gmail.com and Mail Yahoo! and Yandex EMail (SMTP) servers, nowadays has been highly neglected and considered obsolete thus most people prefer to use postfix SMTP or EXIM but still if you happen to be running a number of qmail old rack Mail servers (running a bunch of Email addresses and Virtual Domains straight on the filesystem – very handy by the way for administration much better than when you have a Qmail Mail server configured to store its Mailboxes within MySQL / PostgreSQL or other Database server – because simple vpopmail configured to play nice with Qmail and store all user emails directly on Filesystem (though considered more insecure the email correspondence can be easily red, if the server is hacked it is much better managable for a small and mid-sized mailserver) or have inherited them from another sys admin and you wonder how to redirect a single Mailbox:

(under domain lets say domain's email  my-server1.com should forward to to SMTP domain my-server-whatever2.com (e.g. your-email-username@server-whatever1.com is supposed to forward to your-email-username2@server-whatever2.com).
To achieve it create new file called .qmail

Under the Qmail or VirtualDomain location for example:

/var/qmail/mailnames/myserver1.com/username/.qmail

 

e.g
 

root@qmail-server:~# vim /var/qmail/mailnames/myserver1.com/your-email-username/.qmail
&your-email-username@server-whatever1.com

your-email-username@example1.com
/home/vpopmail/domains/server-whatever2.com/your-email-username/Maildir/


!!! NOTE N.B. !!! the last slash / after Maildir (…Maildir/) is important to be there otherwise mail will not get delivered
That's all now send a test email, just to make sure redirection works properly, assuming the .qmail file is created by root, by default the file permissions will be with privileges root:root.

Note
 

That shouldn't be a problem at all. That's all now enjoy emails being dropped out to the second mail 🙂

 

How to check Linux OS install date / How long ago was Linux installed

Sunday, October 22nd, 2017

If you're sysadmin who inherited a few hundreds of Linux machines from a previous admin and you're in process of investigating how things were configured by the previous administrator one of the crucial things to find out might be

How Long ago was Linux installed?

Here is how to check the Linux OS install date.

The universal way nomatter the Linux distribution is to use fullowing command:

 

root@pcfreak:~# tune2fs -l /dev/sda1 | grep 'Filesystem created:'
Filesystem created:       Thu Sep  6 21:44:22 2012

 

 

Above command assumes the Linux's root partition / is installed on /dev/sda1 however if your case is different, e.g. the primary root partition is installed on /dev/sda2 or /dev/sdb1 / dev/sdb2 etc. just place the right first partition into the command.

If primary install root partition is /dev/sdb1 for example:
 

root@pcfreak:~# tune2fs -l /dev/sdb1 | grep 'Filesystem created:'

 


To find out what is the root partition of the Linux server installed use fdisk command:

 

 

 

root@pcfreak:~# fdisk -l

 

Disk /dev/sda: 465,8 GiB, 500107862016 bytes, 976773168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x00051eda

Device     Boot     Start       End   Sectors   Size Id Type
/dev/sda1  *         2048 965193727 965191680 460,2G 83 Linux
/dev/sda2       965195774 976771071  11575298   5,5G  5 Extended
/dev/sda5       965195776 976771071  11575296   5,5G 82 Linux swap / Solaris

Disk /dev/sdb: 111,8 GiB, 120034123776 bytes, 234441648 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x00000000

 


Other ways to check the Linux OS install date on Debian / Ubuntu / Mint etc. deb. based GNU / Linux

 


Deban based Linux distributions do create an initial /var/log/installer directory containing various install information such as hardware-summary, partition, initial installed deb packages, exact version of Linux distribution, and the way it was installed either it was installed from an ISO image, or it was network install etc.

 

root@pcfreak:~# ls -al /var/log/installer/
total 1228
drwxr-xr-x  3 root root   4096 sep  6  2012 ./
drwxr-xr-x 72 root root  12288 окт 22 06:26 ../
drwxr-xr-x  2 root root   4096 sep  6  2012 cdebconf/
-rw-r–r–  1 root root  17691 sep  6  2012 hardware-summary
-rw-r–r–  1 root root    163 sep  6  2012 lsb-release
-rw——-  1 root root 779983 sep  6  2012 partman
-rw-r–r–  1 root root  51640 sep  6  2012 status
-rw——-  1 root root 363674 sep  6  2012 syslog

 

If those directory is missing was wiped out by the previous administrator, to clear up traces of his previous work before he left job another possible way to find out exact install date is to check timestamp of /lost+found directory;
 

root@pcfreak:~# ls -ld /lost+found/
drwx—— 2 root root 16384 sep  6  2012 /lost+found//

 

Check OS Linux install date on (Fedora, CentOS, Scientific Linux, Oracle and other Redhat RPM based Distros)

 

[root@centos: ~]# rpm -qi basesystem
Name        : basesystem
Version     : 10.0
Release     : 7.el7
Architecture: noarch
Install Date: Mon 02 May 2016 19:20:58 BST
Group       : System Environment/Base
Size        : 0
License     : Public Domain
Signature   : RSA/SHA256, Tue 01 Apr 2014 14:23:16 BST, Key ID     199e2f91fd431d51
Source RPM  : basesystem-10.0-7.el7.src.rpm
Build Date  : Fri 27 Dec 2013 17:22:15 GMT
Build Host  : ppc-015.build.eng.bos.redhat.com
Relocations : (not relocatable)
Packager    : Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla>
Vendor      : Red Hat, Inc.
Summary     : The skeleton package which defines a simple Red Hat Enterprise Linux system
Description :
Basesystem defines the components of a basic Red Hat Enterprise Linux
system (for example, the package installation order to use during
bootstrapping). Basesystem should be in every installation of a system,
and it should never be removed.

 

Check linux install date / How do I find out how long a Linux server OS was installed?

Wednesday, March 30th, 2016

linux-check-install-date-howto-commands-on-debian-and-fedora-tux_the_linux_penguin_by_hello

To find out the Linux install date, there is no one single solution according to the Linux distribution type and version, there are some common ways to get the Linux OS install age.
Perhaps the most popular way to get the OS installation date and time is to check out when the root filesystem ( / ) was created, this can be done with tune2fs command

 

server:~# tune2fs -l /dev/sda1 | grep 'Filesystem created:'
Filesystem created:       Thu Sep  6 21:44:22 2012

 

server:~# ls -alct /|tail -1|awk '{print $6, $7, $8}'
sep 6 2012

 

root home directory is created at install time
 

 

server:~# ls -alct /root

 

root@server:~# ls -lAhF /etc/hostname
-rw-r–r– 1 root root 8 sep  6  2012 /etc/hostname

 

For Debian / Ubuntu and other deb based distributions the /var/log/installer directory is being created during OS install, so on Debian the best way to check the Linux OS creation date is with:
 

root@server:~# ls -ld /var/log/installer
drwxr-xr-x 3 root root 4096 sep  6  2012 /var/log/installer/
root@server:~# ls -ld /lost+found
drwx—— 2 root root 16384 sep  6  2012 /lost+found/

 

On Red Hat / Fedora / CentOS, redhat based Linuces , you can use:

 

rpm -qi basesystem | grep "Install Date"

 

basesystem is the package containing basic Linux binaries many of which should not change, however in some cases if there are some security updates package might change so it is also good to check the root filesystem creation time and compare whether these two match.

How to split / rar in parts large data archive files on Linux and Windows – Transfer big files across servers located in DMZ rescticted areas

Friday, November 28th, 2014

how-to-split-rar-in-parts-large-data-archive-files-on-Linux-and-Windows-Transfer-big-files-across-servers-in-firewalled-restricted-areas

I was working on a Application Migration Project whose goal was to Install a business application called Asset Guardian and then move current company Data from the old server to the new AppServer.
F
or that purpose the company vendor Asset Guardian shipped to a Public access FTP, a huge (12GB) ZIP archive file which had to be transferred into a well secured DMZ-ed corporation network with various implemented Traffic Shaping Network policies, a resctrictive firewall allowing access to Internal Network only and to Few (Restrictive configured) Proxy Server IPs on port 80 and 8080.

One of the proxy servers allowed access to the Internet and I set this one and tried downloading the Huge Archive file  with the Windows 2012 server default browser Internet Explorer 10, though the download started it kept slow between ~ 300 – 500KB sec and when reached 3.4GB download failed. I tried resuming the download but as the remote Public FTP server where files resides doesn't support FTP RESUME function.
I thought it might be that Internet Explorer is badly managing the download so, I go forward and installed Portable Firefox (mirrored version 33.1.1 is here). Re-running download with firefox also failed, so the next logical step was for me to try downloading with Windows version of Wget (Wget) and with Portable Free Download Manager 3.9.14.1481 (mirrored here) using both of them was unable to complete download (probably due to firewall or Proxy screwing the proxy inspected traffic) thus I had to look for another way to copy the enormous archive into the company network.

To get around the issue I tried to download the file from FTP to another Server running Apache and tried re-downloading the big file archive (Asset-Guardian-data.zip) from Apache Webserver via HTTP protocol, this download method didn't work neither using plain HTTP protocol for download when downloaded file reached (3.4GB), thus I realized this is due to restrictive Proxy servers (dropping file downloads) bigger than  3.4GBs).

Then to be able to transfer the huge 12GB file, it seems the only left option was to to chop the big file on smaller file chunks and transfer them one by one.
In my case I had the Asset-Guardian-Files.zip transferred already to the Apache (Webserver) host which is running Linux so basicly the task was to Transfer Big archive file between the SuSE Linux Enterprise Server (SLES) 11 and Windows 2012 Server.

Quickesy way to do that is by using UNIX split command, i.e.:

split -b 1024m Asset-Guardian-Files.zip


The outputted files each 1GB are with naming (xaa, xab, xac, xad, xae, xaf, gaf etc.) in same folder where split command is run:

To later merge the files on the Windows 2012 server (copy) command is used:

copy /b file1 + file2 + file3 + file4 filetogether


In my case the command to issue on Win 2012 server was:

copy /b xaa + xab + xac + xae + xae + xaf + xaf + xag xah xai xaj xak Asset-Guardian-files.zip


This method to chop and transfer the file is most simple one and it doesn't require the two servers to have WinRAR or Console RAR / unrar installed.

If instead of Copy Huge File from Linux -> Windows host you need to copy too big file (lets say 100GB) between 2 Windows servers (Windows server host A and Windows server Host B – both situated in different firewall corporate networks) you will need to download to Win Host A and use Windows UNIX split equivalent tool called sfk (The Swiss File Knife) , sfk has port also for Mac OS so in case of need for need for migrating huge archive file from Mac OS X host it will serve as Linux's split – I've made SFK (current version) mirror here.

Another way to cut the 12GB file in parts and transfer to destination host via HTTP was to use rar (on the Linux host), then download the file on Win 2012 server and use Winrar Portable Free to extract the multiple files:

To make archive separate in parts set out to certain size out of a huge file with rar on Linux use:

cd /var/www
rar -a -v1000000k Asset_Guardian_Files.splitted.rar /var/www/Asset_Guardian_Files.zip

10000000Kbs = 10000000/1024 = 976MBs, hence rar produced parts will be sized to 976MB rar parts.

To find out archives check for *splitted*.rar in your /var/www

ls -al /var/www/*splitted*.rar
-rw-r–r– 1 root root 1048576 ное 28 18:34 Asset-Guardian-Files.splitted.part1.rar
-rw-r–r– 1 root root 1048576 ное 28 18:34 Asset-Guardian-Files.splitted.part2.rar
-rw-r–r– 1 root root 1048576 ное 28 18:34 Asset-Guardian-Files.splitted.part3.rar
-rw-r–r– 1 root root 1048576 ное 28 18:34 Asset-Guaridna-Filse.splitted.part4.rar

 

Then to download the files M$ Win 2012 server IE (http://my-linux-host.com/Asset-Guardian-Files.splitted.part1.rar, http://my-linux-host.com/Asset-Guardian-Files.splitted.part2.rar. etc.)

Thanks God, Problem Solved 🙂

Preserve domain name after redirect with mod_rewrite and some useful mod rewrite redirect and other examples – Redirect domain without changing URL

Friday, July 11th, 2014

redirect_domain_name_without_changing_url_apache_rewrite_rule_preventing_host_in_ip_mod_rewrite
If you're a webhosting company sysadmin, sooner or later you will be asked by application developer or some client to redirect from an Apache webserver to some other webserver / URL's IP, in a way that the IP gets preserved after the redirect.

I'm aware of two major ways to do the redirect on webserver level:

1. To redirect From Apache host A to Webserver on host B using ReverseProxy mod_proxy

2. To use Mod Rewrite to redirect all client requests on host A to host B.

There is quite a lot to be said and is said and written online on using mod_rewrite to redirect URLs.
So in this article I will not say nothing new but just present some basic scenarios on Redirecting with mod rewrite and some use cases.
Hope this examples, will help some colleague sys-admin to solve some his crazy boss redirection tasks 🙂 I'm saying crazy boss because I already worked for a  start-up company which was into internet marketing and the CEO has insane SEO ideas, often impossible to achieve …

a) Dynamic URL Redirect from Apache host A to host B without changing domain name in browser URL and keeping everything after the query in

Lets say you want to redirect incoming traffic to DomainA to DomainB keeping whole user browser request, i.e.

Redirect:

http://your-domainA.com/whole/a/lot/of/sub/directory/query.php


Passthe the whole request including /whole/a/lot/of/sub/directory/query.php

so when Apache redirects to redirect to:

http://your-domainB.com/whole/a/lot/of/sub/directory/query.php

In browser 
To do it with Mod_Rewrite either you have to add in .htaccess mod_rewite rules:

RewriteEngine On
RewriteCond %{HTTP_HOST} ^your-domainA.com [OR]
RewriteCond %{HTTP_HOST} ^http://your-domainA.com
RewriteRule ^(.*) http://your-domainB.com/$1 [P]

or include this somewhere in VirtualHost configuration of your domain
 

Above mod_rewrite will make any request to your-domainA.com to forward to your-domainB.com while preserving the hostname in browser URL bar to old domain http://your-domainA.com, however still contet will be served by http://your-domainB.com
 

http://yourdomainA.com/YOUR-CUSTOM-REQUEST-ADDRESS


to redirect to

http://yourdomainB.com/YOUR-CUSTOM-REQUEST-ADDRESS


WARNING !!  If you're concerned about your SEO well positioning in search Engines, be sure to never ever use such redirects. Making such redirects will cause two domains to show up duplicate content
and will make Search Engines to reduce your Google, Yahoo, Yandex etc. Pagerank !!

Besides that such, redirect will use mod_rewrite on each and every redirect so from performance stand point it is a CPU killer (for such redirect using native mod_proxy ProxyPass is much more efficient – on websites with hundred of thousands of requests daily using such redirects will cause you to spend your  hardware badly  …)

P.S. ! Mod_Rewrite and Proxy modules needs to be previously enabled
On Debian Linux, make sure following links are existing and pointing to proper existing files from /etc/apache2/mods-available/ to /etc/apache2/mods-enabled

debian:~#  ls -al /etc/apache2/mods-available/*proxy*
-rw-r–r– 1 root root  87 Jul 26  2011 /etc/apache2/mods-available/proxy_ajp.load
-rw-r–r– 1 root root 355 Jul 26  2011 /etc/apache2/mods-available/proxy_balancer.conf
-rw-r–r– 1 root root  97 Jul 26  2011 /etc/apache2/mods-available/proxy_balancer.load
-rw-r–r– 1 root root 803 Jul 26  2011 /etc/apache2/mods-available/proxy.conf
-rw-r–r– 1 root root  95 Jul 26  2011 /etc/apache2/mods-available/proxy_connect.load
-rw-r–r– 1 root root 141 Jul 26  2011 /etc/apache2/mods-available/proxy_ftp.conf
-rw-r–r– 1 root root  87 Jul 26  2011 /etc/apache2/mods-available/proxy_ftp.load
-rw-r–r– 1 root root  89 Jul 26  2011 /etc/apache2/mods-available/proxy_http.load
-rw-r–r– 1 root root  62 Jul 26  2011 /etc/apache2/mods-available/proxy.load
-rw-r–r– 1 root root  89 Jul 26  2011 /etc/apache2/mods-available/proxy_scgi.load

debian:/etc/apache2/mods-avaialble:~# ls *proxy*
proxy.conf@  proxy_connect.load@  proxy_http.load@  proxy.load@


If it is is not enabled to enable proxy support in Apache on Debian / Ubuntu Linux, either create the symbolic links as you see them from above paste or issue with root:
 

a2enmod proxy_http
a2enmod proxy

 

b) Redirect Main Domain requests to other Domain specific URL
 

RewriteEngine On
RewriteCond %{HTTP_HOST} ^your-domainA.com
RewriteRule ^(.*) http://your-domainB.com/YOUR-CUSTOM-URL [P]

Note that no matter what kind of subdirectory you request on http://your-domain.com (lets say you type in http://your-domainA.com/My-monkey-sucks ) it will get redirected to:

http://your-domainB.com/YOUR-CUSTOM-URL

Sometimes this is convenient for SEO, because it can make you to redirect any requests (including mistakenly typed requests by users or Bot Crawlers to real existing landing page).

c) Redirecting an IP address to a Domain Name

This probably a very rare thing to do as usually a Domain Name is redirected to an IP, however if you ever need to redirect IP to Domain Name:

RewriteCond %{HTTP_HOST} ^##.##.##.##
RewriteRule (.*) http://your-domainB.com/$1 [R=301,L]

Replace ## with digits of your IP address, the is used to escape the (.) – dots are normally interpreted by mod_rewrite.

d) Rewritting URL extensions from .htm to .php, doc to docx etc.

Lets say you're updating an old website with .htm or .html to serve .php files with same names as old .htmls use following rewrite rules:. Or all your old .doc files are converted and replaced with .docx and you need to make Apache redirect all .doc requests to .docx.
 

Options +FollowSymlinks
RewriteEngine on
RewriteRule ^(.*).html$ $1.php [NC]

Options +FollowSymlinks
RewriteEngine on
RewriteRule ^(.*).doc$ $1.docx [NC]

The [NC] flag at the end means "No Case", or "case-insensitive"; Meaning it will not matter whether files are requested with capital or small letters, they will just show files if file under requested name is matched.

Using such a redirect will not cause Apache to redirect old files .html, .htm, .doc and they will still be accessible again creating duplicate content which will have a negavite impact on Search Engine Optimization.

The better way to do old extensioned files redirect is by using:

Options +FollowSymlinks
RewriteEngine on
RewriteRule ^(.+).htm$ http://your-domainB.com/$1.php [R,NC]

[R] flag would tell make mod_rewrite send HTTP "MOVED TEMPORARILY" redirection, aka, "302" to browser. This would cause search engines and other spidering entities will automatically update their links to the new locations.

e) Grabbing content from URL with Mod Rewrite and passing it to another domain

Lets say you want zip files contained in directory files/ to be redirected from your current webserver on domainA to domainB's download.php script and be passed as argument to the script

Options +FollowSymlinks
RewriteEngine on
RewriteRule ^files/([^/]+)/([^/]+).zip http://www.pc-freak.net/download.php?section=$1&file=$2 [R,NC]


f) Shortening URLs with mod_rewrite

This is ueful If you have a long URL address accessible via some fuzzy long hard to remember URL address and you want to make it acessible via a shorter URL without phyisally moving the files within a short named directory, do:

Options +FollowSymlinks
RewriteEngine On
RewriteRule ^james-brown /james-brown/files/download/download.php

Above rule would make requests coming to http://your-domain.com/james-brown?file=my.zip be opened via http://mysite/public/james-brown/files/download/download.php?file=my.zip

g) Get rid of the www in your domain name

Nowdays many people are used to typing www.your-domain.com, if this annoys you and you want them not to see in served URLs the annoying www nonsense, use this:

Options +FollowSymlinks
RewriteEngine on
RewriteCond %{http_host} ^www.your-domain.com [NC]
RewriteRule ^(.*)$ http://your-domain.com/$1 [R=301,NC]

That's mostly some common uses of mod rewrite redirection, there are thousands of nice ones. If you know others, please share?


References and thanks to:

How to redirect domain without changing the URL

More .htaccess tips and tricks – part 2

 

 

Linux: how to show all users crontab – List all cronjobs

Thursday, May 22nd, 2014

linux-unix-list-all-crontab-users-and-scripts
I'm doing another server services decomissioning and part of decomissioning plan is: Removing application and all related scripts from related machines (FTP, RSYNC, …). In project documentation I found a list with Cron enabled shell scripts:

#Cron tab excerpt:
1,11,21,31,41,51 * * * */webservices/tools/scripts/rsync_portal_sync.sh

that has to be deleted, however there was nowhere mentioned under what kind of credentials (with what kind of user) are the cron scripts running? Hence I had to look up all users that has cronjobs and find inside each user's cronjobs whether respective script is set to run. Herein I will explain shortly how I did that.

Cronjobs by default has few locations from where cronjobs are setupped depending on their run time schedule. First place I checked for the scripts is

/etc/crontabs # cat /etc/crontabs SHELL=/bin/sh
PATH=/usr/bin:/usr/sbin:/sbin:/bin:/usr/lib/news/bin
MAILTO=root
#
# check scripts in cron.hourly, cron.daily, cron.weekly, and cron.monthly
#
-*/15 * * * * root test -x /usr/lib/cron/run-crons && /usr/lib/cron/run-crons >/dev/null 2>&1
59 * * * * root rm -f /var/spool/cron/lastrun/cron.hourly
14 4 * * * root rm -f /var/spool/cron/lastrun/cron.daily
29 4 * * 6 root rm -f /var/spool/cron/lastrun/cron.weekly
44 4 1 * * root rm -f /var/spool/cron/lastrun/cron.monthly

I was not really user via what user is shell script run, therefore I looked first if someone doesn't set the script to run via crontab's standard locations for Daily, Hourly,Weekly and Monthly cronjobs:
 

a) Daily set cron jobs are in:

/etc/cron.daily/

b) Hourly set cron jobs:

/etc/cron.hourly

c) Weekly cron jobs are in:

/etc/cron.weekly/

d) Monthly cron jobs:

/etc/cron.monthly

There is also a location read by crontab for all Software (package distribution) specific cronjobs – all run under root user privileges.:

e) Software specific script cron jobs are in:

/etc/cron.d/  
As the system has about 327 users in /etc/passwd, checking each user's cronjob manually with:

# crontab -u UserName -l

was too much time consuming thus it is a good practice to list

/var/spool/cron/*

directory and to see which users has cron jobs defined

 

# ls -al /var/spool/cron/*
-rw——- 1 root root 11 2007-07-09 17:08 /var/spool/cron/deny

/var/spool/cron/lastrun:
total 0
drwxr-xr-x 2 root root 80 2014-05-22 11:15 .
drwx—— 4 root root 120 2008-02-25 15:45 ..
-rw-r–r– 1 root root 0 2014-05-22 04:15 cron.daily

/var/spool/cron/tabs:
total 8
drwx—— 2 root root 72 2014-04-03 03:43 .
drwx—— 4 root root 120 2008-02-25 15:45 ..
-rw——- 1 root root 4901 2014-04-03 03:43 root
 


/var/spool/cron – is crond (/usr/bin/cron/)'s spool directory.

# ls -al /var/spool/cron/tabs/ total 8
drwx------ 2 root root 72 2014-04-03 03:43 .
drwx------ 4 root root 120 2008-02-25 15:45 ..
-rw------- 1 root root 4901 2014-04-03 03:43 root

Above output shows only root superuser has defined crons.

Alternative way to check all user crontabs is via quick Linux one liner shell script show all user cron jobs

for i in $(cat /etc/passwd | sed -e "s#:# #g" | awk '{ print $1 }'); do
echo "user $i --- crontab ---";
crontab -u $i -l 2>&1 >/dev/null;
echo '----------';
done|less

Note that above short script has to run with root user. Enjoy 🙂

Debian Linux: Installing and monitoring servers with Icanga (Nagios fork soft)

Monday, June 3rd, 2013

icinga-monitoring-processes-and-servers-linux-logo

There is plenty of software for monitoring how server performs and whether servers are correctly up and running. There is probably no Debian Linux admin who didn't already worked or at least tried Nagios and Mointor to monitor and notify whether server is unreachable or how server services operate. Nagios and Munin are play well together to prevent possible upcoming problems with Web / Db / E-mail services or get notify whether they are completely inaccessible. One similar "next-generation" and less known software is Icanga.
The reason, why to use Icinga  instead of Nagios is  more features a list of what does Icinga supports more than Nagios is on its site here
I recently heard of it and decided to try it myself. To try Icanga I followed Icanga's install tutorial on Wiki.Icanga.Org here
In Debian Wheezy, Icinga is already part of official repositories so installing it like in Squeeze and Lenny does not require use of external Debian BackPorts repositories.

1. Install Icinga pre-requirement packages

debian:# apt-get --yes install php5 php5-cli php-pear php5-xmlrpc php5-xsl php5-gd php5-ldap php5-mysql

2. Install Icanga-web package

debian:~# apt-get --yes install icinga-web

Here you will be prompted a number of times to answer few dialog questions important for security, as well as fill in MySQL server root user / password as well as SQL password that will icinga_web mySQL user use.

icinga-choosing-database-type

configuring-icinga-web-debian-linux-configuring-database-shot

debian-config-screenshot-configuring-icinga-idoutils

icinga-password-confirmation-debian-linux
….

Setting up icinga-idoutils (1.7.1-6) …
dbconfig-common: writing config to /etc/dbconfig-common/icinga-idoutils.conf
granting access to database icinga for icinga-idoutils@localhost: success.
verifying access for icinga-idoutils@localhost: success.
creating database icinga: success.
verifying database icinga exists: success.
populating database via sql…  done.
dbconfig-common: flushing administrative password
Setting up icinga-web (1.7.1+dfsg2-6) …
dbconfig-common: writing config to /etc/dbconfig-common/icinga-web.conf

Creating config file /etc/dbconfig-common/icinga-web.conf with new version
granting access to database icinga_web for icinga_web@localhost: success.
verifying access for icinga_web@localhost: success.
creating database icinga_web: success.
verifying database icinga_web exists: success.
populating database via sql…  done.
dbconfig-common: flushing administrative password

Creating config file /etc/icinga-web/conf.d/database-web.xml with new version
database config successful: /etc/icinga-web/conf.d/database-web.xml

Creating config file /etc/icinga-web/conf.d/database-ido.xml with new version
database config successful: /etc/icinga-web/conf.d/database-ido.xml
enabling config for webserver apache2…
Enabling module rewrite.
To activate the new configuration, you need to run:
  service apache2 restart
`/etc/apache2/conf.d/icinga-web.conf' -> `../../icinga-web/apache2.conf'
[ ok ] Reloading web server config: apache2 not running.
root password updates successfully!
Basedir: /usr Cachedir: /var/cache/icinga-web
Cache already purged!

3. Enable Apache mod_rewrite
 

 

debian:~# a2enmod rewrite
debian:~# /etc/init.d/apache2 restart


4. Icinga documentation files

Some key hints on Enabling some more nice Icinga features are mentioned in Icinga README files, check out, all docs files included with Icinga separate packs are into:
 

debian:~# ls -ld *icinga*/
drwxr-xr-x 3 root root 4096 Jun  3 10:48 icinga-common/
drwxr-xr-x 3 root root 4096 Jun  3 10:48 icinga-core/
drwxr-xr-x 3 root root 4096 Jun  3 10:48 icinga-idoutils/
drwxr-xr-x 2 root root 4096 Jun  3 10:48 icinga-web/

debian:~# less /usr/share/doc/icinga-web/README.Debian debian:~# less /usr/share/doc/icinga-idoutils/README.Debian

5. Configuring Icinga

Icinga configurations are separated in two directories:

debian:~# ls -ld *icinga*

drwxr-xr-x 4 root root 4096 Jun  3 10:50 icinga
drwxr-xr-x 3 root root 4096 Jun  3 11:07 icinga-web

>

etc/icinga/ – (contains configurations files for on exact icinga backend server behavior)

 

/etc/icinga-web – (contains all kind of Icinga Apache configurations)
Main configuration worthy to look in after install is /etc/icinga/icinga.cfg.

6. Accessing newly installed Icinga via web

To access just installed Icinga, open in browser URL – htp://localhost/icinga-web

icinga web login screen in browser debian gnu linux

logged in inside Icinga / Icinga web view and control frontend

 

7. Monitoring host services with Icinga (NRPE)

As fork of Nagios. Icinga has similar modular architecture and uses number of external plugins to Monitor external host services list of existing plugins is on Icinga's wiki here.
Just like Nagios Icinga supports NRPE protocol (Nagios Remote Plugin Executor). To setup NRPE, nrpe plugin from nagios is used (nagios-nrpe-server). 

To install NRPE on any of the nodes to be tracked;
debian: ~# apt-get install –yes nagios-nrpe-server

 Then to configure NRPE edit /etc/nagios/nrpe_local.cfg


 

Once NRPE is supported in Icinga, you can install on Windows or Linux hosts NRPE clients like in Nagios to report on server processes state and easily monitor if server disk space / load or service is in critical state.

Captured crackers sslog mysqljackpot MySQL bruteforcer tool / exploit – Xzibit Rootkit and HIDDEN Processes Found: 1 False Positive reports

Monday, October 29th, 2012

XZibit false positive .depend.boot mysqljackpot script kiddie mysql admin user bruteforcer tool and 3 scenarios on how a server could have been hacked
I've noticed some kind of script kiddie gained access somehow on one of the servers, I administrate. A MS-SQL Scanner tool called sslog, was downloaded in tmp and run with root user credentials.

The cracked victim host is running Debian Linux Squeeze and last security update, I've made about few months ago. Inside /tmp/.a/ directory, I've found 0day MS-SQL scanner called mysqljackpot. Maybe the tool is still private exploit scanner because on the Internet I couldn't find it anywhere.:

# ls -al /tmp/.a
total 52
drwxr-xr-x 5 root root 4096 Oct 29 01:10 ./
drwxrwxrwt 10 root root 36864 Oct 29 14:46 ../
drwxr-xr-x 3 root root 4096 Oct 27 21:46 mysqljackpot/
drwxr-xr-x 3 root root 4096 Oct 28 16:58 new/
drwxr-xr-x 3 root root 4096 Oct 29 12:48 pass-multe/
# ls -al /tmp/.a/new/
total 12
drwxr-xr-x 3 root root 4096 Oct 28 16:58 ./
drwxr-xr-x 5 root root 4096 Oct 29 01:10 ../
drwxr-xr-x 3 root root 4096 Oct 29 00:58 mysqljackpot/

After further investigations, I've realized ./sslog is actually a frontend scanner program (Synscan 5.02):

root@host:/tmp/.a/new/mysqljackpot/scanner# ./sslog
Synscan 5.02 (http://www.digit-labs.org/files/tools/synscan/)
by John Anderson ,
Neil Kettle .
./sslog: getuid(): UID or EUID of 0 required

As you see in order for the scanner to run it requires to be root with superuser privileges.

mysqljackpot is actually a brute force tool which as explained in a file (README.mysql), found in its directory :

Here is content of README.mysql:

MySQL Login Scanner
By Kingcope
REQUIRES A FAST SCANSERVER!!! DONT TRY TO RUN THIS FROM A SLOW CONNECTION! ***

Scans for open mysql servers with the following credentials:
root <nopass>
root mysql
root root
admin <nopass>
admin admin
admin mysql
mysql <nopass>
mysql mysql

Runs on linux.
Requirements: mysql development libraries and headers
Compile (try one of the following depending on your system):
$ ./configure LIBS=-lmysqlclient
OR
$ ./configure LIBS="-L/usr/lib/mysql" -lmysqlclient
OR
$ ./configure LIBS="-L/usr/lib64/mysql" -lmysqlclient
OR
$ ./configure CFLAGS="-lmysqlclient"

afterwards type

$ make linux

Run:
terminal 1:
./sslog -v
terminal 2:
./synscan -b <ip block> -p 3306

Inspect Logfile "mysqljack.pot" for open servers.
 

There is one other README in /tmp/.a/new/mysqljackpot/README, here is what I found in it:

less README
Oracle MySQL on Windows Remote SYSTEM Level Exploit zeroday
All owned By Kingcope

Installation Instructions
=============================

1. Install mysql client libraries and headers (UNIX)
RedHat based (e.g. CentOS):
yum install mysql mysql-devel

2. Compile the standalone exploit
issue commands:
gcc mysqljackpot.c -o mysqljackpot -L/usr/lib/mysql -lmysqlclient

3. Compile the reverse shell payload (this is required!)
required because the connect back ip and port are hardcoded in the dll:
use mingw on windows or wine
change REVERSEIP and REVERSEPORT to suit your needs. If you change REVERSEPORT you have
to change the port in mysqljackpot.c too (default port: 443).
issue commands:
set PATH=%PATH%;c:\MinGW\bin\
gcc -c payload.c
gcc -shared -o payload.dll payload.o -lws2_32
copy the payload.dll into the mysqljackpot exploit folder

4. Run The Exploit
./mysqljackpot -u root -p "" -t 99.99.99.99
A valid database admin user and his password are required
for the exploit to work properly.
This exploit is especially useful when used in connection
to a MySQL login scanner, see scanner/README.mysql inside this package.
Be sure to have the firewall open on the desired reverse port
on the attacking machine.

5. Enjoy your SYSTEM Shell!!!

Yours Sincerely,

— Kingcope
 

 

Here is also the header from mysqljackpot.c mysql username brute force tool:

/* Oracle MySQL on Windows Remote SYSTEM Level Exploit zeroday
 * Copyright (C) 2012 Kingcope
 *
 * Thanks to danny.
 */
 

After thinking over the security breach I thought of  few scenarios on how the attacker entered and run as root superuser. One is;

  •   Cracker entered directly via SSH after sniffing somehow the root password.

After however, a review of last cmd, I've concluded this case is not very likely, e.g.:

# last |grep -i root

did not found any logs, of unusual root logins, neither there seem to be any unusual activity with logins with other non-root users. Of course it is possible someone logged in as root and used some tool to clean, his tracks with some kind of user log-cleaner tool like the one I've written in past in bash this doesn't seem very likely however because. It seem the /tmp/.a/, directory was created by some amateur script kiddie, a professional one would create some a bit smarter directory like for example just few empty spaces , i.e. would create it with, lets say::

# mkdir "   "

instead of the so trivial

# mkdir /tmp/.a/

Also the name of the directory containing the script kiddie tool /tmp/.a is not selected intelligently, but just done in a hurry, hence I even assume /tmp/.a, is created by some automated SK tool writen in hurry by some Romanian SK Cracker 🙂

On the host there was webmin and usermin running. So;

  • my second assumption was it could be someone sniffed a login password via encrypted SSL connection, whether the root logged in via webmin, or somehow exploited usermin (though I should say usermin (which listens by default on port number 20000)
     

TCP port 20000 on which usermin listens by default is filtered by an iptables rules for all hosts incoming connections, whether webmin logins are permitted only from few IP addresses. Thus this scenario, though more possible than a direct SSH login with root sniffed password still seems to me not very probable.

  • Therefore as a third scenario (most likely what happened), I assume some of the PHP forms on the server or some other undefined PHP excecutable via Apache variable script was missing definition.

 

Actually saw in /var/log/apache2/error.log plenty of re-occuring warnings of existing undefined variables:

[Mon Oct 29 16:30:43 2012] [error] [client 213.149.142.73] PHP Notice:  Undefined variable: not_assign in /home/site_dir/www/modules/start.mod.php on line 121, referer: http://site-domain-name.com/start?qid=3&answered_id=4
[Mon Oct 29 16:30:43 2012] [error] [client 213.149.142.73] PHP Notice:  Undefined variable: counter_cookie in /home/site_dir/www/modules/start.mod.php on line 130, referer: http://site-domain-name.com/start?qid=3&answered_id=4
[Mon Oct 29 16:30:43 2012] [error] [client 213.149.142.73] PHP Notice:  Undefined variable: campaign_cukie in /home/site_dir/www/modules/start.mod.php on line 135, referer: http://site-domain-name.com/start?qid=3&answered_id=4
[Mon Oct 29 16:30:43 2012] [error] [client 213.149.142.73] PHP Notice:  Undefined index: actions in /home/site_dir/www/counter/count.php on line 11, referer: http://site-domain-name/start?qid=3&answered_id=4
[Mon Oct 29 16:30:43 2012] [error] [client 213.149.142.73] PHP Notice:  Undefined variable: flag2 in /home/site_dir/www/counter/count.php on line 52, referer: http://iqtest.bg/start?qid=3&answered_id=4
 

Taking this in consideration, I assume the attacker, entered the system finding about the undefined variables, defining them and somehow achieving access to the www-data Apache user shell, and through this shell running some 0day Linux kernel exploit to gain root access and download and install mysqljackpot exploit scanner tool.

Logically as it is common in situations like this, I used rkhunter, chkrootkit and unhide tools to check if the server's main binaries and kernel modules are compromised and is there a rootkit installed (earlier written a post on that here)

In short to do checks, installed rkhunter, chkrootkit and unhide with apt-get (as this is a Debian Squeeze server):

apt-get install --yes rkhunter unhide chkrootkit
….

 

Afterwards run in a row:
 

# for i in $(echo proc sys brute); do unhide $i; done
....
......
# chkrootkit
...
.....
# rkhuter --check
...
....

Reports, of the three ones are like so:

Unhide 20100201
http://www.security-projects.com/?Unhide

[*]Searching for Hidden processes through /proc scanning
# for i in $(echo proc sys brute); do unhide $i; done

[*]Starting scanning using brute force against PIDS with fork()

Unhide 20100201
http://www.security-projects.com/?Unhide

[*]Searching for Hidden processes through kill(..,0) scanning

[*]Searching for Hidden processes through  comparison of results of system calls

[*]Searching for Hidden processes through getpriority() scanning

[*]Searching for Hidden processes through getpgid() scanning

[*]Searching for Hidden processes through getsid() scanning

[*]Searching for Hidden processes through sched_getaffinity() scanning

[*]Searching for Hidden processes through sched_getparam() scanning

[*]Searching for Hidden processes through sched_getscheduler() scanning

[*]Searching for Hidden processes through sched_rr_get_interval() scanning

[*]Searching for Hidden processes through sysinfo() scanning

HIDDEN Processes Found: 1
Unhide 20100201
http://www.security-projects.com/?Unhide

Found HIDDEN PID: 4994
Found HIDDEN PID: 13374
Found HIDDEN PID: 14931
Found HIDDEN PID: 18292
Found HIDDEN PID: 19199
Found HIDDEN PID: 22651
[*]Starting scanning using brute force against PIDS with Threads

Found HIDDEN PID: 3296
Found HIDDEN PID: 30790

# chkrootkit -q

/usr/lib/pymodules/python2.5/.path /usr/lib/pymodules/python2.6/.path /lib/init/rw/.ramfs

# rkhunter –check

System checks summary
=====================

File properties checks…
    Files checked: 137
    Suspect files: 0

Rootkit checks…
    Rootkits checked : 245
    Possible rootkits: 2
    Rootkit names    : Xzibit Rootkit, Xzibit Rootkit

Applications checks…
    All checks skipped

The system checks took: 1 minute and 5 seconds

All results have been written to the log file (/var/log/rkhunter.log)

One or more warnings have been found while checking the system.
Please check the log file (/var/log/rkhunter.log)

 

Reports from unhide and chkrootkit,  not seem troubling, however I was concerned about the report from rkhunter – Rootkit names    : Xzibit Rootkit, Xzibit Rootkit.

To get some more info on why chkrootkit, thinks, system is infected with Xzibit (which by the way is an artistic alias of a RAP singer from the 1980's 🙂 I check in /var/log/rkhunter.log

 

# grep -i xzibit /var/log/rkhunter.log
[16:52:48] Checking for Xzibit Rootkit...
[16:52:48] Xzibit Rootkit                                    [ Not found ]
[16:52:56]          Found string 'hdparm' in file '/etc/init.d/hdparm'. Possible rootkit: Xzibit Rootkit
[16:52:56]          Found string 'hdparm' in file '/etc/init.d/.depend.boot'. Possible rootkit: Xzibit Rootkit
[16:53:01] Rootkit names    : Xzibit Rootkit, Xzibit Rootkit
 

Onwards I checked content of hdparm and .depend.boot and there I don't see nothing irregular. They both are files from legitimate Debian install, I've checked if they belong to a deb packages as well if they are existing on other Debian Squeeze servers I administer as well as on my Debian Desktop notebook, everywhere they're present, hdparm is part of hdparm deb and .depend.boot is loaded by /etc/init.d/rc script, containing some user string references:

# grep -rli .depend.boot *
rc
 

# dpkg -S /etc/init.d/hdparm
# hdparm: /etc/init.d/hdparm
  /etc/init.d/.depend.boot
# dpkg -S /etc/init.d/.depend.boot
dpkg: /etc/init.d/.depend.boot not found.

 

Another troubling thing was unhide's return:

HIDDEN Processes Found: 1

 

After a close examination of the system as well as research on the internet, I've figured out this is also a false positive. For sake of not distributing, Script Kiddie tools, which might put in danger other system administrators I will not put a download link to mysqljackpot publicly. Anyways if someone is willing to have it for study purposes, just drop me a mail and I will post you temporary download link to it.

 

Also as webmin and usermin is not frequently used, I've decided to completely stop and disable them to load on boot.

I've done also a clamav scan with (lowered priority) over the whole file system with:

# nice -19 clamscan -r /*

in order to determine, if there is no PHPShell or some kind of other remote admin Script kiddie script in perl / php etc. installed.
Tomorrow, I will continue investigatin what is happening and hopefully once I got, how the abuser entered the server will update this post.