Posts Tagged ‘usr bin’

Simple bash shell Script to easy and quickly automate deploy RHEL Linux Virtual Machines on KVM Virtualization

Friday, November 3rd, 2023

how-to-install-a-kvm-guest-os-from-the-commandline-easily-with-script-virt-install-logo

Earlier I've blogged in about howto build KVM Virtual Machine RHEL 8.3 Linux install on Redhat 8.3 Linux Hypervisor with custom tailored kickstart.cfg as one of the Projects i was involved in my work duty as system administrator, we had the task to build
a KVM virtual machines and build a High Availability Linux  PCS Corosync / Pacemaker / haproxy cluster on it to save some money for the company from purchasing VMWare licenses.

The setup of the KVM Virtual machine on a first glimpse is relatively simple and I thought, this can be done just for 2 / 3 days but it turned out to take up 2 weeks or so to properly prepare the kickstarter file and learn a bit about basic virt-install KVM options and experiment with them until
we can produce a noral working Virtual machines. 

The original developed simple script that I used to bring up a new KVM virtual machines in a bit easier way virt-install-kvm.sh looked like so:
 

#!/bin/sh
# Script to build a new VM based on a kickstart.cfg file template
# with virt-install
# Author Georgi Georgiev hip0
# hipo@Pc-freak.net

KS_FILE='kickstart.cfg';
VM_NAME='RHEL8_3-VirtualMachine';
VM_DESCR='CentOS 8.3 Virtual Machine';
RAM='8192';
CPUS='8';
# size is in Gigabytes
VM_IMG_SIZE='70';
ISO_LOCATION='/vmprivate/rhel-server-8.3-x86_64-dvd.iso';
VM_IMG_FILE_LOC='/vmprivate/RHEL8_3-VirtualMachine.img';

virt-install -n "$VMNAME" –description "$VM_DESCR" –os-type=Linux –os-variant=rhel8.3 –ram=8192 –vcpus=8 –location="$ISO_LOCATION" –disk path=$VM_IMG_FILE,bus=virtio,size=$IMG_VM_SIZE –graphics none –initrd-inject=/root/$KS_FILE –extra-args "console=ttyS0 ks=file:/$KS_FILE"

 

The script basicly did what it was aimed for but modifying the kickstarter ks.cfg every time with multiple additional parameters, like network configurations as well some additional modifications to ks.cfg for VM parameters was annoying.

Recently as we had to repeat the same task again in order to migrate old customer containing a Linux OpenVZ Virtual machines to a newer  OS installed RHEL versionb with KVM virtual machines, i've took some hours and scripted a small script that would easiy our task to build new KVM Virtual Machines
from scratch relatively easy, if we have to repeat the Linux OS build operation again and again.

Before proceeding to use the script of course one has to use LVM and setup the partitions on the Host which will be the Hypervisor where the KVM VMs will be installed as well you need to download an ISO image of Redhat Enterprise Linux / CentOS / Fedora or whatever kind of RPM based Linux
you would like to setup as well as do the basic configurations regarding the emulated Hardware node "Power" of the new Virtual Machine (CPU / Memory / Disk Partition) as well as choose a proper meaningful VM name (preferrably following some good meaningful crafted naming convention, that will talk a bit about what is inside the KVM VM container).

The script that automates a bit the KVM VM installation which you find below you can also download from here virt-install.sh

 

#!/bin/bash
virt_install_path=$(which virt-install);
vm_host='vmname01';
boot_iso='/vmprivate/rhel-8.7-x86_64-dvd.iso';
os_version='rhel8.7';
vm_cpus_count='4';
vm_ram='6144';
vm_img_location='/vmprivate/NAME-of-VM.img';
vm_machine_description='Name Production system';
ks_file_location_template='/root/ks.cfg.templ';
ks_file_location='/root/ks.cfg';
ks_vm_read_loc=$(echo $ks_file_location |sed -e 's#root/##g');
vm_main_ip='192.168.23.52';
vm_netmask='255.255.255.192';
vm_gateway='192.168.23.33';
vm_nameserver='172.30.50.2';
#–ip=192.168.233.52 –netmask=255.255.255.192 –gateway=192.168.233.33 –nameserver=172.20.88.2

echo "Checking if defined $vm_host image $vm_img_location is not already present";
if [ ! -f $vm_img_location ]; then
echo
else
echo 'Exiting $vm_img_location is present';
echo "To destroy exiting VM image $vm_img_location run manually cmds:";
echo 'virsh list; virsh destroy $vm_host; virsh undefine $vm_host –remove-all-storage';
exit 1;
fi

alias cp='cp -f';
/usr/bin/cp -rpf $ks_file_location_template $ks_file_location;

echo "Setting  VM Main IP: $vm_main_ip";
echo "Setting  VM netmask: $vm_netmask";
echo "Setting  VM Gateway: $vm_gateway";
echo "Setting  VM Nameserver $vm_nameserver";
/usr/bin/perl -pi -e "s/ip=a1.a2.a3.a4/ip=$vm_main_ip/" $ks_file_location
/usr/bin/perl -pi -e "s/netmask=b1.b2.b3.b4/netmask=$vm_netmask/" $ks_file_location
/usr/bin/perl -pi -e "s/gateway=c1.c2.c3.c4/nameserver=$vm_nameserver/" $ks_file_location
/usr/bin/perl -pi -e "s/nameserver=d1.d2.d3.d4/gateway=$vm_gateway/" $ks_file_location
/usr/bin/perl -pi -e "s/vm-hostname/$vm_host/" $ks_file_location

echo "Running VM install:";
echo $virt_install_path -n $vm_host –description "$vm_machine_description" –os-type=Linux –os-variant=$os_version –ram=$vm_ram –vcpus=$vm_cpus_count –location=$boot_iso –disk path=$vm_img_location,bus=virtio,size=90 –graphics none –initrd-inject=$ks_file_location –extra-args "console=ttyS0 ks=file:$ks_vm_read_loc"

$virt_install_path -n $vm_host –description "$vm_machine_description" –os-type=Linux –os-variant=$os_version –ram=$vm_ram –vcpus=$vm_cpus_count –location=$boot_iso –disk path=$vm_img_location,bus=virtio,size=90 –graphics none –initrd-inject=$ks_file_location –extra-args "console=ttyS0 ks=file:$ks_vm_read_loc"
 

 

Modify the script to insert the required parameters of the new VM in the script header session, you will have to provide below options.

 

vm_host='vmname01';
boot_iso='/vmprivate/rhel-8.7-x86_64-dvd.iso';
os_version='rhel8.7';
vm_cpus_count='4';
vm_ram='6144';
vm_img_location='/vmprivate/NAME-of-VM.img';
vm_machine_description='Name Production system';
ks_file_location_template='/root/ks.cfg.templ';
ks_file_location='/root/ks.cfg';
ks_vm_read_loc=$(echo $ks_file_location |sed -e 's#root/##g');
vm_main_ip='192.168.23.52';
vm_netmask='255.255.255.192';
vm_gateway='192.168.23.33';
vm_nameserver='172.30.50.2';

The automatic Build virtual machine script is tested and works with Redhat Enterprise Linux and of course is pretty primitive as there is so much available online that would do similar, but still I like it because it works for me 🙂

You will need also the ks.cfg.templ file which has the basic kickstarter configuration that will bring up the Virtual machines according to predefined configurations.
Here is the ks.cfg.templ and ks.cfg files as well, you will have to place them under /root/ or some other directory.
Of course this script is just a basic one, it can be easily updated to accept its variable options as arguments if you need to bring up a multitude of virtual machines relatively quickly with few minor modifications. 

Hope the script is helpful to some sysadmin out there. If so don't forget to donate me for a beer in my Patreon account found in the widgets section 🙂

Fix ruby: /usr/lib/libcrypt.so.1: version `XCRYPT_2.0′ not found in apt upgrade on Debian Linux 10

Saturday, August 5th, 2023

I've an old legacy Thinkpad Laptop that is for simplicty running Window Maker Wmaker which was laying on my home desk for almost an year and I remembered since i'm for few days in my parents home in Dobrich that it will be a good idea to update its software to the latest Debian packages to patch security issues with it. Thus if you're like me and  you tried to update your Debian 10 Linux to the latest Stable release debian packages  and you end up into a critical error that is preventing apt to to resolve conflicts (fix it with) cmds like:

# apt-get update –fix-missing

# apt –fix-broken install

As usual I looked into Google to see about solution and found few articles, claiming to have scripts that fix it but at the end nothing worked.
And the shitty error occured during the standard:

# apt-get update && apt-get upgrade

ruby: /usr/lib/libcrypt.so.1: version `XCRYPT_2.0' not found

Hence the cause and work around seemed to be very unexpected.
For some reason debian makes a link

root@noah:/lib# ls -al /lib/libcrypt.so.1
lrwxrwxrwx 1 root root 19 Aug  3 16:53 /lib/libcrypt.so.1 ->
libcrypt.so.1.bak

root@noah:/lib# ls -al /lib/libcrypt.so.1.bak
lrwxrwxrwx 1 root root 16 Jun 15  2017 /lib/libcrypt.so.1.bak -> libcrypt-2.24.so

Thus to resolve it and force the .deb upgrade package to continue it is up to simply deleting the strange simlink and re-run the

# apt-get update && apt-get upgrade

Setting up libc6:i386 (2.31-13+deb11u6) …
/usr/bin/perl: /lib/libcrypt.so.1: version `XCRYPT_2.0' not found (required by /usr/bin/perl)
dpkg: error processing package libc6:i386 (–configure):
 installed libc6:i386 package post-installation script subprocess returned error exit status 1
Errors were encountered while processing:
 libc6:i386

Few more times. If you get some critical apt failures still, each time make sure to rerun the command after doing a simple removal of the strange simbolic link with cmd:

# rm -f /lib/libcrypt.so.1

That's all folks after a short while your Debian will be updated to latest Enoy folks ! 🙂

Getting Console and Graphical hardware system information on Linux with cpuinfo, neofetch, CPU-X (CPU-Z Unix alternative), I-nex and inxi

Tuesday, September 17th, 2019

getting-console-information-and-graphical-hardware-system-information-Linux-cpuinfo-neofetch-cpu-x-i-nex-1

Earlier I've wrote extensive article on how to get hardware information on Linux using tools such as dmidecode, hardinfo, lshw, hwinfo, x86info and biosdecode but there are few other hardware reporting tools for Linux worthy to mention that has been there for historical reasons such as cpuinfo as we as some new shiny ones such as neofetch (a terminal / console hardware report tool as well the CPU-X and I-Nex  which is Linux equivalent to the all known almost standard for Windows hardware detection CPU-Z worthy to say few words about.
 

1. cpuinfo

 

Perhaps the most basic tool to give you a brief information about your Processor type (model) number of Cores and Logical Processors is cpuinfo

I remember cpuinfo has been there since the very beginning on almost all Linux distributions's repository, nowadays its popularity of the days when the kings on the Linux OS server scenes were Slackware, Caldera OpenLinux and Redhat 6.0 Linux and Debian 3.0  declined but still for scripting purposes it is handy small proggie.

To install and run it in Debian  / Ubuntu / Mint Linux etc.:

 

aptitude install -y cpuinfo

/usr/bin/cpu-info

 

Linux-get-processor-system-info-in-console-cpu-info

 

2. neofetch

 

The next one worthy to install and check is neofetch (a cross-platform and easy-to-use system information
 command line script that collects your Linux system information and display it on the terminal next to an image, it could be your distributions logo or any ascii art of your choice.)

The cool thing about neofetch is besides being able to identify the System server / desktop hardware parameters, it gives some basic info about number of packages installed on the system, memory free and in use, used kernel and exact type of System (be it Dell PowerEdge Model XX, IBM eSeries Model / HP Proliant Model etc.

neofetch-OS-hardware-information-Linux-ascii-system-info-desktop-notebook

neofetch info generated on my home used Lenovo Thikpad T420

neofetch-OS-hardware-information-Linux-ascii-system-info-pcfreak-home-server
neofetch info from www.pc-freak.net running current machine

neofetch even supports Mac OS X and Windows OS ! 🙂

To install neofetch on Mac OS X:
 

/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"


or via Mac ported packages using brew

brew install neofetch


neofetch-screenshot-from-Mac-OS-X

neofetch is even installable on Windows OS that has the scoop command line installer tool installer manager with below PowerShell code in cmd.exe (Command line):

powershell Set-ExecutionPolicy RemoteSigned -scope CurrentUser
iex (new-object net.webclient).downloadstring('https://get.scoop.sh')
scoop install git
scoop install neofetch

neofetch-microsoft-windows-hardware-command-line-report-tool-screenshot


By the way Scoop was quite a finding for me and it is pretty handy to install plenty of useful command line Linux / UNIX tools, such as curl, wget, git etc. in the same easy straight forward way as a standard yum or apt-get on Windows (without explicitly installing things as GnuWin and CygWin).
 

3. CPU-X graphical user interface hardware report Linux GUI alternative to Windows CPU-Z


The packages for CPU-X are a bit outdated and even though there are rpm packages for Fedora, OpenSuSE and .deb package for Debian for Debian, Ubuntu and ArchLinux (pacman), there is no up to date version for Debian 10 and the package builds distributed for different Linux distros are a bit outdated.

Thus to install CPU-X on any Linux distribution it is perhaps best to use the portable version (static binary) of CPU-X.
It is currently available on https://github.com/X0rg/CPU-X/releases

To install latest portable version of CPU-X

wget https://github.com/X0rg/CPU-X/releases/download/v3.2.4/CPU-X_v3.2.4_portable.tar.gz

mkdir CPU-X
cd CPU-X

tar -zxvvf CPU-X_v3.2.4_portable.tar.gz
-rwxr-xr-x yohan/users 4563032 2019-01-13 22:15 CPU-X_v3.2.4_portable.bsd64
-rwxr-xr-x yohan/users 5484968 2019-01-13 22:15 CPU-X_v3.2.4_portable.linux64

 

cp -rpf CPU-X_v3.2.4_portable.linux64 /usr/local/bin/
ln -sf /usr/local/bin/CPU-X_v3.2.4_portable.linux64 /usr/local/bin/cpu-x


Next run as superuser (root)
 

hipo@jeremiah:~$ su -c 'cpu-x'

 

As seen from below screenshots cpu-x reports a lot of concrete specific hardware data on:

  • Processor
  • Motherboard
  • Memory
  • System
  • Graphic card
  • Performance

cpu-x-cpu-cpu-z-alternative-linux-screenshot-CPU-info

cpu-x-cpu-cpu-z-alternative-linux-screenshot-caches-info

cpu-x-cpu-cpu-z-alternative-linux-screenshot-Motherboard-info

cpu-x-cpu-cpu-z-alternative-linux-screenshot-memory-info

cpu-x-cpu-cpu-z-alternative-linux-screenshot-system-info

cpu-x-cpu-cpu-z-alternative-linux-screenshot-graphics-info

CPU-X can be installed also on FreeBSD very easily by just installing from BSD port tree sysutils/cpu-x/
It is also said to work on other *BSDs, NetBSD, OpenBSD Unixes but I guess this will require a manual compilation based on FreeBSD's port Makefile.

4. I-Nex another GUI alternative to CPU-Z for UNIX / Linux

I-Nex is even more useful for general hardware reporting as it reports many hardware specifications not reported by CPU-X such as Battery type and Model Name  (if the hardware report is on a laptop), info on USB devices slots or plugged USB devices brand and specifications, the available Network devices on the system (MAC Addresses) of each of it, Installed and used drivers on Hard Disk (ATA / SATA / SCSI / SSD), HW Sector size, Logical Block size, HDD Sectors count and other specific Hard Drive data as well as information on available Audio (Sound Blaster) devices (HDA-Intel), used Codecs, loaded kernel ALSA driver, Video card used and most importantly indicators on Processor reported CPU (temperature).

 

To install I-nex

Go to https://launchpad.net/i-nex or any of the mirror links where it resides and install the respective package, in my case, I was doing the installation on Debian Linux, so fetched current latest amd64 package which as of moment of writting this article is i-nex_7.6.0-0-bzr977-20161012-ubuntu16.10.1_amd64.deb , next installed it with dpkg
 

dpkg -i i-nex_7.6.0-0-bzr977-20161012-ubuntu16.10.1_amd64.deb

 

As the package was depending on some other .deb packages, which failed to install to install the missing ones I had to further run
 

apt –fix-broken install

i-nex-cpu-info-linux-hardware-info-program

 

hre

I-Nex thermal indicators about CPU temperature on a Linux Desktop notebook

i-nex-gpu-info-linux-hardware-info-program

i-nex-mobo-info-linux-hardware-info-program

i-nex-audio-info-linux-hardware-info-program

i-nex-drivers-info-linux-hardware-info-program

i-nex-system-info-linux-hardware-info-program

i-nex-battery-info-linux-hardware-info-program

 

There are other Hardware identification report tools such as CUDA-Z that are useful to check if you have Nvidia Video Card hardware Installed on the PC to check the status of CUDA enabled GPUs, useful if working with nVidia Geforce, Quadro, Tesla cards and ION chipsets.

If you use it however be aware that CUDA-Z is not compatible with 3rd-party linux drivers for NVidia so make sure you have the current official Nvidia version.

 

5. Inxi full featured system information script

 

Inxi is a 10000 lines mega bash script that fetches hardware details from multiple different sources in /proc /sys and from commands on the system, and generates a beautiful looking console report that non technical users can read easily.

inxi-10-k-mega-bash-shell-script-reporting-on-installed-system-computer-hardware

 

inxi -Fx

 

 

inxi-report-on-installed-hardware-on-my-lenovo-thinkpad-home-laptop

Each of the pointed above tools has different method of collection of Hardware information from various resources e.g. – kernel loaded modules, dmesg, files like /proc/meminfo /proc/version /proc/scsi/scsi /proc/partitions.
Hence some of the tools are likely to report more info than otheres, so in case if some information you need regarding the system plugged in hardware is missing you can perhaps obtain it from another program. Most Linux distribution desktop provided GNOME package are including Hardinfo gui tool, but in many cases above mentioned tools are likely to add even more on info on what is inside your PC Box.
If you're aware of others tools that are useful not mentioned here please share it.

Virtual Keyboard for Linux and other Freedom respecting operating Systems

Monday, July 30th, 2018

How to install and Use Linux Virtual Keyboard and other freedom respecting Operating Systems

  •  Looking for a quick way to use VIRTUAL KEYBOARD ON LINUX COMPUTER OPERATING SYSTEM, you can do it just this 1 task in 3 simple steps  ???
    – Logical question emerges, WHY ??? would you need a virtual keyboard on Free Software OS such as Linux?
    Well, just because sometimes it is much more secure to use a Virtual Keyboard, especially if you have doubt that your keyboard has been tapped or a Key Logger (Sniffer), intercepting the Keyboard IN / OUT jacks, is installed on the computer or you might have sit on a computer of ,a friend running Linux, and you want to make sure he did not install sniffer to intercept your ,SSH login passwords and ,later hack into your Servers, after stealing, the password

 

  • Assuming you're on : – Debian / Ubuntu Linux, or other of the numerous IT systems such as ,FreeBSD / OpeBSD etc. out there, you can run simply this commands:

     

  •  apt-get install –yes florence
    * A. To make it, easily invokable for laters, create a small bash, shell script in directory; – location /usr/bin/virtual-keyboard like, the one below:

    vim /usr/bin/virtual-keyboard

    * B.. INside the file Place following 1 liner code
     

    #!/bin/sh
    /usr/bin/florence

     

    * C… To later invoke it any time:
    Press ALT + F2 (or use Run Command Dialog in GNOME / KDE / Windomaker / IceWM whatever or any other crazy graphic environment of your choice and run:

    /usr/bin/virtual-keyboard

 

How to disable tidy HTML corrector and validator to output error and warning messages

Sunday, March 18th, 2012

I've noticed in /var/log/apache2/error.log on one of the Debian servers I manage a lot of warnings and errors produced by tidy HTML syntax checker and reformatter program.

There were actually quite plenty frequently appearing messages in the the log like:

...
To learn more about HTML Tidy see http://tidy.sourceforge.net
Please fill bug reports and queries using the "tracker" on the Tidy web site.
Additionally, questions can be sent to html-tidy@w3.org
HTML and CSS specifications are available from http://www.w3.org/
Lobby your company to join W3C, see http://www.w3.org/Consortium
line 1 column 1 - Warning: missing <!DOCTYPE> declaration
line 1 column 1 - Warning: plain text isn't allowed in <head> elements
line 1 column 1 - Info: <head> previously mentioned
line 1 column 1 - Warning: inserting implicit <body>
line 1 column 1 - Warning: inserting missing 'title' element
Info: Document content looks like HTML 3.2
4 warnings, 0 errors were found!
...

I did a quick investigation on where from this messages are logged in error.log, and discovered few .php scripts in one of the websites containing the tidy string.
I used Linux find + grep cmds find in all php files the "tidy "string, like so:

server:~# find . -iname '*.php'-exec grep -rli 'tidy' '{}' ;
find . -iname '*.php' -exec grep -rli 'tidy' '{}' ; ./new_design/modules/index.mod.php
./modules/index.mod.php
./modules/index_1.mod.php
./modules/index1.mod.php

Opening the files, with vim to check about how tidy is invoked, revealed tidy calls like:

exec('/usr/bin/tidy -e -ashtml -utf8 '.$tmp_name,$rett);

As you see the PHP programmers who wrote this website, made a bigtidy mess. Instead of using php5's tidy module, they hard coded tidy external command to be invoked via php's exec(); external tidy command invocation.
This is extremely bad practice, since it spawns the command via a pseudo limited apache shell.
I've notified about the issue, but I don't know when, the external tidy calls will be rewritten.

Until the external tidy invocations are rewritten to use the php tidy module, I decided to at least remove the tidy warnings and errors output.

To remove the warning and error messages I've changed:

exec('/usr/bin/tidy -e -ashtml -utf8 '.$tmp_name,$rett);

exec('/usr/bin/tidy --show-warnings no --show-errors no -q -e -ashtml -utf8 '.$tmp_name,$rett);

The extra switches meaning is like so:

q – instructs tidy to produce quiet output
-e – show only errors and warnings
–show warnings no && –show errors no, completely disable warnings and error output

Onwards tidy no longer logs junk messages in error.log Not logging all this useless warnings and errors has positive effect on overall server performance especially, when the scripts, running /usr/bin/tidy are called as frequently as 1000 times per sec. or more

No space left on device with free disk space / Why no space left on device while there is plenty of disk space on drive – Running out of Inodes

Tuesday, November 17th, 2015

no_space_left-on-device-while-there-is-disk-space-running-out-of-file-inodes-unix_linux_file_system_diagram.gif

 

On one of the servers, I'm administrating the websites started showing some Mysql database table corrup errors like:
 

 

Table './database_name/site_news_list_com' is marked as crashed and last (automatic?) repair failed

The server is using Oracle MySQL server community stable edition on Debian GNU / Linux 6.0, so I first thought during work the server crashed either due to some bug issue in MySQL or it crashed due to some PHP cron job that did something messy. Thus to solve the crashed tables, tried using mysqlcheck tool which helped pretty fine, at many times whether there were database / table corruptions. I've run the following set of mysqlcheck commands with root (superuser) in a bash shell after logging in through SSH:

:

server:~# /usr/bin/mysqlcheck –defaults-extra-file=/etc/mysql/debian.cnf \–check –all-databases -u root -p`grep -i password /root/.my.cnf |sed -e 's#password=##g'`>> /var/log/cronwork.log
server:~# /usr/bin/mysqlcheck –defaults-extra-file=/etc/mysql/debian.cnf –analyze –all-databases -u root -p`grep -i password /root/.my.cnf |sed -e 's#password=##g'`>> /var/log/cronwork.log
server:~# /usr/bin/mysqlcheck –defaults-extra-file=/etc/mysql/debian.cnf \–auto-repair –optimize –all-databases -u root -p`grep -i password /root/.my.cnf |sed -e 's#password=##g'`>> /var/log/cronwork.log
server:~# /usr/bin/mysqlcheck –defaults-extra-file=/etc/mysql/debian.cnf \–optimize –all-databases -u root -p`grep -i password /root/.my.cnf |sed -e 's#password=##g'`>> /var/log/cronwork.log


In order for above commands to work, I've created the /root/.my.cnf containing my root (mysql CLI) mysql username and password, e.g. file has content like below:

 

[client]
user=root
password=MySecretPassword8821238

 

Btw a good note here is its generally a good idea (if you want to have consistent mysql databases) to automatically execute via a cron job 2 times a month, I've in root cronjob the following:

 

crontab -u root -l |grep -i mysqlcheck
04 06 5,10,15,20,25,1 * * /usr/bin/mysqlcheck –defaults-extra-file=/etc/mysql/debian.cnf \–check –all-databases –silent -u root -p`grep -i password /root/.my.cnf |sed -e 's#password=##g'`>> /var/log/cronwork.log 07 06 5,10,15,20,25,1 * * /usr/bin/mysqlcheck –defaults-extra-file=/etc/mysql/debian.cnf –analyze –all-databases –silent -u root -p`grep -i password /root/.my.cnf |sed -e 's#password=##g'`>> /var/log/cronwork.log 12 06 5,10,15,20,25,1 * * /usr/bin/mysqlcheck –defaults-extra-file=/etc/mysql/debian.cnf \–auto-repair –optimize –all-databases –silent -u root -p`grep -i password /root/.my.cnf |sed -e 's#password=##g'`>> /var/log/cronwork.log 17 06 5,10,15,20,25,1 * * /usr/bin/mysqlcheck –defaults-extra-file=/etc/mysql/debian.cnf \–optimize –all-databases –silent -u root -p`grep -i password /root/.my.cnf |sed -e 's#password=##g'`>> /var/log/cronwork.log


Strangely I got a lot of errors that some .MYI / .MYD .frm temp files, necessery for the mysql tables recovery can't be written inside /home/mysql/database_name

That was pretty weird and I thought there might be some issues with permissions, causing the inability to write, due to some bug or something so I went straight and checked /home/mysql/database_name permissions, e.g.::

 

server:/home/mysql/database_name# ls -ld soccerfame
drwx—— 2 mysql mysql 36864 Nov 17 12:00 soccerfame
server:/home/mysql/database_name# ls -al1|head -n 10
total 1979012
drwx—— 2 mysql mysql 36864 Nov 17 12:00 .
drwx—— 36 mysql mysql 4096 Nov 17 11:12 ..
-rw-rw—- 1 mysql mysql 8712 Nov 17 10:26 1_campaigns_diez.frm
-rw-rw—- 1 mysql mysql 14672 Jul 8 18:57 1_campaigns_diez.MYD
-rw-rw—- 1 mysql mysql 1024 Nov 17 11:38 1_campaigns_diez.MYI
-rw-rw—- 1 mysql mysql 8938 Nov 17 10:26 1_campaigns.frm
-rw-rw—- 1 mysql mysql 8738 Nov 17 10:26 1_campaigns_logs.frm
-rw-rw—- 1 mysql mysql 883404 Nov 16 22:01 1_campaigns_logs.MYD
-rw-rw—- 1 mysql mysql 330752 Nov 17 11:38 1_campaigns_logs.MYI


As seen from above output, all was perfect with permissions, so it should have been something else, so I decided to try to create a random file with touch command inside /home/mysql/database_name directory:

 

touch /home/mysql/database_name/somefile-to-test-writtability.txt touch: cannot touch ‘/scr1/data/somefile-to-test-writtability.txt‘: No space left on device


Then logically I thought the /home/mysql/ mounted ext4 partition got filled, because of crashed SQL database or a bug thus, checked with disk free command df whether there is enough space on server:

server:~# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/md1 20G 7.6G 11G 42% /
udev 10M 0 10M 0% /dev
tmpfs 13G 1.3G 12G 10% /run
tmpfs 32G 0 32G 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 32G 0 32G 0% /sys/fs/cgroup
/dev/md2 256G 134G 110G 55% /home

Well that's weird? Obviously only 55% of available disk space is used and available 134G which was more than enough so I got totally puzzled why, files can't be written.

Then very logically, I thought it might be that /home directory has remounted as read only, because the SSD memory disk on server is failing and checked for errors in dmesg, i.e.:

 

server:~# dmesg|grep -i error


Also checked how exactly was partition mounted, to check whether it is (RO) read-only:

 

server:~# mount -l|grep -i /home
/dev/md2 on /home type ext4 (rw,relatime,discard,data=ordered)


Now everything become even more weirder, as obviously the disk continued to be claiming no space left on device, while in reality there was plenty of disk space.

Then after running a quick research on the internet for the no space left on device with free disk space, I've come across this great superuser.com thread which let me realize the partition run out of inodes and that's why no new file inodes could be assigned and therefore, the linux kernel is refusing to write the file on ext4 partition.

For those who haven't heard of Linux Partition Inodes here is link to Wikipedia and a quick quote:

 

In a Unix-style file system, the inode is a data structure used to represent a filesystem object, which can be one of various things including a file or a directory. Each inode stores the attributes and disk block location(s) of the filesystem object's data.[1] Filesystem object attributes may include manipulation metadata (e.g. change,[2] access, modify time), as well as owner and permission data (e.g. group-id, user-id, permissions).[3]
Directories are lists of names assigned to inodes. The directory contains an entry for itself, its parent, and each of its children.


Once I understood it is the inodes, I checked how many of them are occupied with cmd:

 

server:~# df -i /home
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/md2 17006592 17006592 0 100% /home


You see, there were 0 (zero) free file inodes on server and that was the reason for no space left on device while there was actually free disk space

To clean up (free) some inodes on partition, first thing I did is to delete all old logs which were inside /home and files I positively know not to be necessery, then to find which directories allocating most innodes used:

 

server:~# find . -xdev -type f | cut -d "/" -f 2 | sort | uniq -c | sort -n


If you're on a regular old fashined IDE Hard Drive and not SSD or you have too much files inside this command will take really long …:

Therefore a better solution might be to frist:

a) Try to find root folders with large inodes count:

for i in /home/*; do echo $i; find $i |wc -l; done
Try to find specific folders:


You should get output like:

 

/home/new_website
606692
/home/common
73
/home/pcfreak
5661
/home/hipo
33
/home/blog
13570
/home/log
123
/home/lost+found
1

b) Then once you know the directory allocating most inodes, run the command again to see the sub-directories with most files (eating) partition innodes:

 

for i in /home/webservice/*; do echo $i; find $i |wc -l; done

 

One usual large folder which could free you some nodes is the linux source headers, but in my case it was simply a lot of tiny old logs being logged on the system for few years in the past without cleaning:

After deleting the log dirs and cache folder in my case /home/new_website/{log,cache}:

server:~# rm -rf /home/new_website/log/*
server:~# rm -rf /home/new_website/cache/*

 

 

a) Then, stopping Apache webserver to check prevent Apache to use MySQl databases while running database repair and restaring MySQL:
 

server:~# /etc/init.d/apache2 stop Restarting MySQL server
..
server:~# /etc/init.d/mysql restart
..


b) And re-issuing MySQL Check / Repair / Optimize database commands:
 

 

mysqlcheck –defaults-extra-file=/etc/mysql/debian.cnf \–check –all-databases -u root -p`grep -i password /root/.my.cnf |sed -e 's#password=##g'`>> /var/log/cronwork.log

mysqlcheck –defaults-extra-file=/etc/mysql/debian.cnf –analyze –all-databases -u root -p`grep -i password /root/.my.cnf |sed -e 's#password=##g'`>> /var/log/cronwork.log

mysqlcheck –defaults-extra-file=/etc/mysql/debian.cnf \–auto-repair –optimize –all-databases -u root -p`grep -i password /root/.my.cnf |sed -e 's#password=##g'`>> /var/log/cronwork.log

mysqlcheck –defaults-extra-file=/etc/mysql/debian.cnf \–optimize –all-databases -u root -p`grep -i password /root/.my.cnf |sed -e 's#password=##g'`>> /var/log/cronwork.log

c) And finally starting the Apache Webserver again:
 

server:~# /etc/init.d/apache2 start


Some innodse got freed up:
 

server:~# df -i /home Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/md2 17006592 16797196 209396 99% /home


And hooray by God's Grace and with help of prayers of The most Holy Theotokos (Virgin) Mary, websites started again !

Howto install GeoTrust RapidSSL certificate on Debian Lenny / Squeeze / Wheezy Linux

Thursday, March 25th, 2010

I faced the task of generating official Validated SSL Certificates by in mydaily duties as a System Administrator at cadiaholding.com . Though generating self-signedSSL certificate is comparatively easy task. It was a pain in the ass setting Apache version 2.2.9-10+lenny6to correctly serve pages through https:// protocol over openssl version 0.9.8g-15+lenny6.I’ll try to go through the whole process of Generating the certificate in order to help some other Debianusers out there to face less setbacks in such a simple task as installing a Trusted SSL Certificate issued(bought) by RapidSSL. Even though this article will mostly deal with SSL certificate issued by RapidSSL,it should be not a problem to apply this methodogy with Verisign or some of the other Geotrust issuedSecure Socket Layer certificates.

In generating the Validated certficate I used enom which is a domain name,ssl certificates, email and hosting company whole-saler.
Fron emon’s website after logging in and using the web interface, there are two major things required to fill inin order to issue your Trusted SSL certificate.

1. Fill in in a form a CSR file, this is usually generated on the Linux server using the openssl.
To issue the CSR file required by Enom use the following commands:

a. First we generate an DES3 RSA encrypted key which we will use next to generate the opeensl CSR file required by ENOM.
debian:~# /usr/bin/openssl genrsa -des3 -out www.domain.com.key 2048
Enter pass phrase for www.domain.com.key:

You’ll be required to fill in a pass-phrase that will be later be required to fill in before Apache servers starts or restarts,so make sure you fill something you either remember or you keep the password stored in a file.
You have to change also the www.domain.com.key in accordance with your domain name.
Now as we already have a proper generated DES3 RSA key afterwards it’s necessery to generate the CSR file with the openssl command line frontend.
So here is how:

debian:~# /usr/bin/openssl req -new -key /home/hipo/www.domain.com.key -out /home/hipo/www.domain.com.csr

Again in the above example change all the paths and file names as you wish.
It’s necessery that the end user fill in a number of questions related to the Certificate Signing Request.
Herein I’ll list what kind of prompts will emerge after executing the above command:

Enter pass phrase for /home/hipo/www.domain.com.key:
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:
State or Province Name (full name) [Some-State]:
Locality Name (eg, city) []:
Organization Name (eg, company) [Internet Widgits Pty Ltd]:
Organizational Unit Name (eg, section) []:
Common Name (eg, YOUR name) []:
Email Address []:
Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:

Note that you’ll hav eto fill in the pass phrase previously entered during the generation of the www.domain.com.key file.
In case if you’d like to read more thoroughly on the subject of howto create a Certificate Signing Request or (CSR) as we called it on multiple times, you can read About Certificate Signing Request (CSR) Generation Instructions – Apache SSL more in depth here

2. Hopefully following the above instructions you’ll now have a file named www.domain.csrJust open the www.domain.scr and copy paste it’s content to the ENOM website CSR * webform.

3. Further on select your Webserver type on Enom’s website:In our case we have to select Apache + ApacheSSL

4. What follows next is filling in your company contact information This is also required for proper certificate generation, you have to think twice before you fill in this data, take a note this can’t be changed later on without issuing a brand SSL new certificate.

Apart from the 3 major above requirements to fill in Enom there are some few more radio buttons to use to make some selections according to your personal preferences, however I won’t take time to dig in that and I’ll leave this to you.
After all the above is fulfilled you’ll have to submit your certificate details and choose an email address to which you will receive in a minute a RapidSSL Certificate Request Confirmation

Following a link from the email, will show you some basic information about the certificate about to be generated. That’s your final chance to cancel the issued Trusted Certificated.
If you’re absolutely sure the information about to enter the certificate is correct then you’ll have to follow a link and approve the certificate.

You’ll be informed that you’ll receive your certificate either through Certifier website (e.g. Enom’s website) or via another email.
I thought it’s more probable I receive it via email but anyways I was wrong. More thank 4 hours has passed since the certificate was issued and is available via Enom’s interface but I haven’t received nothing on my mail.
Therefore my friendly advice is to check about your brand new shiny Trusted Certificate on Emom’s website. I had mine ready in about 10 minutes after the CSR was issued.

Assuming that you’ve succesfully obtained the SSL Trusted certificate from RapidSSL what follows is setting up the certificate.
Initially I tried using documentation from RapidSSL website called Installing your SSL Certificate / Web Server Certificate / Secure Server Certificate from RapidSSL.com
I tried to configure one of my Virtualhost as shown in their example inserting in my /etc/apache/sites-available/www.domain.com file, few directives within the VirtualHost something like the shown below

SSLEngine on
# domain.com.crt cointains the Trusted SSL certificate generated and obtained by you from RapidSSL
SSLCertificateFile /etc/apache2/ssl/www.domain.com.crt
# www.domain.com.key contains the file used to generate the CSR file as described earlier in this post
SSLCertificateKeyFile /etc/apache2/ssl/www.domain.com.key

It is also possible insetad of using the SSLCertificateFile and SSLCertificateKeyFile directvies directives in Apache config to use:

 

Another alternative is to use

SSLCertificateFile /etc/ssl/certs/your-domain-name.crt
SSLCertificateKeyFile /etc/ssl/certs/your-domain-name.key
SSLCACertificateFile /etc/ssl/certs/gd_bundle.crt

The key file is the one generated earlier on the server and handed to the SSL regisrar, the files:

your-domain-name.crt and gd_bundle.crt files are provided by RapidSSL or from whatever SSL registrater the SSL was purchased.

After trying the above configuration and restarting apache with:

/etc/init.d/apache2 restart

Apache failed to start, it might be helpful to somebody out there the error I had in my apache error.log:
The error.log red the following:

[warn] RSA server certificate is a CA certificate (BasicConstraints: CA == TRUE !?)

After some 30 minutes or an hour of Googling on the error I came to the conclusion that the error is caused, becauseApache is supposed to work with .PEM files instead of the classical .CRT and .KEY files asnormally approached in most of the other Unix operating systems.

It took me a bit more of reading on the internet to find out that actually the .pem files so widely adopted in Debian simply contain both the www.domain.com.key file and the www.domain.com.crt key simply pasted one after another, this I also observed from the default Apache self-signed certificate that I believe comes with debian /etc/apache2/ssl/apache.pem .
So I copied both the content of my www.domain.com.key and www.domain.com.crt and store it in one file:
/etc/apache2/ssl/www.domain.com.pem

Also the following configuration:
SSLEngine on
SSLCertificateFile /etc/apache2/ssl/www.domain.com.pem

had to go in your
/etc/apache2/sites-enabled/www.domain.com

Last thing that’s left is to restart your Apache;

/etc/init.d/apache2 restart

Apache will prompt you for your certificate password entered by you during the www.domain.com.key generation. Type your password and with a bit of luck and hopefully with God’s help you’ll be having a Trusted Certificate on your webserver.

Last step is to check if the certificate is okay accessing your domain https://www.domain.com.

Well this is the end of the article, hope you enjoy.If you do please leave your comments, any corrections are also welcomed 🙂

Install VMWare tools on Debian and Ubuntu Linux – Enable VMWare Fullscreen and copy paste between OS host and Virtual machine

Wednesday, May 28th, 2014

install-vmware-tools-on-debian-gnu-linux-and-ubuntu-howto

If you need to use Virtual Machine to run some testing on heterogenous Operating Systems and you have chosen VMWare as a Virtual Machine. You will soon notice some of Virtual Machines functionality like copy between host operating system and Virtual Machine, true fullscreen mode and most importantly Copy paste between your host operating system and VMWare is not working. I'm not too much into Virtualization these days so for me it was truely shocking that a proprietary software like VMWare, claimed to be the best and most efficient Virtual Machine nowadays is not supporting copy / paste, fullscreen and copy between host and guest OS.  For those arguing why I'm using VMWare at all as it is proprietary and there is already free software Virtual Machines like QEMU and Oracle's VirtualBox its simply because now I have the chance to install and use VMWare 9 Enterprise on my work place at HP with a free Corporate license – in other words I'm using VMWare just for the sake of educating myself and would always recommend VirtualBox for those looking for good substitute free alternative to VMWare.

Before trying out VMWare, I tried Virtualbox to emulate Linux on my HP work PC running Windows with VirtualBox I was having issues with keyboard not working (because of lack of support of USB, no full screen support and lack of copy / paste between OS-es), I've just recently understood this is not because Virtualbox is bad Virtualization solution but because I forgot to install VirtualBox Oracle VM VirtualBox Extension Pack which allows support for USB, enables copy paste and full screen support. The equivalent to Virtualbox Oracle VM VirtualBox in VMWare world is called VMWare-Tools and once the guest operating system is installed inside VMWare VM, its necessary to install vmware-tools to enable better screen resolution and copy paste.
 

In Windows Virtual Machine installation of vmware-tools is pretty straight forward you go through VMWare's menus

 

VM -> Install Vmware-tools

install-vmware-tools-on-linux-guest-host-os-debian-redhat-screenshot

follow the instructions and you're done, however as always installing VMWare-tools on Linux is little bit more complicated you need to run few commands from Linux installed inside the Virtual Machine to install vmware-tools. Here is how vmware-tools is installed on Debian / Ubuntu / Linux Mint and rest of Debian based operating systems:

  1. Install Build essentials and gcc You need to have this installed some developer tools as well as GCC compiler in order for the vmware-tools to compile a special Linux kernel module which enables extra support (integration) between the VMWare VM and the installed inside VM Linux distro

apt-get install --yes build-essential gcc
...

2. Install appropriate Linux headers corresponding to current Linux OS installed kernel

apt-get install --yes linux-headers-$(uname -r)
....

3. Mount CD (Virtual) Content to obtain the vmware-tools version for your Linux

Be sure to have first checked from VMWare menus on menus VM -> Intall Vmware Tools
This step is a little bit strange but just do it without too much questioning …


mount /dev/cdrom /mnt/
umount /media/cdrom0/
mount /media/cdrom
mount /dev/sr0 /mnt/cdrom/
mount /dev/sr0 /mnt/

 

Note that /dev/sr0, might already be mounted and sometimes it might be necessary to unmount it first (don't remember exactly if I unmounted it or not)

4. Copy and Untar VMwareTools-9.2.0-799703.tar.gz

cp -rpf /media/cdrom/VMwareTools-9.2.0-799703.tar.gz /tmp/
cd /tmp/
tar -zxvvf VMwareTools-9.2.0-799703.tar.gz
...

5. Run vmware-tools installer

cd vmware-tools-distrib/
./vmware-install.pl

You will be asked multiple questions you can safely press enter to answer with default settings to all settings, hopefully if all runs okay this will make VMWare Tools installed
 

Creating a new VMware Tools installer database using the tar4 format.
Installing VMware Tools.
In which directory do you want to install the binary files?
[/usr/bin]
What is the directory that contains the init directories (rc0.d/ to rc6.d/)?
[/etc]
What is the directory that contains the init scripts?
[/etc/init.d]
In which directory do you want to install the daemon files?
[/usr/sbin]
In which directory do you want to install the library files?
[/usr/lib/vmware-tools]
The path "/usr/lib/vmware-tools" does not exist currently. This program is
going to create it, including needed parent directories. Is this what you want?
[yes]
In which directory do you want to install the documentation files?
[/usr/share/doc/vmware-tools]
The path "/usr/share/doc/vmware-tools" does not exist currently. This program
is going to create it, including needed parent directories. Is this what you
want? [yes]
The installation of VMware Tools 9.2.0 build-799703 for Linux completed
successfully. You can decide to remove this software from your system at any
time by invoking the following command: "/usr/bin/vmware-uninstall-tools.pl".
Before running VMware Tools for the first time, you need to configure it by
invoking the following command: "/usr/bin/vmware-config-tools.pl". Do you want
this program to invoke the command for you now? [yes]
Initializing…
Making sure services for VMware Tools are stopped.
Stopping VMware Tools services in the virtual machine:
Guest operating system daemon: done
Unmounting HGFS shares: done
Guest filesystem driver: done
[EXPERIMENTAL] The VMware FileSystem Sync Driver (vmsync) is a new feature that creates backups of virtual machines. Please refer to the VMware Knowledge Base for more details on this capability. Do you wish to enable this feature?
[no]
Before you can compile modules, you need to have the following installed…
make
gcc
kernel headers of the running kernel
Searching for GCC…
Detected GCC binary at "/usr/bin/gcc-4.6".
The path "/usr/bin/gcc-4.6" appears to be a valid path to the gcc binary.
Would you like to change it? [no]
Searching for a valid kernel header path…
Detected the kernel headers at "/lib/modules/3.2.0-4-amd64/build/include".
The path "/lib/modules/3.2.0-4-amd64/build/include" appears to be a valid path
to the 3.2.0-4-amd64 kernel headers.
Would you like to change it? [no]
The vmblock enables dragging or copying files between host and guest in a
Fusion or Workstation virtual environment. Do you wish to enable this feature?
[no] yes
make: Leaving directory `/tmp/vmware-root/modules/vmblock-only'

No X install found.
Creating a new initrd boot image for the kernel.
update-initramfs: Generating /boot/initrd.img-3.2.0-4-amd64
Checking acpi hot plug done
Starting VMware Tools services in the virtual machine:
Switching to guest configuration: done
VM communication interface: done
VM communication interface socket family: done
File system sync driver: done
Guest operating system daemon: done
The configuration of VMware Tools 8.6.10 build-913593 for Linux for this
running kernel completed successfully.
You must restart your X session before any mouse or graphics changes take
effect.
You can now run VMware Tools by invoking "/usr/bin/vmware-toolbox-cmd" from the
command line or by invoking "/usr/bin/vmware-toolbox" from the command line
during an X server session.
To enable advanced X features (e.g., guest resolution fit, drag and drop, and
file and text copy/paste), you will need to do one (or more) of the following:
1. Manually start /usr/bin/vmware-user
2. Log out and log back into your desktop session; and,
3. Restart your X session.
Enjoy,
–the VMware team
Found VMware Tools CDROM mounted at /mnt. Ejecting device /dev/sr0 …

.To make sure vmware-tools compiled modules are loaded into Linux kernel inside VM, restart the Virtual Machine. Once Linux boots again and you login to gnome-terminal to check what is vmware-tools status (e.g. if properly loaded) run:

service vmware-tools status
vmtoolsd is running

install-vmware-tools-on-debian-gnu-linux-and-ubuntu-virtual-machine-screenshot

This method of installing works on Debian 7 (Wheezy) but same steps should work on any Ubuntu and rest of Debian derivatives. For Redhat (RPM) based Linux distributions to install vmware-tools after mounting cdrom drive following above instructions you will have an rpm package instead of .tar.gz archive so all you have to do is install the rpm, e.g. launch smth. like:

rpm -Uhv /mnt/cdrom/VMwareTools-9.2.0-799703.i386.rpm
Cheers 😉

Nokia Communicator 9300i connection via Bluetooth on Debian GNU/Linux Unstable

Saturday, October 24th, 2009

I wanted to connect my Nokia Communicator 9300i device via bluetooth on Linux on
my Thinkpad running Debian GNU/Linux unstable. By default my bluetooth is disabled
because I usually do not use the bluetooth.
To make the bluetooth back running I had to issue the command:
# echo "enable" > /proc/acpi/ibm/bluetooth
Next the bluetooth-wizard came to play:
# /usr/bin/bluetooth-wizard and then I had to follow the intuitive dialogs,a 4 digit code was generated by my Linux system the whole system was required to be provided
on the Nokia Communicator.
The only left thing is to use gnome’s bluetooth-applet “Blowse Files” feature.
Here is a screenshot showing up the bluetooth-applet in case if you wonder how the app looks like
Bluetooth Applet
In case if bluetooth-applet is not appearing as an applet in your gnome panel. You might need to:
# /usr/bin/gnome-control-center
– Search for the Bluetooth menu:
Make sure the “Show Bluetooth icon”‘s tick is presented
Last thing to note is that I tried using the bluez-gnome debian package
as a mean to communicate with the Phone unsuccesfully. As far as I could understand
from a couple of posts I’ve red online the Nokia communicator requires the obex
protocol to communicate data. This is not supported by bluez-gnome.
I’ve also red a couple of interesting posts which explained how to usep3nfs to mount your telephone via the unix nfs protocol
There is an old article explaining how to do the mount via p3nfs in Ubuntu which
can be red here