Posts Tagged ‘echo’

How to automate open xen Hypervisor Virtual Machines backups shell script

Tuesday, June 22nd, 2021

openxen-backup-logo As a sysadmin that have my own Open Xen Debian Hypervisor running on a Lenovo ThinkServer few months ago due to a human error I managed to mess up one of my virtual machines and rebuild the Operating System from scratch and restore files service and MySQl data from backup that really pissed me of and this brought the need for having a decent Virtual Machine OpenXen backup solution I can implement on the Debian ( Buster) 10.10 running the free community Open Xen version 4.11.4+107-gef32c7afa2-1. The Hypervisor is a relative small one holding just 7 VM s:

HypervisorHost:~#  xl list
Name                                        ID   Mem VCPUs      State   Time(s)
Domain-0                                     0 11102    24     r—–  214176.4
pcfrxenweb                                  11 12288     4     -b—-  247425.5
pcfrxen                                     12 16384    10     -b—-  1371621.4
windows7                                    20  4096     2     -b—-   97887.2
haproxy2                                    21  4096     2     -b—-   11806.9
jitsi-meet                                  22  2048     2     -b—-   12843.9
zabbix                                      23  2048     2     -b—-   20275.1
centos7                                     24  2040     2     -b—-   10898.2

HypervisorHost:~# xl list|grep -v 'Name ' |grep  -v 'Domain-0'  |wc -l
7


The backup strategy of the script is very simple to shutdown the running VM machine, make a copy with rsync to a backup location the image of each of the Virtual Machines in a bash shell loop for each virtual machine shown in output of xl command and backup to a preset local directory in my case this is /backups/ the backup of each virtual machine is produced within a separate backup directory with a respective timestamp. Backup VM .img files are produced in my case to mounted 2x external attached hard drives each of which is a 4 Terabyte Seagate Plus Backup (Storage). The original version of the script was made to be a slightly different by Zhiqiang Ma whose script I used for a template to come up with my xen VM backup solution. To prevent the Hypervisor's load the script is made to do it with a nice of (nice -n 10) this might be not required or you might want to modify it to better suit your needs. Below is the script itself you can fetch a copy of it /usr/sbin/xen_vm_backups.sh :

#!/bin/bash

# Author: Zhiqiang Ma (http://www.ericzma.com/)
# Modified to work with xl and OpenXen by Georgi Georgiev – https://pc-freak.net
# Original creation dateDec. 27, 2010
# Script takes all defined vms under xen_name_list and prepares backup of each
# after shutting down the machine prepares archive and copies archive in externally attached mounted /backup/disk1 HDD
# Latest update: 08.06.2021 G. Georgiev – hipo@pc-freak.net

mark_file=/backups/disk1/tmp/xen-bak-marker
log_file=/var/log/xen/backups/bak-$(date +%Y_%m_%d).log
err_log_file=/var/log/xen/backups/bak_err-$(date +%H_%M_%Y_%m_%d).log
xen_dir=/xen/domains
xen_vmconfig_dir=/etc/xen/
local_bak_dir=/backups/disk1/tmp
#bak_dir=xenbak@backup_host1:/lhome/xenbak
bak_dir=/backups/disk1/xen-backups/xen_images/$(date +%Y_%m_%d)/xen/domains
#xen_name_list="haproxy2 pcfrxenweb jitsi-meet zabbix windows7 centos7 pcfrxenweb pcfrxen"
xen_name_list="windows7 haproxy2 jitsi-meet zabbix centos7"

if [ ! -d /var/log/xen/backups ]; then
echo mkdir -p /var/log/xen/backups
 mkdir -p /var/log/xen/backups
fi

if [ ! -d $bak_dir ]; then
echo mkdir -p $bak_dir
 mkdir -p $bak_dir

fi


# check whether bak runned last week
if [ -e $mark_file ] ; then
        echo  rm -f $mark_file
 rm -f $mark_file
else
        echo  touch $mark_file
 touch $mark_file
  # exit 0
fi

# set std and stderr to log file
        echo mv $log_file $log_file.old
       mv $log_file $log_file.old
        echo mv $err_log_file $err_log_file.old
       mv $err_log_file $err_log_file.old
        echo "exec 2> $err_log_file"
       exec 2> $err_log_file
        echo "exec > $log_file"
       exec > $log_file


# check whether the VM is running
# We only backup running VMs

echo "*** Check alive VMs"

xen_name_list_tmp=""

for i in $xen_name_list
do
        echo "/usr/sbin/xl list > /tmp/tmp-xen-list"
        /usr/sbin/xl list > /tmp/tmp-xen-list
  grepinlist=`grep $i" " /tmp/tmp-xen-list`;
  if [[ “$grepinlist” == “” ]]
  then
    echo $i is not alive.
  else
    echo $i is alive.
    xen_name_list_tmp=$xen_name_list_tmp" "$i
  fi
done

xen_name_list=$xen_name_list_tmp

echo "Alive VM list:"

for i in $xen_name_list
do
   echo $i
done

echo "End alive VM list."

###############################
date
echo "*** Backup starts"

###############################
date
echo "*** Copy VMs to local disk"

for i in $xen_name_list
do
  date
  echo "Shutdown $i"
        echo  /usr/sbin/xl shutdown $i
        /usr/sbin/xl shutdown $i
        if [ $? != ‘0’ ]; then
                echo 'Not Xen Disk image destroying …';
                /usr/sbin/xl destroy $i
        fi
  sleep 30

  echo "Copy $i"
  echo "Copy to local_bak_dir: $local_bak_dir"
      echo /usr/bin/rsync -avhW –no-compress –progress $xen_dir/$i/{disk.img,swap.img} $local_bak_dir/$i/
     time /usr/bin/rsync -avhW –no-compress –progress $xen_dir/$i/{disk.img,swap.img} $local_bak_dir/$i/
      echo /usr/bin/rsync -avhW –no-compress –progress $xen_vmconfig_dir/$i.cfg $local_bak_dir/$i.cfg
     time /usr/bin/rsync -avhW –no-compress –progress $xen_vmconfig_dir/$i.cfg $local_bak_dir/$i.cfg
  date
  echo "Create $i"
  # with vmmem=1024"
  # /usr/sbin/xm create $xen_dir/vm.run vmid=$i vmmem=1024
          echo /usr/sbin/xl create $xen_vmconfig_dir/$i.cfg
          /usr/sbin/xl create $xen_vmconfig_dir/$i.cfg
## Uncomment if you need to copy with scp somewhere
###       echo scp $log_file $bak_dir/xen-bak-111.log
###      echo  /usr/bin/rsync -avhW –no-compress –progress $log_file $local_bak_dir/xen-bak-111.log
done

####################
date
echo "*** Compress local bak vmdisks"

for i in $xen_name_list
do
  date
  echo "Compress $i"
      echo tar -z -cfv $bak_dir/$i-$(date +%Y_%m_%d).tar.gz $local_bak_dir/$i-$(date +%Y_%m_%d) $local_bak_dir/$i.cfg
     time nice -n 10 tar -z -cvf $bak_dir/$i-$(date +%Y_%m_%d).tar.gz $local_bak_dir/$i/ $local_bak_dir/$i.cfg
    echo rm -vf $local_bak_dir/$i/ $local_bak_dir/$i.cfg
    rm -vrf $local_bak_dir/$i/{disk.img,swap.img}  $local_bak_dir/$i.cfg
done

####################
date
echo "*** Copy local bak vmdisks to remote machines"

copy_remote () {
for i in $xen_name_list
do
  date
  echo "Copy to remote: vm$i"
        echo  scp $local_bak_dir/vmdisk0-$i.tar.gz $bak_dir/vmdisk0-$i.tar.gz
done

#####################
date
echo "Backup finishes"
        echo scp $log_file $bak_dir/bak-111.log

}

date
echo "Backup finished"

 

Things to configure before start using using the script to prepare backups for you is the xen_name_list variable

#  directory skele where to store already prepared backups
bak_dir=/backups/disk1/xen-backups/xen_images/$(date +%Y_%m_%d)/xen/domains

# The configurations of the running Xen Virtual Machines
xen_vmconfig_dir=/etc/xen/
# a local directory that will be used for backup creation ( I prefer this directory to be on the backup storage location )
local_bak_dir=/backups/disk1/tmp
#bak_dir=xenbak@backup_host1:/lhome/xenbak
# the structure on the backup location where daily .img backups with be produced with rsync and tar archived with bzip2
bak_dir=/backups/disk1/xen-backups/xen_images/$(date +%Y_%m_%d)/xen/domains

# list here all the Virtual Machines you want the script to create backups of
xen_name_list="windows7 haproxy2 jitsi-meet zabbix centos7"

If you need the script to copy the backup of Virtual Machine images to external Backup server via Local Lan or to a remote Internet located encrypted connection with a passwordless ssh authentication (once you have prepared the Machines to automatically login without pass over ssh with specific user), you can uncomment the script commented section to adapt it to copy to remote host.

Once you have placed at place /usr/sbin/xen_vm_backups.sh use a cronjob to prepare backups on a regular basis, for example I use the following cron to produce a working copy of the Virtual Machine backups everyday.
 

# crontab -u root -l 

# create windows7 haproxy2 jitsi-meet centos7 zabbix VMs backup once a month
00 06 1,2,3,4,5,6,7,8,9,10,11,12 * * /usr/sbin/xen_vm_backups.sh 2>&1 >/dev/null


I do clean up virtual machines Images that are older than 95 days with another cron job

# crontab -u root -l

# Delete xen image files older than 95 days to clear up space from backup HDD
45 06 17 * * find /backups/disk1/xen-backups/xen_images* -type f -mtime +95 -exec rm {} \; 2>&1 >/dev/null

#### Delete xen config backups older than 1 year+3 days (368 days)
45 06 17 * * find /backups/disk1/xen-backups/xen_config* -type f -mtime +368 -exec rm {} \; 2>&1 >/dev/null

 

# Delete xen image files older than 95 days to clear up space from backup HDD
45 06 17 * * find /backups/disk1/xen-backups/xen_images* -type f -mtime +95 -exec rm {} \; 2>&1 >/dev/null

#### Delete xen config backups older than 1 year+3 days (368 days)
45 06 17 * * find /backups/disk1/xen-backups/xen_config* -type f -mtime +368 -exec rm {} \; 2>&1 >/dev/null

Howto Configure Linux shell Prompt / Setup custom Terminal show Prompt using default shell variables PS1, PS2, PS3, PS4

Tuesday, August 27th, 2019

how-to-configure-lunux-bsd-shell-prompt-ps1-howto-make-your-terminal-console-shell-nice-and-shiny-1

System Console, Command Operation Console  or Terminal is a Physical device for text (command) input from keyboard, getting the command output and monitoring the status of a shell or programs I/O operations generated traditionally with attached screen. With the development of Computers, physical consoles has become emulated and the input output is translated on the monitor usually via a data transfer  protocol historically mostly over TCP/IP connection to remote IP with telnet or rsh, but due to security limitations Consoles are now accessed over data (encrypted) network protocols with SHA2 / MD5 cryptography algorithm enabled such as over SSH (Secure Shell) network protocol..
The ancestors of physical consoles which in the past were just a Terminal (Monitoring / Monitor device attached to a MainFrame system computer).

Mainframe-physical-terminal-monitor-Old-Computer

What is Physical Console
A classical TTY (TeleTYpewriter) device looked like so and served the purpose of being just a communication and display deivce, whether in reality the actual computing and storage tape devices were in a separate room and communicating to Terminal.

mainframe-super-computer-computing-tape-machine
TTYs are still present in  modern UNIX like GNU / Linux distrubions OSes and the BSD berkley 4.4 code based FreeBSD / NetBSD / OpenBSD if you have installed the OS on a physical computer in FreeBSD and Solaris / SunOS there is also tty command. TTY utility in *nix writes the name of the terminal attached to standard input to standard output, in Linux there is a GNU remake of same program part called GNU tty of coreutils package (try man tty) for more.

The physical console is recognizable in Linux as it is indicated with other tree letters pts – (pseudo terminal device) standing for a terminal device which is emulated by an other program (example: xterm, screen, or ssh are such programs). A pts is the slave part of a pts is pseudo there is no separate binary program for it but it is dynamically allocated in memory.
PTS is also called Line consle in Cisco Switches / Router devices, VTY is the physical Serial Console connected on your Cisco device and the network connection emulation to network device is creates with a virtual console session VTL (Virtual Terminal Line). In freebsd the actual /dev/pts* /dev/tty* temporary devices on the OS are slightly different and have naming such as /dev/ttys001.
But the existence of tty and pts emulator is not enough for communicating interrupts to Kernel and UserLand binaries of the Linux / BSD OS, thus to send the commands on top of it is running a System Shell as CSH / TSH / TCSH or BASH which is usually the first program set to run after user logs in over ptty or pseudo tty virtual terminal.

linux-tty-terminal-explained-brief-intro-to-linux-device-drivers-20-638

 

Setting the Bash Prompt in Terminal / Console on GNU / Linux

Bash has system environments to control multiple of variables, which are usually visible with env command, one important variable to change in the past was for example USER / USERNAME which was red by IRC Chat clients  such as BitchX / irssi and could be displayed publicly so if not changed to a separate value, one could have known your Linux login username by simple /whois query to the Nickname in question (if no inetd / xinetd service was running on the Linux box and usually inetd was not running).

Below is my custom set USER / USERNAME to separate

hipo@pcfreak:~$ env|grep USER
USERNAME=Attitude
USER=Attitude

There is plenty of variables to  tune email such as MAIL store directory, terminal used TERM, EDITOR etc. but there are some
variables that are not visible with env query as they're not globally available for all users but just for the single user, to show this ones you need to use declare command instead, to get a full list of All Single and System Wide defined variables and functions type declare in the bash shell, for readability, below is last 10 returned results:

 

hipo@pcfreak:~$ declare | tail -10
{
    local quoted=${1//\'/\'\\\'\'};
    printf "'%s'" "$quoted"
}
quote_readline ()
{
    local quoted;
    _quote_readline_by_ref "$1" ret;
    printf %s "$ret"
}

 

PS1 is present there virtually on any modern Linux distribution and is installed through user home's directory $HOME/.bashrc , ~/.profile or .bash_profile or System Wide globally for all existing users in /etc/passwd (password database file) from /etc/bash.bashrc
In Debian / Ubuntu / Mint GNU / Linux this system variable is set in user home's .bashrc but in Fedora / RHEL Linux distro,
PS1 is configured from /home/username/.bash_profile to find out where PS1 is located for ur user:

cd ~
grep -Rli PS1 .bash*

Here is one more example:

hipo@pcfreak:~$ declare|grep -i PS1|head -1
PS1='\[\e]0;\u@\h: \w\a\]${debian_chroot:+($debian_chroot)}\u@\h:\w\$ '
 

hipo@pcfreak:~$ grep PS1 /etc/bash.bashrc
[ -z “$PS1” ] && return
# but only if not SUDOing and have SUDO_PS1 set; then assume smart user.
if ! [ -n “${SUDO_USER}” -a -n “${SUDO_PS1}” ]; then
  PS1='${debian_chroot:+($debian_chroot)}\u@\h:\w\$ '


Getting current logged in user shell configured PS1 variable can be done with echo:

hipo@pcfreak:~$ echo $PS1
\[\e]0;\u@\h: \w\a\]${debian_chroot:+($debian_chroot)}\u@\h:\w\$

So lets observe a little bit the meaning of this obscure line of (code) instructions code which are understood by BASH when being red from PS1 var to do so, I'll give a list of meaning of main understood commands, each of which is defined with \.

The ${debian_chroot} shell variable is defined from /etc/bash.bashrc

Easiest way to change PS1 is to export the string you like with the arguments like so:

 

root@linux:/home/hipo# export PS1='My-Custom_Server-Name# '
My-Custom_Server-Name# echo $PS1
My-Custom_Server-Name#

 

  •     \a : an ASCII bell character (07)
  •     \d : the date in “Weekday Month Date” format (e.g., “Tue May 26”)
  •     \D{format} : the format is passed to strftime(3) and the result is inserted into the prompt string; an empty format results in a locale-specific time representation. The braces are required
  •     \e : an ASCII escape character (033)
  •     \h : the hostname up to the first ‘.’
  •     \H : the hostname
  •     \j : the number of jobs currently managed by the shell
  •     \l : the basename of the shell's terminal device name
  •     \n : newline
  •     \r : carriage return
  •     \s : the name of the shell, the basename of $0 (the portion following the final slash)
  •     \t : the current time in 24-hour HH:MM:SS format
  •     \T : the current time in 12-hour HH:MM:SS format
  •     \@ : the current time in 12-hour am/pm format
  •     \A : the current time in 24-hour HH:MM format
  •     \u : the username of the current user
  •     \v : the version of bash (e.g., 2.00)
  •     \V : the release of bash, version + patch level (e.g., 2.00.0)
  •     \w : the current working directory, with $HOME abbreviated with a tilde
  •     \W : the basename of the current working directory, with $HOME abbreviated with a tilde
  •     \! : the history number of this command
  •     \# : the command number of this command
  •     \$ : if the effective UID is 0, a #, otherwise a $
  •     \nnn : the character corresponding to the octal number nnn
  •     \\ : a backslash
  •     \[ : begin a sequence of non-printing characters, which could be used to embed a terminal control sequence into the prompt
  •     \] : end a sequence of non-printing characters

The default's PS1 set prompt on Debian Linux is:
 

echo $PS1
\[\e]0;\u@\h: \w\a\]${debian_chroot:+($debian_chroot)}\u@\h:\w\$


As you can see \u (print username) \h (print hostname)  and \W (basename of current working dir) or \w (print $HOME/current working dir)
are the most essential, the rest are bell character, escape character etc.

A very good way to make your life easier and learn the abbreviations / generate exactly the PS1 PROMPT you want to have is with Easy Bash PS1 Generator Web Utility
with which you can just click over buttons that are capable to produce all of the PS1 codes.
 

1. How to show current hour:minute:seconds / print full date in Prompt Shell (PS)


Here is an example with setting the Bash Shell prompt  to include also the current time in format hour:minute:seconds (very useful if you're executing commands on a critical servers and you run commands in some kind of virtual terminal like screen or tmux.
 

root@pcfreak:~# PS1="\n\t \u@\h:\w# "
14:03:51 root@pcfreak:/home#


PS1-how-to-setup-date-time-hour-minutes-and-seconds-in-bash-shell-prompt
 

 

export PS1='\u@\H \D{%Y-%m-%d %H:%M;%S%z}] \W ] \$ '

 


export-PS1-Linux-set-full-date-time-clock-prompt-screenshot-console


Make superuser appear in RED color (adding PS1 prompt custom color for a User)
 

root@pcfreak:~$  PS1="\\[$(tput setaf 1)\\]\\u@\\h:\\w #\\[$(tput sgr0)\\]"

 

how-to-change-colors-in-bash-prompt-shell-on-linux-shell-environment

In above example the Shell Prompt Color changed is changed for administrator (root) to shebang symbol # in red, green, yellow and blue for the sake to show you how it is done, however this example can be adapted for any user on the system. Setting different coloring for users is very handy if you have to administer Mail Server service like Qmail or other Application that consists of multiple small ones of multiple daemons such as qmail + vpopmail + clamd + mysql etc. Under such circumstances, coloring each of the users in different color like in the example for debugging is very useful.

Coloring the PS1 system prompt on Linux to different color has been a standard practice in Linux Server environments running Redhat Enterprise Linux (RHEL) and SuSE Enterprise Linux and some Desktop distributions such as Mint Linux.

To make The Root prompt Red colored only for system super user (root) on any Linux distribution
, add the following to /etc/bashrc, e.g.

vim /etc/bashrc
 


# If id command returns zero, you've root access.
if [ $(id -u) -eq 0 ];
then # you are root, set red colour prompt
  PS1="\\[$(tput setaf 1)\\]\\u@\\h:\\w #\\[$(tput sgr0)\\]"
else # normal
  PS1="[\\u@\\h:\\w] $"
fi

 

 

2. How to make the prompt of a System user appear Green


Add to ~/.bashrc  following line

 

 

PS1="\\[$(tput setaf 2)\\]\\u@\\h:\\w #\\[$(tput sgr0)\\]"
 

 

3. Print New line, username@hostname, base PTY, shell level, history (number), newline and full working directory $PWD

 

export PS1='\n[\u@\h \l:$SHLVL:\!]\n$PWD\$ '

 

4. Showing the numbert of jobs the shell is currently managing.


This is useful if you run and switch with fg / bg (foreground / background) commands
to switch between jobs and forget some old job.

 

export PS1='\u@\H \D{%Y-%m-%d %H:%M;%S%z}] \W \$]'

 

Multi Lines Prompt / Make very colorful Shell prompt full of stats info

PS1="\n\[\033[35m\]\$(/bin/date)\n\[\033[32m\]\w\n\[\033[1;31m\]\u@\h: \[\033[1;34m\]\$(/usr/bin/tty | /bin/sed -e ‘s:/dev/::’): \[\033[1;36m\]\$(/bin/ls -1 | /usr/bin/wc -l | /bin/sed ‘s: ::g’) files \[\033[1;33m\]\$(/bin/ls -lah | /bin/grep -m 1 total | /bin/sed ‘s/total //’)b\[\033[0m\] -> \[\033[0m\]"

 

 

prompt-show-how-many-files-and-virtual-pts-ps1-linux
 

5. Set color change on command failure


If you have a broken command or the command ended with non zero output with some kind of bad nasty message and you want to make, that more appearing making it red heighlighted, here is how:

 

PROMPT_COMMAND='PS1="\[\033[0;33m\][\!]\`if [[ \$? = “0” ]]; then echo “\\[\\033[32m\\]”; else echo “\\[\\033[31m\\]”; fi\`[\u.\h: \`if [[ `pwd|wc -c|tr -d ” “` > 18 ]]; then echo “\\W”; else echo “\\w”; fi\`]\$\[\033[0m\] “; echo -ne “\033]0;`hostname -s`:`pwd`\007"'

 

6. Other beautiful PS1 Color Prompts with statistics

 

PS1="\n\[\e[32;1m\](\[\e[37;1m\]\u\[\e[32;1m\])-(\[\e[37;1m\]jobs:\j\[\e[32;1m\])-(\[\e[37;1m\]\w\[\e[32;1m\])\n(\[\[\e[37;1m\]! \!\[\e[32;1m\])-> \[\e[0m\]"

 

 

another-very-beuatiful-bash-colorful-prompt

 

7. Add Muliple Colors to Same Shell prompt

 

function prompt { local BLUE="\[\033[0;34m\]” local DARK_BLUE=”\[\033[1;34m\]” local RED=”\[\033[0;31m\]” local DARK_RED=”\[\033[1;31m\]” local NO_COLOR=”\[\033[0m\]” case $TERM in xterm*|rxvt*) TITLEBAR=’\[\033]0;\u@\h:\w\007\]’ ;; *) TITLEBAR=”” ;; esac PS1=”\u@\h [\t]> ” PS1=”${TITLEBAR}\ $BLUE\u@\h $RED[\t]>$NO_COLOR " PS2='continue-> ' PS4='$0.$LINENO+ ' }

colorful-prompt-blue-and-red-linux-console-PS1
 

8. Setting / Change Shell background Color


changing-background-color-of-bash-shell-prompt-linux

 

export PS1="\[$(tput bold)$(tput setb 4)$(tput setaf 7)\]\u@\h:\w $ \[$(tput sgr0)\]"

 

tput Color Capabilities:

  • tput setab [1-7] – Set a background color using ANSI escape
  • tput setb [1-7] – Set a background color
  • tput setaf [1-7] – Set a foreground color using ANSI escape
  • tput setf [1-7] – Set a foreground color

tput Text Mode Capabilities:

  • tput bold – Set bold mode
  • tput dim – turn on half-bright mode
  • tput smul – begin underline mode
  • tput rmul – exit underline mode
  • tput rev – Turn on reverse mode
  • tput smso – Enter standout mode (bold on rxvt)
  • tput rmso – Exit standout mode
  • tput sgr0 – Turn off all attributes

Color Code for tput:

  • 0 – Black
  • 1 – Red
  • 2 – Green
  • 3 – Yellow
  • 4 – Blue
  • 5 – Magenta
  • 6 – Cyan
  • 7 – White

 

9. Howto Use bash shell function inside PS1 variable

If you administrate Apache or other HTTPD servers or any other server whose processes are forked and do raise drastically at times to keep an eye while actively working on the server.

 

function httpdcount { ps aux | grep apache2 | grep -v grep | wc -l } export PS1="\u@\h [`httpdcount`]> "

10. PS2, PS3, PS4 little known variables
 

I'll not get much into detail to PS2, PS3, PS4 but will mention them as perhaps many people are not even aware they exist.
They're rarely used in the daily system administrator's work but useful for Shell scripting purposes of Dev Ops and Shell Scripting Guru Programmers.

  • PS2 – Continuation interactive prompt

A very long unix command can be broken down to multiple line by giving \ at the end of the line. The default interactive prompt for a multi-line command is “> “.  Let us change this default behavior to display “continue->” by using PS2 environment variable as shown below.

hipo@db-host :~$ myisamchk –silent –force –fast –update-state \
> –key_buffer_size=512M –sort_buffer_size=512M \
> –read_buffer_size=4M –write_buffer_size=4M \
> /var/lib/mysql/bugs/*.MYI
[Note: This uses the default “>” for continuation prompt]

  • PS3 – Prompt used by “select” inside shell script (usefulif you write scripts with user prompts)

     

  • PS4 – Used by “set -x” to prefix tracing output
    The PS4 shell variable defines the prompt that gets displayed.

You can find  example with script demonstrating PS2, PS3, PS4 use via small shell scripts in thegeekstuff's article Take control of PS1, PS2, PS3, PS4 read it here

 

Summary


In this article, I've shortly reviewed on what is a TTY, how it evolved into Pseudo TTY and how it relates to current shells which are the interface communicating with the modern UNIX like Operating systems's userland and kernel.
Also it was reviewed shortly how the current definitions of shell variables could be viewed with declare cmd. Also I went through on how to display the PS1 variable and  on how to modify PS1 and make the prompt different statistics and monitoring parameters straight into the command shell. I've shown some common PS1 strings that report on current date hour, minute, seconds, modify the coloring of the bash prompt shell, show processes count, and some PS1 examples were given that combines beuatiful shell coloring as well as how the Prompt background color can be changed.
Finally was shown how a combination of commands can be executed by exporting to PS1 to update process counf of Apache on every shell prompt iteration.
Other shell goodies are mostly welcome

 

 

How to build Linux logging bash shell script write_log, logging with Named Pipe buffer, Simple Linux common log files logging with logger command

Monday, August 26th, 2019

how-to-build-bash-script-for-logging-buffer-named-pipes-basic-common-files-logging-with-logger-command

Logging into file in GNU / Linux and FreeBSD is as simple as simply redirecting the output, e.g.:
 

echo "$(date) Whatever" >> /home/hipo/log/output_file_log.txt


or with pyping to tee command

 

echo "$(date) Service has Crashed" | tee -a /home/hipo/log/output_file_log.txt


But what if you need to create a full featured logging bash robust shell script function that will run as a daemon continusly as a background process and will output
all content from itself to an external log file?
In below article, I've given example logging script in bash, as well as small example on how a specially crafted Named Pipe buffer can be used that will later store to a file of choice.
Finally I found it interesting to mention few words about logger command which can be used to log anything to many of the common / general Linux log files stored under /var/log/ – i.e. /var/log/syslog /var/log/user /var/log/daemon /var/log/mail etc.
 

1. Bash script function for logging write_log();


Perhaps the simplest method is just to use a small function routine in your shell script like this:
 

write_log()
LOG_FILE='/root/log.txt';
{
  while read text
  do
      LOGTIME=`date "+%Y-%m-%d %H:%M:%S"`
      # If log file is not defined, just echo the output
      if [ “$LOG_FILE” == “” ]; then
    echo $LOGTIME": $text";
      else
        LOG=$LOG_FILE.`date +%Y%m%d`
    touch $LOG
        if [ ! -f $LOG ]; then echo "ERROR!! Cannot create log file $LOG. Exiting."; exit 1; fi
    echo $LOGTIME": $text" | tee -a $LOG;
      fi
  done
}

 

  •  Using the script from within itself or from external to write out to defined log file

 

echo "Skipping to next copy" | write_log

 

2. Use Unix named pipes to pass data – Small intro on what is Unix Named Pipe.


Named Pipe –  a named pipe (also known as a FIFO (First In First Out) for its behavior) is an extension to the traditional pipe concept on Unix and Unix-like systems, and is one of the methods of inter-process communication (IPC). The concept is also found in OS/2 and Microsoft Windows, although the semantics differ substantially. A traditional pipe is "unnamed" and lasts only as long as the process. A named pipe, however, can last as long as the system is up, beyond the life of the process. It can be deleted if no longer used.
Usually a named pipe appears as a file, and generally processes attach to it for IPC.

 

Once named pipes were shortly explained for those who hear it for a first time, its time to say named pipe in unix / linux is created with mkfifo command, syntax is straight foward:
 

mkfifo /tmp/name-of-named-pipe


Some older Linux-es with older bash and older bash shell scripts were using mknod.
So idea behind logging script is to use a simple named pipe read input and use date command to log the exact time the command was executed, here is the script.

 

#!/bin/bash
named_pipe='/tmp/output-named-pipe';
output_named_log='
/tmp/output-named-log.txt ';

if [ -p $named_pipe ]; then
rm -f $named_pipe
fi
mkfifo $named_pipe

while true; do
read LINE <$named_pipe
echo $(date): "$LINE" >>/tmp/output-named-log.txt
done


To write out any other script output and get logged now, any of your output with a nice current date command generated output write out any output content to the loggin buffer like so:

 

echo 'Using Named pipes is so cool' > /tmp/output-named-pipe
echo 'Disk is full on a trigger' > /tmp/output-named-pipe

  • Getting the output with the date timestamp

# cat /tmp/output-named-log.txt
Mon Aug 26 15:21:29 EEST 2019: Using Named pipes is so cool
Mon Aug 26 15:21:54 EEST 2019: Disk is full on a trigger


If you wonder why it is better to use Named pipes for logging, they perform better (are generally quicker) than Unix sockets.

 

3. Logging files to system log files with logger

 

If you need to do a one time quick way to log any message of your choice with a standard Logging timestamp, take a look at logger (a part of bsdutils Linux package), and is a command which is used to enter messages into the system log, to use it simply invoke it with a message and it will log your specified output by default to /var/log/syslog common logfile

 

root@linux:/root# logger 'Here we go, logging'
root@linux:/root # tail -n 3 /var/log/syslog
Aug 26 15:41:01 localhost CRON[24490]: (root) CMD (chown qscand:qscand -R /var/run/clamav/ 2>&1 >/dev/null)
Aug 26 15:42:01 localhost CRON[24547]: (root) CMD (chown qscand:qscand -R /var/run/clamav/ 2>&1 >/dev/null)
Aug 26 15:42:20 localhost hipo: Here we go, logging

 

If you have took some time to read any of the init.d scripts on Debian / Fedora / RHEL / CentOS Linux etc. you will notice the logger logging facility is heavily used.

With logger you can print out message with different priorities (e.g. if you want to write an error message to mail.* logs), you can do so with:
 

 logger -i -p mail.err "Output of mail processing script"


To log a normal non-error (priority message) with logger to /var/log/mail.log system log.

 

 logger -i -p mail.notice "Output of mail processing script"


A whole list of supported facility named priority valid levels by logger (as taken of its current Linux manual) are as so:

 

FACILITIES AND LEVELS
       Valid facility names are:

              auth
              authpriv   for security information of a sensitive nature
              cron
              daemon
              ftp
              kern       cannot be generated from userspace process, automatically converted to user
              lpr
              mail
              news
              syslog
              user
              uucp
              local0
                to
              local7
              security   deprecated synonym for auth

       Valid level names are:

              emerg
              alert
              crit
              err
              warning
              notice
              info
              debug
              panic     deprecated synonym for emerg
              error     deprecated synonym for err
              warn      deprecated synonym for warning

       For the priority order and intended purposes of these facilities and levels, see syslog(3).

 


If you just want to log to Linux main log file (be it /var/log/syslog or /var/log/messages), depending on the Linux distribution, just type', even without any shell quoting:

 

logger 'The reason to reboot the server Currently was a System security Update

 

So what others is logger useful for?

 In addition to being a good diagnostic tool, you can use logger to test if all basic system logs with its respective priorities work as expected, this is especially
useful as I've seen on a Cloud Holsted OpenXEN based servers as a SAP consultant, that sometimes logging to basic log files stops to log for months or even years due to
syslog and syslog-ng problems hungs by other thirt party scripts and programs.
To test test all basic logging and priority on system logs as expected use the following logger-test-all-basic-log-logging-facilities.sh shell script.

 

#!/bin/bash
for i in {auth,auth-priv,cron,daemon,kern, \
lpr,mail,mark,news,syslog,user,uucp,local0 \
,local1,local2,local3,local4,local5,local6,local7}

do        
# (this is all one line!)

 

for k in {debug,info,notice,warning,err,crit,alert,emerg}
do

logger -p $i.$k "Test daemon message, facility $i priority $k"

done

done

Note that on different Linux distribution verions, the facility and priority names might differ so, if you get

logger: unknown facility name: {auth,auth-priv,cron,daemon,kern,lpr,mail,mark,news, \
syslog,user,uucp,local0,local1,local2,local3,local4, \
local5,local6,local7}

check and set the proper naming as described in logger man page.

 

4. Using a file descriptor that will output to a pre-set log file


Another way is to add the following code to the beginning of the script

#!/bin/bash
exec 3>&1 4>&2
trap 'exec 2>&4 1>&3' 0 1 2 3
exec 1>log.out 2>&1
# Everything below will go to the file 'log.out':

The code Explaned

  •     Saves file descriptors so they can be restored to whatever they were before redirection or used themselves to output to whatever they were before the following redirect.
    trap 'exec 2>&4 1>&3' 0 1 2 3
  •     Restore file descriptors for particular signals. Not generally necessary since they should be restored when the sub-shell exits.

          exec 1>log.out 2>&1

  •     Redirect stdout to file log.out then redirect stderr to stdout. Note that the order is important when you want them going to the same file. stdout must be redirected before stderr is redirected to stdout.

From then on, to see output on the console (maybe), you can simply redirect to &3. For example
,

echo "$(date) : Do print whatever you want logging to &3 file handler" >&3


I've initially found out about this very nice bash code from serverfault.com's post how can I fully log all bash script actions (but unfortunately on latest Debian 10 Buster Linux  that is prebundled with bash shell 5.0.3(1)-release the code doesn't behave exactly, well but still on older bash versions it works fine.

Sum it up


To shortlysummarize there is plenty of ways to do logging from a shell script logger command but using a function or a named pipe is the most classic. Sometimes if a script is supposed to write user or other script output to a a common file such as syslog, logger command can be used as it is present across most modern Linux distros.
If you have a better ways, please drop a common and I'll add it to this article.

 

OSCommerce how to change / reset lost admin password

Monday, October 16th, 2017

reset-forgotten-lost-oscommerce-password-howto-Os_commerce-logo.svg

How to change / reset OSCommerce lost / forgotten admin password?

The password in OSCommerce is kept in table "admin", so to reset password connect to MySQL with mysql cli client.

First thing to do is to generate the new hash string, you can do that with a simple php script using the md5(); function

 

root@pcfreak:/var/www/files# cat 1.php
<?
$pass=md5('password');
echo $pass;
?>

 

root@pcfreak:/var/www/files# php 1.php
5f4dcc3b5aa765d61d8327deb882cf99
root@pcfreak:/var/www/files#

 

Our just generated string (for text password password) is hash: 5f4dcc3b5aa765d61d8327deb882cf99

Next to update the new hash string into SQL, we connect to MySQL:

 

$ mysql -u root -p

 


And issue following command to modify the encrypted hash string:

 

UPDATE `DB`.`admin` SET `admin_password` = '5f4dcc3b5aa765d61d8327deb882cf99' WHERE `admin`.`admin_id` = 6;

Where is Firefox plugin / bookmarks / temp files directory on Windows?

Saturday, February 13th, 2016

windows-appdata-mozilla-plugins-how-to-check-the-extensions-folder-firefox-windows-screenshot

If you want to find out where Firefox downloads and keeps installed extensions in a quick manner, just press together:
KBD Windows flag button + R

This shortcut will open WIndows Run prompt
And paste inside the run prompt

%appdata%\Mozilla\plugins

 

The %appdata% is Windows internal variable that keeps inside  path to C:\Users\Your-Username\AppData\Roaming

On my workPC this contains:

C:\Users\georgi7>echo %appdata%
C:\Users\georgi7\AppData\Roaming

 

mozilla-plugins-folder-in-windows-explorer-screenshot

Enjoy 🙂

Check Windows load avarage command – Get CPU usage from Windows XP / 7 / 8 / 2012 server cmd prompt

Wednesday, August 19th, 2015

Check_Windows-load-avarage-command-Get_CPU_usage_from_Windows_XP-7-8-2003-2010-2012_server_cmd_prompt

If you used to be a long years Linux / UNIX sysadmin and you suddenly have to also admistrate a bunch of Windows hosts via RDP (Remote Desktop Protocol)  / Teamviewer etc. and you need to document The Load Avarage of a Windows XP / 7 / 8 servers but you're puzzled how to get an overall load avarage of Windows host via command in a UNIX way like with the good old uptime  Linux / BSD command e.g.

 ruth:$ uptime
 11:43  up 713 days 22:44,  1 user,  load average: 0.22, 0.17, 0.15

Then its time to you to get used to WMICWMIC extends WMI for operation from several command-line interfaces and through batch scripts. wmic is a wonderful command for Command addicted Linux guys and gives a lot of opportunities to query and conduct various sysadmin tasks from Windows command prompt.

To get an loadavarage with wmic use:
 

C:\>wmic cpu get loadpercentage
LoadPercentage
1

 


or
 

@for /f "skip=1" %p in ('wmic cpu get loadpercentage') do @echo %p%
1%
%

 

on Windows 7 / 8 and 10 as well Windows Server 2010 and Windows Server 2012 for more precise CPU loadavarage results, you can also use:
 

C:\> typeperf "\processor(_total)\% processor time"

"(PDH-CSV 4.0)","\\Win-Host\processor(_total)\% processor time"
"08/19/2015 12:52:53.343","0.002288"
"08/19/2015 12:52:54.357","0.000000"
"08/19/2015 12:52:55.371","0.000000"
"08/19/2015 12:52:56.385","0.000000"
"08/19/2015 12:52:57.399","0.000799"
"08/19/2015 12:52:58.413","0.000000"
"08/19/2015 12:52:59.427","0.000286"
"08/19/2015 12:53:00.441","0.000000"
"08/19/2015 12:53:01.455","0.000000"
"08/19/2015 12:53:02.469","0.008678"
"08/19/2015 12:53:03.483","0.000000"
"08/19/2015 12:53:04.497","0.002830"
"08/19/2015 12:53:05.511","0.000621"
"08/19/2015 12:53:06.525","0.768834"
"08/19/2015 12:53:07.539","0.000000"
"08/19/2015 12:53:08.553","1.538296"

 

Windows batch read variable – equivalent of Linux read line command

Wednesday, March 12th, 2014

bat-file-icon-windows-read-variable
If you need to do some basic batch scripting sooner or later you will have to insert input from command line to a variable. In Linux this is done with read command, i.e.:
$ echo -n "Type a password for admin:";
$ read line;
$ echo $line;

So here is how to do the same if you need it for a Windows Batch (.BAT) file

C:\\Users\\> Set /p string='What do you want to ask?:'
'What do you want to ask?:'

This will define the string variable, to later print out the variable use:
> echo %string%
variable input output

Play Dune2 on Debian Linux with dosbox – Dune 2 Mother of all Real Time Strategy games

Saturday, March 1st, 2014

medium_1809-dune-ii-the-building-of-a-dynasty_one_of_best_games_ever_linux_windows.gif

Dune II: The Building of a Dynasty (known also as Dune II: Battle for Arrakis in Europe is a game that my generation will never forget. Dune 2 is the "first" computer Real Time Strategy (RTE) game of the genre of the Warcraft I and Warcraft II / III and later Command and Conquer – Red Aleart, Age of Empires I / II and Starcraft …

dune2-unit-destroyed

I've grown up with Dune2 and the little computer geek community in my school was absolutely crazy about playing it. Though not historically being the first Real Time Strategy game, this Lucas Inc. 
game give standards that for the whole RTE genre for years and will stay in history of Computer Games as one of best games of all times.

I've spend big part of my teenager years with my best friends playing Dune2 and the possibility nowadays to resurrect the memories of these young careless years is a blessing.  Younger computer enthusiasts and gamers probably never heard of Dune 2 and this is why I decided to place a little post here about this legendary game.

dune-2-tank-vehicle - one of best games computer games ever

Its worthy out of curiosity or for fun to play Dune 2 on modern OS be it Windows or Linux. Since Dune is DOS game, it is necessary to play it via DOS emulator i.e. – (DosBox). 
Here is how I run dune2 on my Debian Linux:

1. Install dosbox DOS emulator

apt-get install --yes dosbox

2. Download Dune2 game executable

You can download my mirror of dune2 here

Note that you will need unzip to uanrchive it, if you don't have it installed do so:

apt-get install --yes unzip

cd ~/Downloads/
wget https://www.pc-freak.net/files/dune-2.zip

3.  Unzip archive and create directory to mount it emulating 'C:\' drive

mkdir -p ~/.dos/Dune2
cd ~/.dos/Dune2

unzip ~/Downloads/dune-2.zip
 

4. Start dosbox and create permanent config for C: drive auto mount


dosbox

To make C:\ virtual drive automatically mounted you have to write a dosbox config from inside dbox console

config -writeconf /home/hipo/.dosbox.conf

My home dir is in /home/hipo, change this with your username /home/username

Then exit dosbox console with 'exit' command

To make dune2 game automatically mapped on Virtual C: drive:

echo "mount c /home/hipo/.dos" >> ~/.dosbox.conf

Further to make dosbox start each time with ~/.dosbox.conf add alias to your ~/.bashrc 

vim ~/.bashrc
echo "alias dosbox='dosbox -conf /home/hipo/.dosbox.conf'" >> ~/.bashrc
source ~/.bashrc

Then to run DUNE2 launch dosbox:

dosbox

and inside console type:

c:
cd Dune2
Dune2.exe

dune2-first-real-time-strategy-game-harkonen-screenshot

For the lazy ones who would like to test dune you can play dune 2 online on this website

Finding spam sending php scripts on multiple sites servers – Tracing and stopping spammer PHP scripts

Monday, April 14th, 2014

stop_php_mail-spam-find-spammer-and-stop-php-spammer-websites
Spam has become a severe issue for administrators, not only for mail server admins but also for webshosting adms. Even the most secure spam protected mail server can get affected by spam due to fact it is configured to relay mail from other servers acting as web hosting sites.

Webhosting companies almost always suffer seriously from spam issues and often their mail servers gets blocked (enter spam blacklists), because of their irresponsible clients uploading lets say old vulnerable Joomla, WordPress without Akismet or proper spam handling plugin,a CMS which is not frequently supported / updated or custom client insecure php code.

What I mean is Shared server A is often configured to sent mail via (mail) server B. And often some of the many websites / scripts hosted on server A gets hacked and a spam form is uploaded and tons of spam start being shipped via mail server B.

Of course on mail server level it is possible to configure delay between mail sent and adopt a couple of policies to reduce spam, but the spam protection issue can't be completely solved thus admin of such server is forced to periodically keep an eye on what mail is sent from hosting server to mail server.
 


If you happen to be one of those Linux (Unix) webhosting admins who find few thousand of spammer emails into mail server logs or your eMail server queue and you can't seem to find what is causing it, cause there are multiple websites shared hosting using mainly PHP + SQL and you can't identify what php script is spamming by reviewing  Apache log / PHP files. What you can do is get use of:

PHP mail.log directive

Precious tool in tracking spam issues is a PHP Mail.log parameter, mail log paramater is available since PHP version >= 5.3.0 and above.
PHP Mail.log parameter records all calls to the PHP mail() function including exact PHP headers, line numbers and path to script initiating mail sent.

Here is how it is used:
 

1. Create empty PHP Mail.log file

touch /var/log/phpmail.log

File has to be writtable to same user with which Apache is running in case of Apache with SuPHP running file has to be writtable by all users.

On Debian, Ubunut Linux:

chown www:data:www-data /var/log/phpmail.log

On CentOS, RHEL, SuSE phpmail.log has to be owned by httpd:

chown httpd:httpd /var/log/phpmail.log

On some other distros it might be chown nobody:nobody etc. depending on the user with which Apache server is running.

 

2. Add to php.ini configuration following lines

mail.add_x_header = On
mail.log = /var/log/phpmail.log

PHP directive instructs PHP to log complete outbund Mail header sent by mail() function, containing the UID of the web server or PHP process and the name of the script that sent the email;
 

(X-PHP-Originating-Script: 33:mailer.php)


i.e. it will make php start logging to phpmail.log stuff like:
 

 

mail() on [/var/www/pomoriemonasteryorg/components/com_xmap/2ktdz2.php:1]: To: info@globalremarketing.com.au — Headers: From: "Priority Mail" <status_93@pomoriemon
astery.org> X-Mailer: MailMagic2.0 Reply-To: "Priority Mail" <status_93@pomoriemonastery.com> Mime-Version: 1.0 Content-Type: multipart/alternative;boundary="——
—-13972215105347E886BADB5"
mail() on [/var/www/pomoriemonasteryorg/components/com_xmap/2ktdz2.php:1]: To: demil7167@yahoo.com — Headers: From: "One Day Shipping" <status_44@pomoriemonastery.
org> X-Mailer: CSMTPConnectionv1.3 Reply-To: "One Day Shipping" <status_44@pomoriemonastery.com> Mime-Version: 1.0 Content-Type: multipart/alternative;boundary="—
——-13972215105347E886BD344"
mail() on [/var/www/pomoriemonasteryorg/components/com_xmap/2ktdz2.php:1]: To: domainmanager@nadenranshepovser.biz — Headers: From: "Logistics Services" <customer.
id86@pomoriemonastery.com> X-Mailer: TheBat!(v3.99.27)UNREG Reply-To: "Logistics Services" <customer.id86@pomoriemonastery.com> Mime-Version: 1.0 Content-Type: mult
ipart/alternative;boundary="———-13972215105347E886BF43E"
mail() on [/var/www/pomoriemonasteryorg/components/com_xmap/2ktdz2.php:1]: To: bluesapphire89@yahoo.com — Headers: From: "Priority Mail" <status_73@pomoriemonaster
y.org> X-Mailer: FastMailer/Webmail(versionSM/1.2.6) Reply-To: "Priority Mail" <status_73@pomoriemonastery.com> Mime-Version: 1.0 Content-Type: multipart/alternativ
e;boundary="———-13972215105347E886C13F2"

 

On Debian / Ubuntu Linux to enable this logging, exec:

echo 'mail.add_x_header = On' >> /etc/php5/apache2/php.ini
echo 'mail.log = /var/log/phpmail.log' >> /etc/php5/apache2/php.ini


I find it useful to symlink /etc/php5/apache2/php.ini to /etc/php.ini its much easier to remember php location plus it is a standard location for many RPM based distros.

ln -sf /etc/php5/apache2/php.ini /etc/php.ini

Or another "Debian recommended way" to enable mail.add_x_header logging on Debian is via:

echo 'mail.add_x_header = On' >> /etc/php5/conf.d/mail.ini
echo 'mail.log = /var/log/phpmail.log' >> /etc/php5/conf.d/mail.ini

On Redhats (RHEL, CentOS, SuSE) Linux issue:

echo 'mail.add_x_header = On' >> /etc/php.ini
echo 'mail.log = /var/log/phpmail.log' >> /etc/php.ini

3. Restart Apache

On Debian / Ubuntu based linuces:

/etc/init.d/apache2 restart

P.S. Normally to restart Apache without interrupting client connections graceful option can be used, i.e. instead of restarting do:

/etc/init.d/apache2 graceful

On RPM baed CentOS, Fedora etc.:

/sbin/service httpd restart

or

apachectl graceful
 

4. Reading the log

To review in real time exact PHP scripts sending tons of spam tail it:

tail -f /var/log/phpmail.log

 

mail() on [/var/www/remote-admin/wp-includes/class-phpmailer.php:489]: To: theosfp813@hotmail.com — Headers: Date: Mon, 14 Apr 2014 03:27:23 +0000 Return-Path: wordpress@remotesystemadministration.com From: WordPress Message-ID: X-Priority: 3 X-Mailer: PHPMailer (phpmailer.sourceforge.net) [version 2.0.4] MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="UTF-8"
mail() on [/var/www/pomoriemonasteryorg/media/rsinstall_4de38d919da01/admin/js/tiny_mce/plugins/inlinepopups/skins/.3a1a1c.php:1]: To: 2070ccrabb@kiakom.net — Headers: From: "Manager Elijah Castillo" <elijah_castillo32@pomoriemonastery.com> X-Mailer: Mozilla/5.0 (Windows; U; Windows NT 5.0; es-ES; rv:1.9.1.7) Gecko/20100111 Thunderbird/3.0.1 Reply-To: "Manager Elijah Castillo" <elijah_castillo32@pomoriemonastery.com> Mime-Version: 1.0 Content-Type: multipart/alternative;boundary="———-1397463670534B9A76017CC"
mail() on [/var/www/pomoriemonasteryorg/media/rsinstall_4de38d919da01/admin/js/tiny_mce/plugins/inlinepopups/skins/.3a1a1c.php:1]: To: 20wmwebinfo@schools.bedfordshire.gov.uk — Headers: From: "Manager Justin Murphy" <justin_murphy16@pomoriemonastery.com> X-Mailer: Opera Mail/10.62 (Win32) Reply-To: "Manager Justin Murphy" <justin_murphy16@pomoriemonastery.com> Mime-Version: 1.0 Content-Type: multipart/alternative;boundary="———-1397463670534B9A7603ED6"
mail() on [/var/www/pomoriemonasteryorg/media/rsinstall_4de38d919da01/admin/js/tiny_mce/plugins/inlinepopups/skins/.3a1a1c.php:1]: To: tynyrilak@yahoo.com — Headers: From: "Manager Elijah Castillo" <elijah_castillo83@pomoriemonastery.com> X-Mailer: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; pl; rv:1.9.1.9) Gecko/20100317 Thunderbird/3.0.4 Reply-To: "Manager Elijah Castillo" <elijah_castillo83@pomoriemonastery.com> Mime-Version: 1.0 Content-Type: multipart/alternative;boundary="———-1397463670534B9A7606308"
mail() on [/var/www/pomoriemonasteryorg/media/rsinstall_4de38d919da01/admin/js/tiny_mce/plugins/inlinepopups/skins/.3a1a1c.php:1]: To: 2112macdo1@armymail.mod.uk — Headers: From: "Manager Justin Murphy" <justin_murphy41@pomoriemonastery.com> X-Mailer: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; pl; rv:1.9.1.9) Gecko/20100317 Thunderbird/3.0.4 Reply-To: "Manager Justin Murphy" <justin_murphy41@pomoriemonastery.com> Mime-Version: 1.0 Content-Type: multipart/alternative;boundary="———-1397463670534B9A76086D1"

 

As you can see there is a junky spam mails sent via some spammer script uploaded under name .3a1a1c.php, so to stop the dirty bastard, deleted the script:

rm -f /var/www/pomoriemonasteryorg/media/rsinstall_4de38d919da01/admin/js/tiny_mce/plugins/inlinepopups/skins/.3a1a1c.php

It is generally useful to also check (search) for all hidden .php files inside directoring storing multiple virtualhost websites, as often a weirdly named hidden .php is sure indicator of either a PHP Shell script kiddie tool or a spammer form.

Here is how to Find all Hidden Perl / PHP scripts inside /var/www:

find . -iname '.*.php'
./blog/wp-content/plugins/fckeditor-for-wordpress-plugin/ckeditor/plugins/selection/.0b1910.php
./blog/wp-content/plugins/fckeditor-for-wordpress-plugin/filemanager/browser/default/.497a0c.php
./blog/wp-content/plugins/__MACOSX/feedburner_feedsmith_plugin_2.3/._FeedBurner_FeedSmith_Plugin.php

find . -iname '.*.pl*'

….

Reviewing complete list of all hidden files is also often useful to determine shitty cracker stuff

 find . -iname ".*"

Debugging via  /var/log/phpmail.log enablement is useful but is more recommended on development and staging (QA) environments. Having it enable on productive server with high amounts of mail sent via PHP scripts or just on dedicated shared site server could cause both performance issues, hard disk could quickly get and most importantly could be a severe security hole as information from PHP scripts could be potentially exposed to external parties.

Find all hidden files in Linux, Delete, Copy, Move all hidden files

Tuesday, April 15th, 2014

search-find-all-hidden-files-linux-delete-all-hidden-files
Listing hidden files is one of the common thing to do as sys admin. Doing manipulations with hidden files like copy / delete / move is very rare but still sometimes necessary here is how to do all this.

1. Find and show (only) all hidden files in current directory

find . -iname '.*' -maxdepth 1

maxdepth – makes files show only in 1 directory depth (only in current directory), for instance to list files in 2 subdirectories use -maxdepth 3 etc.

echo .*;

Yeah if you're Linux newbie it is useful to know echo command can be used instead of ls.
echo * command is very useful on systems with missing ls (for example if you mistakenly deleted it 🙂 )

2. Find and show (only) all hidden directories, sub-directories in current directory

To list all directories use cmd:

find /path/to/destination/ -iname ".*" -maxdepth 1 -type d

3. Log found hidden files / directories

find . -iname ".*" -maxdept 1 -type f | tee -a hidden_files.log

find . -iname ".*" -maxdepth 1 type d | tee -a hidden_directories.log
4. Delete all hidden files in current directory

cd /somedirectory
find . -iname ".*" -maxdepth 1 -type f -delete

5. Delete all hidden files in current directory

cd /somedirectory
find . -iname ".*" -maxdepth 1 -type d -delete

6. Copy all hidden files from current directory to other "backup" dir

find . -iname ".*" -maxdepth 1 -type f -exec cp -rpf '{}' directory-to-copy-to/ ;

7. Copy and move all hidden sub-directories from current directory to other "backup" dir

find . -iname ".*" -maxdepth 1 -type d -exec cp -rpf '{}' directory-to-copy-to/ ;

– Moving all hidden sub-directories from current directory to backup dir

find . -iname ".*" -maxdepth 1 -type d -exec mv '{}' directory-to-copy-to/ ;