Posts Tagged ‘example’

Putty load as default session another session – Save other Putty session configuration to default howto

Thursday, November 29th, 2018

putty-load-button-screenshot

Recently I had to use PuTTY which I haven't used for years to open a number of SSH Pernanent Tunnels necessery for my daily work as a SAP Consultant.

I've saved them under a certain new profile and saved the set SSH Tunnel configuration not in the default Session but in separate named one, therefore had to press Load button every time after clicking over my Putty shortcut icon. 

That was annoying and took few seconds out of my life every next morning for about a week, so finally I found osme time to google it and it seemed it is pretty easy to have any Putty sessoin loaded you like.

Here is how:

1. Create a new Putty Shortcut

putty-screenshot1

putty-shortcut-screenshot-windows

Click over Putty icon while holding CTRL + SHIFT (Control SHIFT keys simultaneously ) and move the mouse somewhere on the desktop to create the shortcut.
 

2. Right click on Putty Shortcut

putty-target-screenshot-windows1

putty-target-screenshot-windows2

 

"C:\Program Files\PuTTY\putty.exe" -load "your_saved_session" "username@your_server_address" -pw "your_password"


fill out "target" field of shortcut using above code (alter to your own properties).
click Apply button.

If you need to pass a user and password from Shortcut itself (which is a bad practice for security but sometimes useful, for not so important Tunnels – for example a tunnel to an Open Proxy), do it by typing in the target field like so:
 

"C:\Program Files\PuTTY\putty.exe" -load "your_saved_session" "username@your_server_address" -pw "your_password"

 

And Hooray !!! After that when you click on PuTTy shortcut it loads your session automatically using given username and password.

rc.local missing in Debian 8 Jessie and Debian 9 Stretch and newer Ubuntu 16, Fedora, CentOS Linux – Why is /etc/rc.local not working and how to make it work again

Monday, September 11th, 2017

rc.local-not-working-solve-fix-linux-startup-with-rc.local-explained-how-to-make-rc.local-working-again-on-newer-linux-distributions

If you have installed a newer version of Debian GNU / Linux such as Debian Jessie or Debian  9 Stretch or Ubuntu 16 Xenial Xerus either on a server or on a personal Desktop laptop and you want tto execute a number of extra commands next to finalization of system boot just like we GNU / Linux users used to do already for the rest 25+ years you will be surprised that /etc/rc.local is no longer available (file is completely missing!!!).

This kind of behaviour (to avoid use of /etc/rc.local and make the file not present by default right after Linux OS install) was evident across many RedHack (Redhat) distributions such as Fedora and CentOS Linux for the last number of releases and the tendency was to also happen in Debian based distros too as it often does, however there was a possibility on this RPM based distros as well as rest of Linux distros to have the /etc/rc.local manually created to work around the missing file.

But NOoooo, the smart new generation GNU / Linux architects with large brains decided to completely wipe out the execution on Linux boot of /etc/rc.local from finalization stage, SMART isn't it??

For instance If you used to eat certain food for the last 25+ years and they suddenly prohibit you to eat it because they say this is not necessery anymore how would you feel?? Crazy isn't it??

Yes I understand the idea to wipe out /etc/rc.local did have a reason as the developers are striving to constanly improve the boot speed process (and the introduction of systemd (system and service manager) in Debian 8 Jessie over the past years did changed significantly on how Linux boots (earlier used SysV boot and LSB – linux standard based init scripts), but come on guys /etc/rc.local
doesn't stone the boot process with minutes, including it will add just 2, 3 seconds extra to boot runtime, so why on earth did you decided to remove it??

What I really loved about Linux through the years was the high level of consistency and inter-operatibility, most things worked just the same way across distributions and there was some logic upgrade, but lately this kind of behaviour is changing so in many of the new things in both GUI and text mode (console) way to interact with a GNU / Linux PC all becomes messy sadly …

So the smart guys who develop Gnu / Linux distros said its time to depreciate /etc/rc.local to prevent the user to be able to execute his set of finalization commands at the end of each booted multiuser runlevel.

The good news is you can bring back (resurrect) /etc/rc.local really easy:

To so, just execute the following either in Physical /dev/tty Console or in Gnome-Terminal (for GNOME users) or for KDE GUI environment users in KDE's terminal emulator konsole:

 

cat <<EOF >/etc/rc.local
#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing.

exit 0
EOF
chmod +x /etc/rc.local
systemctl start rc-local
systemctl status rc-local


I think above is self-explanatory /etc/rc.local file is being created and then to enable it we run systemctl start rc-local and then to check the just run rc-local service status systemctl status

You will get an output similar to below:
 

 

root@jericho:/home/hipo# systemctl start rc-local
root@jericho:/home/hipo# systemctl status rc-local
● rc-local.service – /etc/rc.local Compatibility
   Loaded: loaded (/lib/systemd/system/rc-local.service; static; vendor preset:
  Drop-In: /lib/systemd/system/rc-local.service.d
           └─debian.conf
   Active: active (exited) since Mon 2017-09-11 13:15:35 EEST; 6s ago
  Process: 5008 ExecStart=/etc/rc.local start (code=exited, status=0/SUCCESS)
    Tasks: 0 (limit: 4915)
   CGroup: /system.slice/rc-local.service
sep 11 13:15:35 jericho systemd[1]: Starting /etc/rc.local Compatibility…
setp 11 13:15:35 jericho systemd[1]: Started /etc/rc.local Compatibility.

To test /etc/rc.local is working as expected you can add to print any string on boot, right before exit 0 command in /etc/rc.local

you can add for example:
 

echo "YES, /etc/rc.local IS NOW AGAIN WORKING JUST LIKE IN EARLIER LINUX DISTRIBUTIONS!!! HOORAY !!!!";


On CentOS 7 and Fedora 18 codename (Spherical Cow) or other RPM based Linux distro if /etc/rc.local is missing you can follow very similar procedures to have it enabled, make sure

/etc/rc.d/rc.local

is existing

and /etc/rc.local is properly symlined to /etc/rc.d/rc.local

Also don't forget to check whether /etc/rc.d/rc.local is set to be executable file with ls -al /etc/rc.d/rc.local

If it is not executable, make it be by running cmd:
 

chmod a+x /etc/rc.d/rc.local


If file /etc/rc.d/rc.local happens to be missing just create it with following content:

 

#!/bin/sh

# Your boot time rc.commands goes somewhere below and above before exit 0

exit 0


That's all folks rc.local not working is solved,
enjoy /etc/rc.local working again 🙂

 

How Orthodox Easter is calculated or Why Eastern Orthodox Christians celebrate Pascha (later) on a different date from Roman Catholics ?

Tuesday, April 5th, 2016

 

 

why-eastern-orthodox-eastern-is-on-different-date-from-roman-catholics-why-we-orthodox-celebrate-easter-later

 

 


Why Does Eastern Orthodox Churches celebrate their Eastern often on a later or a different date from (Western Christians) – Roman Catholic Church, Greek Catholic Church, Anglican, Presbyterian or Protestant (Evangelist), Old Catholic Methodist, Calvinist or some other kind of Lutheran or Charismatic Christian sects?

If you happen to be born innd raised in America or Western European contry it is most likely, you're little if at all or not at all aware of the fact that in the Eastern Orthodox Churches the date of Eastern (Passcha) is celebrated often on a different date from the Roman Catholic and the other schismatic churches.

For example, this year 2016 Roman Catholics already celebrated Passcha on March 27th and we Eastern Orthodox Christians are still in the Great Lent (Fasting) period and we'll have the Easter celebrated on 1st of May.
That certainly raises some questions across people who are not Eastern Orthodox, because the Western Roman Catholic Church and the rest of Protestant Christians are the biggest group of Christians out there and even atheists and people belonging to other religions such as Buddhism or Islam might be puzzled why the Eastern Orthodox Christians celebrate easter on a different they if after all they're Christian and what and why is this division among Christians?

With the increased globalization of the World and the fact that currently there are a lot of multi-national international companies and the fact that the companies and businesses today are mostly multi-cultural with people from all the world and all religions across the globe, for some countries situated within countries with predominant Eastern Orthodox population such as Bulgaria, Romania, Serbia, Greece, Ukraine, Belarus, Russia etc. it might be quite a disturbance because there Orthodox Christian country situated branches might be not able to operate on the days of Orthodox Easter. 

Here is a short example on the martch and dismatch by years between Roman Catholic and Eastern Orthodox Church by years from 2010 to 2020

 

Roman Catholic and Eastern Orthodox Easters from year 2010 to 2020

Year 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020
                       
Catholic 4/4 24/4 8/4 31/3 20/4 5/4 27/3 16/4 1/4 21/4 12/4
Orthodox 4/4 24/4 15/4 5/5 20/4 12/4 1/5 16/4 8/4 28/4 19/4

Easter 2000 to 2009

Year 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009
                     
Catholic 23/4 15/4 31/3 20/4 11/4 27/3 16/4 8/4 23/3 12/4
Orthodox 30/4 15/4 5/5 27/4 11/4 1/5 23/4 8/4 27/4 19/4


A complete list of Eastern Orthodox Pascha feast days is here

Funnly or strangely e ven many people who are situated within an Eastern Orthodox country doesn't have a good understanding why and what is the reason our Eastern Orthodox Easter feast is celebrated on a different date.
There is a lot written on the reasons why Eastern Orthodox Church celebrates on a different date Passcha from Catholics however in this article, I'll try to make it as short and clear as the reasons behind are often presented quiet messy.

 

In short there are 2 reasons (rules) why Eastern Orthodox Christian Church celebrates Eastern often on a different date:

1) The Julian calendar which is still used by the Eastern Orthodox Church for counting time
2) Adherence of the Orthodox to the early practices of the Original and Ancient Christian Church for Christian Passcha to not coincide with the Jewish Passover

The consequence of these 2 rules is that the Eastern Orthodox Christians on about 50% of the time throughout  years celebrate Christistmas later from Catholics because the Orthodox Church is following the First Church Ecumenical Council of Nicea (325 A.D.) that has set the rule that Passcha must take place always after the Jewish Passover  because in the Church the Biblical described consequence of events of the Passover and the Crucifix are followed, it appears rest of the Christian churches ignore (break) this requirement and this makes them fall under the Anatemas of the Church fathers from the I Ecumenical Council (see the Full decisions of the 7 Ecumenical Canonical Councils of the Original Church here).


Here the difference for Catholics celebrating on a different date Eastern stems from the fact since the XVI century they're using in Catholic Church the Gregorian calendar (adopted by Pope Gregory the XIII-th) because of scientifical superiority claimed by the Gregorian calendar.

Gregorian Julian calendar-fall-behind-dates by years from 1582 to year 2200

Well as you can see the scientific accuracy of Gregorian calendar is better and with years the Julian calendar is starting to miss astronomical re-calculated dates, causing mathematical incorrectness but the problem with the acceptance of the Gregorian Calendar and the reason why the Eastern Orthodox Church decided not to use the Gregorian calendar for Pascha is the fact that when the Gregorian Calendar was accepted in secular western world and the  Roman Catholic Church in order to matematically align the missing date 13 days were simply dropped off (or from Thursday 4th of October 1582 under Julian Calendar the next day Friday people started being in 15th of October, see the picture below:

inconsistency_change_from_Julian_to_Gregorian_calendar-Date_Change-and-the-missing-13-days

Well as you can see Friday October the 15,  1582 was quite a crazy year for the people as they lay down as 4th of Thursday and they woke up on 15th the next day 🙂

The Eastern Orthodox still to follow the Julian Calendar for Easter feast and some Churches adopted to using partly the Gregorian (Secular) calendar for some of the feasts under the pressure of the Greek Ecumenical Patriarchate , whether some Churches such as the Russian Orthodox Church (ROC) which as of time of writting is the biggest Orthodox Crhuch as well as the Serbian Church are completely using the Julian calendar for calculating all feast days. Now what should be known is difference between Julian and Gregorian calendar is 13 days falling back (Gregorian Calendar is 13 days in future), while Julian calendar is 13 days in time behind the Gregorian.
However as prior said sometimes throughout years the Easterns of (Western) Roman Catholic Church and Eastern Orthodox Church coincide, such a date on which we celebrated Eastern on one date is on 24 April 2011.
The two dates coincide when the full moon following the equinox comes so late that it counts as the first full moon after 21 March in the Julian calendar as well as the Gregorian.
In recent years the conciding years was frequent in 2010, 2011, 2014 and will be again  2017 but after that there will be no coincidence of Orthodox and  Catholic Easter feasts until 2034.

For those who might be wondering why the Eastern Orthodox Church choose to not celebrate Easter on a different date here is some information on how Pascha (Easter) was determined historically in the One and Holy Apostolic Church
which for historical and dogmatical reasons is today Our Holy Eastern Orthodox Church.

During the first three centuries of Christianity, some Christians celebrates Pascha on the First date after Jewish Passover and others celebrates the feast on the same time as Passover.
This was causing confusions in across Churches and this together with other heretical teachings caused the Holy Church Fathers (successors of Holy Apostols) of the First Ecumenical Council to gather and decide
how to solve the issue.

The Holy Fathers therefore devices a uniform formula for calculating the date of the Pascha that was inline and taking in consideration the most early traditions of the Church as well as the Biblical
sequence of events.

Here is the Easter calculation forumla devices on I Ecumenical Council:

Pascha is to be always celebrated on the first Sunday after the first full moon following the vernal equinox but  always after the Jewish Passover.
 

To even better ensure that there was no confusion as to when the vernal equinox occured the date of the vernal equinox was set to be March 21 (which is April 3rd or 13 days later on the Julain Calendar).
The formula was universally accepted by all of Christianity, ensuring the Pascha was celebrated on the same day throughout the entire Universe (world).
The Eastern Orthodox Church which is keeping to strictly keep the decisions of the fathers 7 Councils and thus strictly keeps the formula in the Church.
The Western Church (Roman Catholic Church) used to be holding also the Eastern calculation formula however in modern times it rejected the Nicene formulate that requires that Pascha to always follow the Jewish  Passover.

Now as the apostacy from True Christian faith is increasing Western theologians and many of the misguided or so called heretical (ecumenism movement oriented) orthodox theologians are starting to claim that the provision about the formula
was not important and thus provision of the formula is rejected and thus ignoring the fact that during years 325 – 1582 as well as the historical date from early Christian historians and Early Canons such as Canon VII of the Apostolic Canons clearly reads:

“If any Bishop, or Presbyter, or Deacon celebrate the holy day of Pascha before the vernal equinox with the Jews, let him be deposed.”
 

 The Calendar Issue came to be in 1582  by Pope Gregory XIII with  instituted reform of the traditional Julian calendar. Actually the Protestant Reformation in its beginning harshly defended the Julian Calendar and kept the new Julian Calendar as an unnecessery information at times
when large chunks of the protestantized countries such as Germany and Netherlands were thinking actively of joining the Orthodox Church, however for some historical reason and by the settle backs of the Roman Catholic Church the rejoining of large sea of protestants with Orthodox sadly never happened.

Until 1923 all Eastern Orthodox Chuches was following the Julian calendar for all yearly feasts however as in 1923 an inter-Orthodox congress was helpd in Constantinople attended by some but not all of the Orthodox Churches, the Church fathers gathering took the very
controversial decision to follow a revised Julian calendar (the so called new Orthodox Church Calendar / New Style) which is essentially the same as Gregorian calendar for all things except the celebration of Pascha and few other feasts which continued to be celebrated across all Orthodox Churches on the same dates across all  Eastern Orthodox Churches who are in full eucharistic communion.

The official claimed reason for accepting the Gregorian calendar and moving most of the feasts +13 days in future  was that within some eastern countries many of the orthodox christian workers couldn't attend many of the 12 important feasts because of work duties (for example 7th of January is a working day in most of the world except Russia and thus it is a problem for many to attend the Church service and prepare and much more confortable if the feast is celebrated on 24th of December like it is in Roman Catholic and Protestant christians) – i.e. the change was initiated most for economical reasons right after the end of the 1st world war.

The result is now many of the great 12 Church Feasts in Orthodox Church like Christmas (Nativity of Christ), Epiphany, the Nativity of the Holy Theotokos, Entrance Presentation of the Theotokos in many of the Orthodox Churches together with Catholics but not with Russian Church and the other Orthodox Churches who still stick to the Julian Calendar.
The only other feasts which are being celebrated together with Russian Orthodox Church is Pentecost and Ascension as they're movable feasts depending on Pascha according to Julian calendar.

For us Orthodox tradition and Church teachings are of paramount importance and thus sadly the change to re-visited Julian calendar just created further confusion internally in the Eastern Orthodox Church, today many people who completely stick to the Julian calendar and refuse the re-visited Julian calendar has joined Schismatic Self-Proclaimed Orthodox Old Calendar Churches or for those who prefered to stay in full communion with the rest of the Eastern Orthodox Churches and stick to old calendar joined the Russian Orthodox Church and Serbian Church as they're considered to better keep the true faith tradition and spiritual descendance from the holy apostles. The question of the calendar difference has been largely discussed within the Orthodox Church and hopefully on the upcoming Holy All Orthodox Council upcoming this year in June 16 – 27 2016 the question of returning back the Old Julian calendar for all Church feasts might be also raised and resolved, once and for all.

As a closure I'll say that the most important thing why Pascha tends to falls different from Catholics is to show that actually the Eastern Orthodox Church is the original Church and different from Roman Catholic and the true Church of Christ falling the prescription of the Church fathers.

P.S. As the so called "Holy Orthodox Council", often called by many 8th ecumenical counil already has taken place in Crete and there was no real signs for it to be neither ecuminical nor holy as the Bulgarian, The Russian, Antiochian and Georgian Orthodox Churches refused to take participation and the overall observation online from their website holycouncil.org  evidently shows that this council is not needed neither solved any of the real important non-doctrinal problems of the Orthodox Church it also prooved to be a PR kind of council trying to make in the eyes of the world the ecumenical patriarch Bartholomeo to look like an Orthodox Church (head) pope and perhaps the ones who organized this council had this intention together with the intention to put even more confusion concerning the unity of the Orthodox Church and separate the chirch into two parties (a conservatives) and (liberals) sides.
Also after the council finalized it prooved not to be really what it meant to be Thanks be to our Lord and Saviour Jesus Christ. The refusal of the Bulgarian Orthodox Church to take participation in the council from my point of view as a member of the Orthodox Church was a right one and has delayed the plans of global enemies of the church to destroy it.
An interesting rumor is the holy council in Crete was sponsored with 70 000 000 from anonymous donators within United States some priests claim the money come from secret societies such as free masonry also it is mostly ridiculous the costs that this ecumenical council impeded.

 

How to mount NFS network filesystem to remote server via /etc/fstab on Linux

Friday, January 29th, 2016

mount-nfs-in-linux-via--etc-fstab-howto-mount-remote-partitions-from-application-server-to-storage-server
If you have a server topology part of a project where 3 (A, B, C) servers need to be used to deliver a service (one with application server such as Jboss / Tomcat / Apache, second just as a Storage Server holding a dozens of LVM-ed SSD hard drives and an Oracle database backend to provide data about the project) and you need to access server A (application server) to server B (the Storage "monster") one common solution is to use NFS (Network FileSystem) Mount. 
NFS mount is considered already a bit of obsoleted technology as it is generally considered unsecre, however if SSHFS mount is not required due to initial design decision or because both servers A and B are staying in a serious firewalled (DMZ) dedicated networ then NTS should be a good choice.
Of course to use NFS mount should always be a carefully selected Environment Architect decision so remote NFS mount, imply  that both servers are connected via a high-speed gigabyte network, e.g. network performance is calculated to be enough for application A <-> to network storage B two sides communication not to cause delays for systems end Users.

To test whether the NFS server B mount is possible on the application server A, type something like:

 

mount -t nfs -o soft,timeo=900,retrans=3,vers=3, proto=tcp remotenfsserver-host:/home/nfs-mount-data /mnt/nfs-mount-point


If the mount is fine to make the mount permanent on application server host A (in case of server reboot), add to /etc/fstab end of file, following:

1.2.3.4:/application/local-application-dir-to-mount /application/remote-application-dir-to-mount nfs   rw,bg,nolock,vers=3,tcp,timeo=600,rsize=32768,wsize=32768,hard,intr 1 2


If the NTFS server has a hostname you can also type hostname instead of above example sample IP 1.2.3.4, this is however not recommended as this might cause in case of DNS or Domain problems.
If you want to mount with hostname (in case if storage server IP is being commonly changed due to auto-selection from a DHCP server):

server-hostA:/application/local-application-dir-to-mount /application/remote-application-dir-to-mount nfs   rw,bg,nolock,vers=3,tcp,timeo=600,rsize=32768,wsize=32768,hard,intr 1 2

In above example you need to have the /application/local-application-dir-to-mount (dir where remote NFS folder will be mounted on server A) as well as the /application/remote-application-dir-to-mount
Also on server Storage B server, you have to have running NFS server with firewall accessibility from server A working.

The timeou=600 (is defined in) order to make the timeout for remote NFS accessibility 1 hour in order to escape mount failures if there is some minutes network failure between server A and server B, the rsize and wsize
should be fine tuned according to the files that are being red from remote NFS server and the network speed between the two in the example are due to environment architecture (e.g. to reflect the type of files that are being transferred by the 2)
and the remote NFS server running version and the Linux kernel versions, these settings are for Linux kernel branch 2.6.18.x which as of time of writting this article is obsolete, so if you want to use the settings check for your kernel version and
NTFS and google and experiment.

Anyways, if you're not sure about wsize and and rise, its perfectly safe to omit these 2 values if you're not familiar to it.

To finally check the NFS mount is fine,  grep it:

 

# mount|grep -i nfs
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
server-hostA:/application/remote-application-dir-to-mount on /application/remote-application-dir-to-mount type nfs (rw,bg,nolock,nfsvers=3,tcp,timeo=600,rsize=32768,wsize=32768,hard,intr,addr=1.2.3.4)


That's all enjoy 🙂

 

 

How to Copy large data directories between 2 Linux / Unix servers without direct ssh / ftp access between server1 and server2 other by using SSH, TAR and Unix pipes

Monday, April 27th, 2015

how-to-copy-large-data-directories-between-2-linux-unix-servers-without-direct-ssh-ftp-access-btween-each-other

In a Web application data migration project, I've come across a situation where I have to copy / transfer 500 Gigabytes of data from Linux server 1 (host A) to Linux server 2 (host B). However the two machines doesn't have direct access to each other (via port 22) for security reasons and hence I cannot use sshfs to mount remotely host dir via ssh and copy files like local ones.

As this is a data migration project its however necessery to migrate the data finding a way … Normal way companies do it is to copy the data to External Hard disk storage and send it via some Country Post services or some employee being send in Data center to attach the SAN to new server where data is being migrated However in my case this was not possible so I had to do it different.

I have access to both servers as they're situated in the same Corporate DMZ network and I can thus access both UNIX machines via SSH.

Thanksfully there is a small SSH protocol + TAR archiver and default UNIX pipe's capabilities hack that makes possible to transfer easy multiple (large) files and directories. The only requirement to use this nice trick is to have SSH client installed on the middle host from which you can access via SSH protocol Server1 (from where data is migrated) and Server2 (where data will be migrated).

If the hopping / jump server from which you're allowed to have access to Linux  servers Server1 and Server2 is not Linux and you're missing the SSH client and don't have access on Win host to install anything on it just use portable mobaxterm (as it have Cygwin SSH client embedded )

Here is how:
 

jump-host:~$ ssh server1 "tar czf – /somedir/" | pv | ssh server2 "cd /somedir/; tar xf


As you can see from above command line example an SSH is made to server1  a tar is used to archive the directory / directories containing my hundred of gigabytes and then this is passed to another opened ssh session to server 2  via UNIX Pipe mechanism and then TAR archiver is used second time to unarchive previously passed archived content. pv command which is in the middle is not obligitory though it is a nice way to monitor status about data transfer like below:
 

500GB 0:00:01 [10,5MB/s] [===================================================>] 27%


P.S. If you don't have PV installed install it either with apt-get on Debian:

 

debian:~# apt-get install –yes pv

 

Or on CentOS / Fedora / RHEL etc.

 

[root@centos ~]# yum -y install pv

 

Below is a small chunk of PV manual to give you better idea of what it does:

NAME
       pv – monitor the progress of data through a pipe

SYNOPSIS
       pv [OPTION] [FILE]…
       pv [-h|-V]

DESCRIPTION
       pv  allows  a  user to see the progress of data through a pipeline, by giving information such as time elapsed, percentage
       completed (with progress bar), current throughput rate, total data transferred, and ETA.

       To use it, insert it in a pipeline between two processes, with the appropriate options.  Its standard input will be passed
       through to its standard output and progress will be shown on standard error.

       pv  will  copy  each  supplied FILE in turn to standard output (- means standard input), or if no FILEs are specified just
       standard input is copied. This is the same behaviour as cat(1).

       A simple example to watch how quickly a file is transferred using nc(1):

              pv file | nc -w 1 somewhere.com 3000

       A similar example, transferring a file from another process and passing the expected size to pv:

              cat file | pv -s 12345 | nc -w 1 somewhere.com 3000


Note that with too big file transfers using PV will delay data transfer because everything will have to pass through another 2 pipes, however for file transfers up to few gigabytes its really nice to include it.

If you only need to transfer huge .tar.gz archive and you don't bother about traffic security (i.e. don't care whether transferred traffic is going through encrypted SSH tunnel and don't want to put an overhead to both systems for encrypting the data and you have some unfiltered ports between host 1 and host 2 you can run netcat on host 2 to listen for connections and forward .tar.gz content via netcat's port like so:
 

linux2:~$ nc -l -p 12345 > /path/destinationfile
linux2:~$ cat /path/sourcfile | nc desti.nation.ip.address 12345


Another way to transfer large data without having connection with server1 and server2 but having connection to a third host PC is to use rsync and good old SSH Tunneling, like so:
 

jump-host:~$ ssh -R 2200:Linux-server1:22 root@Linux-server2 "rsync -e 'ssh -p 2200' –stats –progress -vaz /directory/to/copy root@localhost:/copy/destination/dir"

How to check Apache Webserver and MySQL server uptime – Check uptime of a running daemon with PS (process) command

Tuesday, March 10th, 2015

check_Apache_Webserver_and_MySQL_server_uptime_-_Check-uptime-of-running-daemon-service-with-PS-process-command

Something very useful that most Apache LAMP (Linux Apache MySQL PHP) admins should know is how to check Apache Webserver uptime and MySQL server running (uptime).
Checking Apache / MySQL uptime is primary useful for scripting purposes – creating auto Apache / MySQL service restart scripts, or just as a quick console way to check what is the status and uptime of Webserver / SQL.

My experience as a sysadmin shows that lack of Periodic Apache and MySQL restart every week or every month often creates sys-admin a lot of a headaches cause (Apache / NGINX / SQL  server) starts eating too much memory or under some circumstances leads to service or system crashes. Periodic system main services restart is especially helpful in case if Website's backend programming code is writetn in a bad and buggy uneffient way by unprofessional (novice) programmers.
While I was still working as Senior SysAdmin in Design.BG, I've encountered many such Crappy Web applications developed by dozen of different programmers (because company's programmers changed too frequently and many of the hired Web Developers ,were still learning to program, I guess same is true also for other Start-UP Web / IT Company where crappy programming code is developed you will certainly need to keep an eye on Apache / MYSQL uptime.  If that's the case below 2 quick one liners with PS command will help you keep an eye on Apache / MYSQL uptime

 

ps -eo "%U %c %t"| grep apache2 | grep -v grep|grep root
root     apache2            02:30:05

Note that above example is Debian specific on RPM based distributions you will have to grep for httpd instead of apache2
 

ps -eo "%U %c %t"| grep http| grep -v grep|grep root

root     apache2            10:30:05

To check MySQL uptine:
 

ps -eo "%U %c %t"| grep mysqld
root     mysqld_safe        20:42:53
mysql    mysqld             20:42:53


Though example is for mysql and Apache you can easily use ps cmd in same way to check any other Linux service uptime such as Java / Qmail / PostgreSQL / Postfix etc.
 

ps -eo "%U %c %t"|grep qmail
qmails   qmail-send      19-01:10:48
qmaill   multilog        19-01:10:48
qmaill   multilog        19-01:10:48
qmaill   multilog        19-01:10:48
root     qmail-lspawn    19-01:10:48
qmailr   qmail-rspawn    19-01:10:48
qmailq   qmail-clean     19-01:10:48
qmails   qmail-todo      19-01:10:48
qmailq   qmail-clean     19-01:10:48
qmaill   multilog        40-18:02:53

 

 ps -eo "%U %c %t"|grep -i nginx|grep -v root|uniq
nobody   nginx           55-01:22:44

 

ps -eo "%U %c %t"|grep -i java|grep -v root |uniq
hipo   java            27-22:02:07

 

Find all hidden files in Linux, Delete, Copy, Move all hidden files

Tuesday, April 15th, 2014

search-find-all-hidden-files-linux-delete-all-hidden-files
Listing hidden files is one of the common thing to do as sys admin. Doing manipulations with hidden files like copy / delete / move is very rare but still sometimes necessary here is how to do all this.

1. Find and show (only) all hidden files in current directory

find . -iname '.*' -maxdepth 1

maxdepth – makes files show only in 1 directory depth (only in current directory), for instance to list files in 2 subdirectories use -maxdepth 3 etc.

echo .*;

Yeah if you're Linux newbie it is useful to know echo command can be used instead of ls.
echo * command is very useful on systems with missing ls (for example if you mistakenly deleted it 🙂 )

2. Find and show (only) all hidden directories, sub-directories in current directory

To list all directories use cmd:

find /path/to/destination/ -iname ".*" -maxdepth 1 -type d

3. Log found hidden files / directories

find . -iname ".*" -maxdept 1 -type f | tee -a hidden_files.log

find . -iname ".*" -maxdepth 1 type d | tee -a hidden_directories.log
4. Delete all hidden files in current directory

cd /somedirectory
find . -iname ".*" -maxdepth 1 -type f -delete

5. Delete all hidden files in current directory

cd /somedirectory
find . -iname ".*" -maxdepth 1 -type d -delete

6. Copy all hidden files from current directory to other "backup" dir

find . -iname ".*" -maxdepth 1 -type f -exec cp -rpf '{}' directory-to-copy-to/ ;

7. Copy and move all hidden sub-directories from current directory to other "backup" dir

find . -iname ".*" -maxdepth 1 -type d -exec cp -rpf '{}' directory-to-copy-to/ ;

– Moving all hidden sub-directories from current directory to backup dir

find . -iname ".*" -maxdepth 1 -type d -exec mv '{}' directory-to-copy-to/ ;

 

Use rsync to copy from files from destination host to source host (rsync reverse copy) / few words on rsync

Monday, January 9th, 2012

I've recently had to set up a backup system to synchronize backup archive files between two remote servers and as I do usually with this situation I just set up a crontab job to periodically execute rsync to copy data from source server to the destination server . Copying SRC to DEST is the default behaviour rsync uses, however in this case I had to copy from the destination server to the source server host (in other words sync files the reversely.

The usual way to copy with rsync via SSH (from SRC to DEST) is using a cmd line like:

debian:~$ /usr/bin/rsync -avz -e ssh backup-user@xxx.xxx.xxx.xxx:/home/backup-user/my-directory .

Where the xxx.xxx.xxx.xxx is my remote server IP with which files are synched.
According to rsync manual, the proposed docs SYNOPSIS is in the format;
Local: rsync [OPTION…] SRC… [DEST

Obviusly the default way to use rsync is to copy source to destination which I used until now, but in this case I had to the other way around and copy files from a destination host to the source server. It was logical that swapping the SRC and DEST would complete my required task. Anyways I consulted with some rsync gurus in irc.freenode.net , just to make sure it is proper to just swap the SRC, DEST arguments.
I was told this is possible, so I swapped args;

debian:~$ /usr/bin/rsync -avz -e ssh . backup-user@xxx.xxx.xxx.xxx:/home/backup-user/my-directory
...

Surprisingly this worked 😉 Anyways I was adviced by by a good guy nick named scheel , that putting -e ssh to command line is generally unnecessery except if there is no some uncommon used SSH port over which the data is transferred. An example case in which -e 'ssh is necessery would be if transferring via lets say SSH port 1234;

rsync -avz -e 'ssh -p1234' /source user@host:/dest

In all other cases omitting '-e ssh' is better as '-e ssh' is rsync default. Therefore my final swapped line I put in cron to copy from a destinatio to source host with rsync looked like so:

05 03 2 * * /usr/bin/ionice -c 3 /usr/bin/rsync -avz my-directory backup-user@xxx.xxx.xxx.xxx:/home/backup-user/ >/dev/null 2>&1
 

How to create ssh tunnels / ssh tunneling on Linux and FreeBSD with openssh

Saturday, November 26th, 2011

ssh-tunnels-port-forwarding-windows-linux-bypassing-firewall-diagram
SSH tunneling
allows to send and receive traffic using a dedicated port. Using an ssh traffic can have many reasons one most common usage reason is to protect the traffic from a host to a remote server or to access port numbers which are by other means blocked by firewall, e.g.: (get around firewall filtering)
SSH tunneling works only with TCP traffic. The way to make ssh tunnel is with cmds:

host:/root# ssh -L localhost:deshost:destport username@remote-server.net
host:/root# ssh -R restport:desthost:localport username@remote-server.net
host:/root# ssh -X username@remote-server.net

This command will make ssh to bind a port on localhost of the host host:/root# machine to the host desthost:destport (destination host : destinationport). Important to say deshost is the host destination visible from the remote-server.net therefore if the connection is originating from remote-server.net this means desthost will be localhost.
Mutiple ssh tunnels to multiple ports using the above example commands is possible. Here is one example of ssh tunneling
Let’s say its necessery to access an FTP port (21) and an http port (80), listening on remote-server.net In that case desthost will be localhost , we can use locally the port (8080) insetad of 80, so it will be no necessery to make the ssh tunnel with root (admin privileges). After the ssh session gets opened both services will be accessible on the local ports.

host:/home/user$ ssh -L 21:localhost:21 -L 8080:localhost:80 user@remote-server.net

That’s all enjoy 😉

How to add a new MySQL user to have INSERT,UPDATE,DELETE permissions to a Database

Tuesday, October 25th, 2011

I needed to add a newly created MySQL user with no access to any database with no special permissions (user is created from phpmyadmin) with some permissions to a specific database which is used for the operation of a website, here are the MySQL CLI client commands I issued to make it work:

# mysql -u root -p
mysql> GRANT ALL ON Sql_User_DB.* TO Sql_User@localhost;
mysql> FLUSH PRIVILEGES;

Where in the Example Sql_User_DB is my example database to which the user is granted access and my sample user is Sql_User .
Note that without FLUSH PRIVILEGES; new privileges might not be active. 

To test further if all is fine login with Sql_User and try to list database tables.

$ mysql -u Sql_User -p
password:
mysql> USE Sql_User_DB;
mysql> SHOW TABLES;
...