Posts Tagged ‘cmd line’

Monitor service log is continously growing with Zabbix on Windows with batch userparameter script and trigger Alert if log is unchanged

Thursday, March 17th, 2022

monitor-if-log-file-is-growing-with-zabbix-zabbix-userparameter-script-howto

Recently we had an inteteresting Monitoring work task to achieve. We have an Application that is constantly simulating encrypted connections traffic to a remote side machine and sending specific data on TCP/IP ports.
Communiucation between App Server A -> App Server B should be continous and if all is working as expected App Server A messages output are logged in the Application log file on the machine which by the way Runs
Windows Server 2020.

Sometimes due to Network issues this constant reconnections from the Application S. A to the remote checked machine TCP/IP ports gets interrupted due to LAN issues or a burned Network Switch equipment, misconfiguration on the network due to some Network admin making stoopid stuff etc..

Thus it was important to Monitor somehow whether the log is growing or not and feed the output of whether Application log file is growing or it stuck to a Central Zabbix Server. 
To be able to better understand the task, lets divide the desired outcome in few parts on required:

1. Find The latest file inside a folder C:\Path-to-Service\Monitoring\Log\
2. Open the and check it is current logged records and log the time
3. Re-open file in a short while and check whether in few seconds new records are written
4. Report the status out to Zabbix
5. Make Zabbix Item / Trigger / Action in case if monitored file is not growing

In below article I'll briefly explain how Monitoring a Log on a Machine for growing was implemented using a pure good old WIN .BAT (.batch) script and Zabbix Userparameter key

 

1. Enable userparameter script for Local Zabbix-Agent on the Windows 10 Server Host


Edit Zabbix config file usually on Windows Zabbix installs file is named:

zabbix_agentd.win ]


Uncomment the following lines to enable userparameter support for zabbix-agentd:

 

# Include=c:\zabbix\zabbix_agentd.userparams.conf

Include=c:\zabbix\zabbix_agentd.conf.d\

# Include=c:\zabbix\zabbix_agentd.conf.d\*.conf


2. Create folders for userparameter script and for the userparameter.conf

Before creating userparameter you can to create the folder and grant permissions

Folder name under C:\Zabbix -> zabbix_agentd.conf.d

If you don't want to use Windows Explorer) but do it via cmd line:

C:\Users\LOGUser> mkdir \Zabbix\zabbix_agentd.conf\
C:\User\LOGUser> mkdir \Zabbix\zabbix_scripts\


3. Create Userparameter with some name file ( Userparameter-Monitor-Grow.conf )

In the directory C:\Zabbix\zabbix_agentd.conf.d you should create a config file like:
Userparameter-Monitor-Grow.conf and in it you should have a standard userparameter key and script so file content is:

UserParameter=service.check,C:\Zabbix\zabbix_scripts\GROW_LOG_MONITOR-USERPARAMETER.BAT


4. Create the Batch script that will read the latest file in the service log folder and will periodically check and report to zabbix that file is changing

notepad C:\Zabbix\zabbix_scripts\GROW_LOG_MONITOR-USERPARAMETER.BAT

REM "SCRIPT MONITOR IF FILE IS GROWING OR NOT"
@echo off

set work_dir=C:\Path-to-Service\Monitoring\Log\

set client=client Name

set YYYYMMDD=%DATE:~10,4%%DATE:~4,2%%DATE:~7,2%

set name=csv%YYYYMMDD%.csv

set mytime=%TIME:~0,8%

for %%I in (..) do set CurrDirName=%%~nxI

 

setlocal EnableDelayedExpansion

set "line1=findstr /R /N "^^" %work_dir%\output.csv | find /C ":""


for /f %%a in ('!line1!') do set number1=%%a

set "line2=findstr /R /N "^^" %work_dir%\%name% | find /C ":""


for /f %%a in ('!line2!') do set number2=%%a

 

IF  %number1% == %number2% (

echo %YYYYMMDD% %mytime% MAJOR the log is not incrementing for %client%

echo %YYYYMMDD% %mytime% MAJOR the log is not incrementing for %client% >> monitor-grow_err.log

) ELSE (

echo %YYYYMMDD% %mytime% NORMAL the log is incrementing for %client%

SETLOCAL DisableDelayedExpansion

del %work_dir%\output.csv

FOR /F "usebackq delims=" %%a in (`"findstr /n ^^ %work_dir%\%name%"`) do (

    set "var=%%a"

    SETLOCAL EnableDelayedExpansion

    set "var=!var:*:=!"

    echo(!var! >> %work_dir%\output.csv

    ENDLOCAL

)

)
 

 

To download GROW_LOG_MONITOR-USERPARAMETER.BAT click here.
The script needs to have configured the path to directory containing multiple logs produced by the Monitored Application.
As prior said it will, list the latest created file based on DATE timestamp in the folder will output a simple messages:

If the log file is being fed with data the script will output to output.csv messages continuously, either:

%%mytime%% NORMAL the log is incrementing for %%client%%

Or if the Monitored application log is not writting anything for a period it will start writting messages

%%mytime%%mytime MAJOR the log is not incrementing for %client%

The messages will also be sent in Zabbix.

Before saving the script make sure you modify the Full Path location to the Monitored file for growing, i.e.:

set work_dir=C:\Path-to-Service\Monitoring\Log\


5. Create The Zabbix Item

Set whatever service.check name you would like and a check interval to fetch the info from the userparameter (if you're dealing with very large log files produced by Monitored log of application, then 10 minutes might be too frequent, in most cases 10 minutes should be fine)
monitor-if-log-grows-windows-zabbix-item-service-check-screenshot
 

6. Create Zabbix Trigger


You will need a Trigger something similar to below:

Now considering that zabbix server receives correctly data from the client and the monitored log is growing you should in Zabbix:

%%mytime%% NORMAL the log is incrementing for %%client%%


7. Lastly create an Action to send Email Alert if log is not growing

Fix MySQL ibdata file size – ibdata1 file growing too large, preventing ibdata1 from eating all your server disk space

Thursday, April 2nd, 2015

fix-solve-mysql-ibdata-file-size-ibdata1-file-growing-too-large-and-preventing-ibdata1-from-eating-all-your-disk-space-innodb-vs-myisam

If you're a webhosting company hosting dozens of various websites that use MySQL with InnoDB  engine as a backend you've probably already experienced the annoying problem of MySQL's ibdata1 growing too large / eating all server's disk space and triggering disk space low alerts. The ibdata1 file, taking up hundreds of gigabytes is likely to be encountered on virtually all Linux distributions which run default MySQL server <= MySQL 5.6 (with default distro shipped my.cnf). The excremental ibdata1 raise appears usually due to a application software bug on how it queries the database. In theory there are no limitation for ibdata1 except maximum file size limitation set for the filesystem (and there is no limitation option set in my.cnf) meaning it is quite possible that under certain conditions ibdata1 grow over time can happily fill up your server LVM (Storage) drive partitions.

Unfortunately there is no way to shrink the ibdata1 file and only known work around (I found) is to set innodb_file_per_table option in my.cnf to force the MySQL server create separate *.ibd files under datadir (my.cnf variable) for each freshly created InnoDB table.
 

1. Checking size of ibdata1 file

On Debian / Ubuntu and other deb based Linux servers datadir is /var/lib/mysql/ibdata1

server:~# du -hsc /var/lib/mysql/ibdata1
45G     /var/lib/mysql/ibdata1
45G     total


2. Checking info about Databases and Innodb storage Engine

server:~# mysql -u root -p
password:

mysql> SHOW DATABASES;
+——————–+
| Database           |
+——————–+
| information_schema |
| bible              |
| blog               |
| blog-sezoni        |
| blogmonastery      |
| daniel             |
| ezmlm              |
| flash-games        |


Next step is to get some understanding about how many existing InnoDB tables are present within Database server:

 

mysql> SELECT COUNT(1) EngineCount,engine FROM information_schema.tables WHERE table_schema NOT IN ('information_schema','performance_schema','mysql') GROUP BY engine;
+————-+——–+
| EngineCount | engine |
+————-+——–+
|         131 | InnoDB |
|           5 | MEMORY |
|         584 | MyISAM |
+————-+——–+
3 rows in set (0.02 sec)

To get some more statistics related to InnoDb variables set on the SQL server:
 

mysqladmin -u root -p'Your-Server-Password' var | grep innodb


Here is also how to find which tables use InnoDb Engine

mysql> SELECT table_schema, table_name
    -> FROM INFORMATION_SCHEMA.TABLES
    -> WHERE engine = 'innodb';

+————–+————————–+
| table_schema | table_name               |
+————–+————————–+
| blog         | wp_blc_filters           |
| blog         | wp_blc_instances         |
| blog         | wp_blc_links             |
| blog         | wp_blc_synch             |
| blog         | wp_likes                 |
| blog         | wp_wpx_logs              |
| blog-sezoni  | wp_likes                 |
| icanga_web   | cronk                    |
| icanga_web   | cronk_category           |
| icanga_web   | cronk_category_cronk     |
| icanga_web   | cronk_principal_category |
| icanga_web   | cronk_principal_cronk    |


3. Check and Stop any Web / Mail / DNS service using MySQL

server:~# ps -efl |grep -E 'apache|nginx|dovecot|bind|radius|postfix'

Below cmd should return empty output, (e.g. Apache / Nginx / Postfix / Radius / Dovecot / DNS etc. services are properly stopped on server).

4. Create Backup dump all MySQL tables with mysqldump

Next step is to create full backup dump of all current MySQL databases (with mysqladmin):

server:~# mysqldump –opt –allow-keywords –add-drop-table –all-databases –events -u root -p > dump.sql
server:~# du -hsc /root/dump.sql
940M    dump.sql
940M    total

 

If you have free space on an external backup server or remotely mounted attached (NFS or SAN Storage) it is a good idea to make a full binary copy of MySQL data (just in case something wents wrong with above binary dump), copy respective directory depending on the Linux distro and install location of SQL binary files set (in my.cnf).
To check where are MySQL binary stored database data (check in my.cnf):

server:~# grep -i datadir /etc/mysql/my.cnf
datadir         = /var/lib/mysql

If server is CentOS / RHEL Fedora RPM based substitute in above grep cmd line /etc/mysql/my.cnf with /etc/my.cnf

if you're on Debian / Ubuntu:

server:~# /etc/init.d/mysql stop
server:~# cp -rpfv /var/lib/mysql /root/mysql-data-backup

Once above copy completes, DROP all all databases except, mysql, information_schema (which store MySQL existing user / passwords and Access Grants and Host Permissions)

5. Drop All databases except mysql and information_schema

server:~# mysql -u root -p
password:

 

mysql> SHOW DATABASES;

DROP DATABASE blog;
DROP DATABASE sessions;
DROP DATABASE wordpress;
DROP DATABASE micropcfreak;
DROP DATABASE statusnet;

          etc. etc.

ACHTUNG !!! DON'T execute!DROP database mysql; DROP database information_schema; !!! – cause this might damage your User permissions to databases

6. Stop MySQL server and add innodb_file_per_table and few more settings to prevent ibdata1 to grow infinitely in future

server:~# /etc/init.d/mysql stop

server:~# vim /etc/mysql/my.cnf
[mysqld]
innodb_file_per_table
innodb_flush_method=O_DIRECT
innodb_log_file_size=1G
innodb_buffer_pool_size=4G

Delete files taking up too much space – ibdata1 ib_logfile0 and ib_logfile1

server:~# cd /var/lib/mysql/
server:~#  rm -f ibdata1 ib_logfile0 ib_logfile1
server:~# /etc/init.d/mysql start
server:~# /etc/init.d/mysql stop
server:~# /etc/init.d/mysql start
server:~# ps ax |grep -i mysql

 

You should get no running MySQL instance (processes), so above ps command should return blank.
 

7. Re-Import previously dumped SQL databases with mysql cli client

server:~# cd /root/
server:~# mysql -u root -p < dump.sql

Hopefully import should went fine, and if no errors experienced new data should be in.

Altearnatively if your database is too big and you want to import it in less time to mitigate SQL downtime, instead import the database with:

server:~# mysql -u root -p
password:
mysql>  SET FOREIGN_KEY_CHECKS=0;
mysql> SOURCE /root/dump.sql;
mysql> SET FOREIGN_KEY_CHECKS=1;

 

If something goes wrong with the import for some reason, you can always copy over sql binary files from /root/mysql-data-backup/ to /var/lib/mysql/
 

8. Connect to mysql and check whether databases are listable and re-check ibdata file size

Once imported login with mysql cli and check whther databases are there with:

server:~# mysql -u root -p
SHOW DATABASES;

Next lets see what is currently the size of ibdata1, ib_logfile0 and ib_logfile1
 

server:~# du -hsc /var/lib/mysql/{ibdata1,ib_logfile0,ib_logfile1}
19M     /var/lib/mysql/ibdata1
1,1G    /var/lib/mysql/ib_logfile0
1,1G    /var/lib/mysql/ib_logfile1
2,1G    total

Now ibdata1 will grow, but only contain table metadata. Each InnoDB table will exist outside of ibdata1.
To better understand what I mean, lets say you have InnoDB table named blogdb.mytable.
If you go into /var/lib/mysql/blogdb, you will see two files
representing the table:

  •     mytable.frm (Storage Engine Header)
  •     mytable.ibd (Home of Table Data and Table Indexes for blogdb.mytable)

Now construction will be like that for each of MySQL stored databases instead of everything to go to ibdata1.
MySQL 5.6+ admins could relax as innodb_file_per_table is enabled by default in newer SQL releases.


Now to make sure your websites are working take few of the hosted websites URLs that use any of the imported databases and just browse.
In my case ibdata1 was 45GB after clearing it up I managed to save 43 GB of disk space!!!

Enjoy the disk saving! 🙂

Windows how to check which process locks file command – A M$ Windows equivalent of lsof command

Monday, February 23rd, 2015

windows-how-to-check-which-process-locks-file-command-a-ms-windows-equivalent-of-lsof-command

I've had a task today to deploy a new WAR (Web Application Archive) Tomcat file on Apache Tomcat server running  on Windows server 2008 R2 UAT environment.
The client Tomcat application within war is providing a frontend to an proprietary Risk Analysis application called Risiko Management (developed by a German vendor called Schleupen).
The update of WAR file was part of a version upgrade of application so, both "Risk Analysis" desktop standalone server RiskKit and the Web frontend was developed by Schleupen had to be updated.
In order to update I followed the usual .WAR Tomcat Javafile upadate Tomcat process.

1. Stopped Tomcat running service Instance via services.msc command e.g.
 

Start (menu) -> Run
 

services.msc

 

stopping-tomcat-application-howto-stop-service-ms-windows-screenshot
 


2. Move (by Renaming) old risk-analysis.war to risk-analysis_backup_2015.war

and also rename the automatically Tomcat extracted folder (named same name as the WAR archive file directory – D:\web\Apache-Tomcat-7.0.33\webapps\Risiko-Analysis\ to :\web\Apache-Tomcat-7.0.33\webapps\Risiko-Analysis_backup_2015, i.e. run:
 

C:\Users\risk-analysis> D:
D:\>
D:\> CD \Web\Apache-Tomcat-7.0.33\webapps\

D:\Web\Apache-Tomcat-7.0.33\webapps> move risk-analysis.war risk-analysis_2015.war
D:\Web\Apache-Tomcat-7.0.33\webapps> move  
Risiko-Analysis\  Risiko-Analysis_backup_2015\


But unfortunately I couldn't rename it and I got below error:

move-windows-command-access-is-denied-tiny-screenshot

Also I tried copying it using Windows Explorer Copy / Paste but this didn't worked either, and I got below error :

cant-move-risk-analysis-tomcat-java-application-error-ms-windows-screenshot

3. Finding what Locks a directory or File on M$ Windows


Obviously, the reason for unable to copy the directory was something was locking it. Actually there are plenty of locked files many running applications like Explorer do. A good example for all time locked file is Windows (swap file) pagefile.sys – this is Windows Linux equivalent of swap filesystem (enabled / disabled with spapon / swapoff commands)

Having the directory locked was a strange problem, because the Tomcat process was not running as I checked closely both in Windows taskmgr GUI interface and manually grepped for the process with tasklist command like so:

 

d:\>tasklist /m|find /i "tomcat"


tomcat7.exe                   4396 ntdll.dll, kernel32.dll, KERNELBASE.dll,

For people like me who use primary Linux , above command shows you very precious debugging information, it shows which Windows libraries (DLL) are loaded in memory and used by the process 

 

(Note that when Tomcat is running, it is visible with command)
 

D:\> wmic.exe process list brief | find /i "tomcat"
526          tomcat7.exe          8         4396       49           156569600


Just for those wondering the 156569600 number is number of bytes loaded in Windows memory used by Tomcat.

After tomcat was stopped above command returned empty string meaning obviously that tomcat is stopped ..

BTW, wmic command is very useful to get a list of process names (to list all running processes):

 D:> wmic.exe process list brief

get-all-process-names-in-command-line-with-windows-wmic-command-screenshot

Well obviously something was locking this directory (some of its subdirectories or a file name within the directory / folder), so I couldn't rename it just like that.
In Linux finding which daemon (service) is locking a file is pretty easy with lsof command (for those new to lsof check my previous article how to how to check what process listens on network port in Linux), however it was unknown to me how I can check which running service is locking a file and did a quick google search which pointed me to the famous handle part of SysInternals tools.
The command tool Handle.exe was exactly what I was looking for. 

handle-sysinternals-tool-to-windows-see-all-locked-files-and-what-is-locking-them-ms-windows-screenshot

To get list of all opened (locked) files and see which application has opened it just exec command without arguments, you will get
plenty of useful info which will help you to better understand what Windows OS is doing invisible in the background and what app uses what.

handle-command-part-of-sysinternals-witout-any-arguments-display-opened-locked-files-in-windows

handle is pretty much Windows equivalent command of Linux lsof

To get which file was locked by Tomcat I used handle in conjuntion with find /i command which is pretty much like Linux's grep equivalent

 

C:\TEMP> Handle.exe | FIND /I "Tomcat"
   1C: File  (RW-)   D:\Web\Apache-Tomcat-7.0.33\webapps\Risk-Analysis\images\app


Alternatively if you have sysinternals and prefer GUI environment you can use SysInternals Process Explorer (press CTRL + F) and look for a string:

process-explorer-toolbar-find-what-is-locking-a-file-or-directory-windows

Next to handle I found also another GUI program (Internet Explorer extension) WhoLockMe, that can be used to show you all running programs and locked files by this programs.
WhoLockMe is pretty straight forward to use, though it shows GUI output you have to run the command from cmd line. Below is sample output screenshot of wholockme.


who-lock-me-windows-screenshot-see-which-files-running-programs-are-locking-on-ms-windows

 

To Install Wholockme 


Unzip "WhoLockMe.zip" in a directory (for exemple : "C:\Program Files\WhoLockMe")
Launch "Install.bat" or execute this Windows registry modification command :
 

regsvr32 "C:\Program Files\WhoLockMe\WhoLockMe.dll"


To Uninstall WhoLockMe – if you need to later:

 

Execute command :
 

regsvr32 /u "C:\Program Files\WhoLockMe\WhoLockMe.dll"


Reboot (Or Kill Explorer.exe).

Removes the "C:\Program Files\WhoLockMe" directory and its contents.

Probably there are other ways to find out what is locking a file or direcotry using powershell scripts or .bat (batch) scripting. If you know of other way using default Windows embedded commands, please share in comments.

 

ConEmu and Clink – Add Linux shell shortcuts, tabs and tab completion to Windows command line – Far Manager Windows text file explorer

Thursday, June 12th, 2014

If you're a Linux geek and forced to work on Windows for some reason. You will probably want to add some Linux & Feel to Windows console. Yesterday in a discussion with a colleague who is considered to be our team's Windows Guru, I noticed his Windows 7 command line looks slightly different and asked him, "is it some cmd.exe customization?" and this is how I learned about existence of ConEmu and Clink.
ConEmu is advanced console window where you can run any shell of your choice.

conemu-tabbed-command-prompt-cmd-wrapper-like-gnome-terminal-for-windows

ConEmu (Console Emulator) + Clink combination is very much like MobaXterm (tabbed SSH client and win cmd line) in many sense except it doesn't have an embbed ssh support client. By using Conemu + Clink (Powerful bash style line editing) wrappers to cmd.exe you get the invaluable Linux / UNIX tab completion, command line shortcuts like CTRL + R, CTRL + A, CTRL + P etc. into Windows OS. ConEmu is very useful in Windows environments when you have to heavily use Windows PowerShell

If you're get used to Linux's Midnight Commander (MC)  and you're looking for MC for Windows you will certainly want to check out Far Manager (File and Archive Manager). Far Manager provides simple and intuitive ncurses (mc) like interface on Windows and supports view, edit, copy rename and a lot of other basic operations supported also by mc.

far-manager-mc-like-text-file-browser-for-windows-linux-mc-windows-alternative-software


Clink combines the native Windows shell cmd.exe with the powerful command line editing features of the GNU Readline library, which provides rich completion, history, and line-editing capabilities. Readline is well-known and used in Unix shell Bash, the standard shell for Mac OS X and most of Linux distributions.

In my view coolest Clink command shortcut is reverse-search allowing you to auto-type by letters commands (from command line history), lets say you want to invoke dir – you press CTRL + r and type D and all occurances earlier typed starting with "d", pops up as possible choice.

clink_bring-linux-bash-shell-powerful-command-line-editing-for-windows
Everyone who had to administer Windows servers know how annoying is that by default cmd.exe doesn't support window resize.
Using ConEmu solves that as it supports resize of command prompt Window (horizontal and vertical resize).
ConEmu has a status bar, support (configuration colors) and even transperancy but most importantly for old school dos geeks is that it supports integration with dosbox run old DOS applications (games) in Windows 7.

To use the tools use gui installers, here is latest mirror copy of Conemu (for download), current latest Far Manager release and latest present release of Clink (download link). Once installed it is a must to spend some time configure ConEmu to match your preferences. To configure it press with right mouse button on top of Cemu window and do the needful 🙂

windows-conemu-good-substitute-for-linux-bash-shell-for-microsoft-windows-users-split-screen-how-to-tab-completion-windows
Once everything is installed, you will have to get into the habit to launch ConEmu instead of cmd.exe (as ConEmu doesn't interfere with cmd.exe). To launch far manager from ConEmu command line simply type:

C:> far

To open new tabs in ConEmu the well known Firefox used CTRL + T will not work, but 'open new tab' key shortcut can be customized and also set to CTRL + T.  Default key shortcut for opening new tab in ConEmu is 'Windows Button' + SHIFT + W. While Opening new consoles in ConEmu, there is also the useful option of Splitting console (top and right).

Here is also list with all Clink command shortcuts

C-@     : set-mark
C-a     : beginning-of-line
C-b     : backward-char
C-c     : ctrl-c
C-d     : delete-char
C-e     : end-of-line
C-f     : forward-char
C-g     : abort
C-h     : backward-delete-char
C-i     : clink-completion-shim
C-j     : accept-line
C-k     : kill-line
C-l     : clear-screen
C-m     : accept-line
C-n     : next-history
C-p     : previous-history
C-q     : reload-lua-state
C-r     : reverse-search-history
C-s     : forward-search-history
C-t     : transpose-chars
C-u     : unix-line-discard
C-v     : paste-from-clipboard
C-w     : unix-word-rubout
C-y     : yank
C-z     : undo
C-]     : character-search
C-_     : undo
A-C-c   : copy-line-to-clipboard
A-C-e   : expand-env-vars
A-C-g   : abort
A-C-h   : backward-kill-word
A-C-i   : tab-insert
A-C-j   : vi-editing-mode
A-C-m   : vi-editing-mode
A-C-r   : revert-line
A-C-u   : up-directory
A-C-y   : yank-nth-arg
A-C-[   : complete
A-C-]   : character-search-backward
A-      : set-mark
A-#     : insert-comment
A-&     : tilde-expand
A-*     : insert-completions
A–     : digit-argument
A-.     : yank-last-arg
A-0     : digit-argument
A-1     : digit-argument
A-2     : digit-argument
A-3     : digit-argument
A-4     : digit-argument
A-5     : digit-argument
A-6     : digit-argument
A-7     : digit-argument
A-8     : digit-argument
A-9     : digit-argument
A-<     : beginning-of-history
A-=     : possible-completions
A->     : end-of-history
A-?     : possible-completions
A-     : delete-horizontal-space
A-_     : yank-last-arg
A-b     : backward-word
A-c     : capitalize-word
A-d     : kill-word
A-f     : forward-word
A-h     : show-rl-help
A-l     : downcase-word
A-n     : non-incremental-forward-search-history
A-p     : non-incremental-reverse-search-history
A-r     : revert-line
A-t     : transpose-words
A-u     : upcase-word
A-y     : yank-pop
A-~     : tilde-expand
C-x,C-g : abort
C-x,C-r : re-read-init-file
C-x,C-u : undo
C-x,C-x : exchange-point-and-mark
C-x,(   : start-kbd-macro
C-x,)   : end-kbd-macro
C-x,e   : call-last-kbd-macro


 

 

 

Creating Dynamic SSH Tunnel on Windows with Plink – Scriptable SSH Tunnels on Windows

Friday, July 4th, 2014

creating-ssh-tunnel-on-windows-with-plink-ssh-tunnel-diagram-tunnel-email-traffic

In my earlier articles I've explained about Creating SSH Tunnels on Linux and BSD and how to create SSH Tunnels with Putty Terminal client on Windows.

Creating SSH Tunnels is a precious knowledge every advanced computer user, a system administrator or security expert should be well aware of.

Probably still there are plenty of people for which the SSH Tunnelling is something never heard of or even if heard it is a mythical term covered by ancient shadows 🙂

So What is an SSH Tunnel? – SSH Tunnels shortly explained.

A SSH tunnel consists of an encrypted tunnel created through a SSH protocol
connection
.

An SSH tunnel can be used to transfer unencrypted traffic over a
network through an encrypted channel
between two hosts (host A and Host B) usually using remote and local port.

Requirements for SSH tunnel

Where Host B has to be running a version of OpenSSH server on some reachable port from Host B, (lets say 22).
Host B OpenSSH server has to be with (enabled X11Forwarding X11Forwarding yes in /etc/ssh/sshd_config) and Host A needs to have some SSH client supporting port forwarding (ssh command on Linux – part of (openssh-client package) and on Windows – Putty / Plink or any other of the many available ssh tunneling clients).

Probably most common SSH Tunnel use is to bypass firewalls that prohibits or filter certain internet services.
In other words SSH Tunnels can be used to get around firewall filtering. If you never heard of Bypassing firewalls with port forwarding I recommend to check this article

We can use a ssh tunnel to securely transfer files between a FTP server and a client even though the FTP
protocol itself is not encrypted
, Tnnel traffic to an SQL server, Tunnel traffic from your Desktop PC to a Proxy
SSH tunnels can be used to tunnel outbound E-mail traffic back to your work PC to avoid having to change SMTP servers etc.

A very common SSH Tunnel use is by office workers who don't want their Custom Web browsing habbits and Web history be tracked,

For people who has to travel a lot tunneling SSH traffic from notebook to your home based SMTP is mandatory, otherwise your e-mail passwords, might end up captured by someone who manages the Free WI-FI used by you somewhere on the road or cafeteria. Another good case use of SSH Tunnel is to tunnel VNC traffic.

SSH Tunneling was historically very famous in days where IRC (Internet Relay Chat) and used to be common way (for people who want to prevent their real IP address from publicly visible) on the Internet.
SSH Tunneling is great way to maintain high communication security, however because it requires some basic technical knowledge most people neglect it or never heard of it, so I hope this article will give at least basic ideas to people new to tunneling.

How to create SSH Tunnels on Microsoft Windows

SSH Tunneling has been originally invented on UNIX / Linux platform, luckily nowdays it is easily possible to create SSH tunnels on almost any moden OS (including Mac OS X and M$ Windows).

Using Plink it is possible to create easily (batch) scriptable SSH tunnels on Windows

Generally here is example of basic plink use syntax:
 

plink.exe {remote-host} -P 22 -C -D 10080 -l username -pw password
 

To make a Tunnel with Plink to remote SSH Server (remote-ssh-server.com) listening for connections on port 22 on localhost 10022

"C:Program Files (x86)PuTTYplink.exe" -ssh remote-username@tunnel-to-ssh-server-host.com -pw PASSWORD123 -C -T -D 127.0.0.1:10022 -N

-D – option instructs Plink to make "Dynamic SOCKS-based port forwarding"
Dynamic Socks-based port forwarding will make your ssh client a simple socks server which listen to port 10022 for incoming connections.
-Ttells plink not to open interactive shell on remote server (disable pty allocation)
-Cenables traffic compression

Using the -D option you can prepare tunnel to traffic all your outgoing server traffic via the SSH tunneling host.

E. g. -D allows you to use the remote ssh server as a proxy server with no need for remote server to run anything except SSH service and have Internet connection.

-N – stands for "don't start shell/command SSH-2 only"

On success of SSH Tunnel establishing you will get a pop-up window similar to below screenshot (note that some of the messages in cmd line are from a batch script – if no batch script is used you wil only get a window with "Using Username":

plink_establishing_ssh_tunnel-on-microsoft-windows-screenshot

Note: That if you're about to be using Plink command frequently add it to your Windows SYSTEM PATH (check my previous article – how to add Putty to System PATH) to enable it invokable without writting the full command path location.

Once this Window is open if you still have doubt, where the tunnel is established, the quickest way to test whether tunnel is working is to open telnet to localhost port 10022.

In Windows command prompt type:
 

C:> telnet localhost 10022

 

You will get a remote server SSH version printed, like on below shot:

windows-testing-whether-ssh-tunnel-is-working-with-telnet-screenshot-black-screen

Now lets say now you would like to access the internet via just created SSH tunnel.
You can do it by simply setting Socks Proxy in your Firefox / Opera (or whatever browser you use).

Whether you want to run the SSH Tunnel permanent and periodically check whether tunnel is allve and respawn it in case plink quit, you can use a very simple batch script like one below:
 

@ECHO OFF
tasklist /FI "IMAGENAME eq plink.exe" 2>NUL | find /I /N "plink.exe">NUL

if "%ERRORLEVEL%"=="0"  ( echo Programm is running exiting
exit
)
echo "Not Running"
start /b "C:Program Files (x86)PuTTYplink.exe" -ssh remote-username@tunnel-to-ssh-server-host.com -pw Password123 -T -C -D 127.0.0.1:10022 -N"


If you're a Linux guy and you don't want to mess around with Plink but still want to create your SSH tunnels following SSH client UNIX command line syntax, setup the Cygwin port of OpenSSH for Windows.

Or if you're unsure whether remote server is always reachable (you're moving with your notebook from (Country) network to network), you can use also portqry – windows port scanner to check whether remote ssh server port is opened:
 

@ECHO OFF
tasklist /FI "IMAGENAME eq plink.exe" 2>NUL | find /I /N "plink.exe">NUL
if "%ERRORLEVEL%"=="0"  ( echo Programm is running exiting
pause
exit
)
portqry -n tunnel-to-ssh-server-host.com -p tcp -e 22
if %ERRORLEVEL%==0 echo tunnel-to-ssh-server-host.com Reachable Connecting ..
if NOT %ERRORLEVEL%==0 (echo tunnel-to-ssh-server-host.com.com Unreachable
exit)
echo "Not Running. Starting"
"C:Program Files (x86)PuTTYplink.exe" -ssh remote-username@tunnel-to-ssh-server-host.com -pw Password123 -T -C -D 127.0.0.1:10080 –N


Here is another sample use of Plink to create tunnel via SSH host tunnel-to-ssh-server-host.com  to listen for connections on localhost port 1234 and forward all incoming traffic to squid proxy server on 192.168.1.5 on port 3128.
 

C:Usersgeorgi>plink.exe -v -x -a -T -C -noagent -ssh -L 127.0.0.1:1234:192.168.1.5:3128 remote-username@tunnel-to-ssh-server-host.com

Setting then 127.0.0.1:1234 in  Firefox Proxy browser settings will make all traffic from your browser to flow securily to your own proxy server (letting you hide your "custom traffic" from company Web sniffers (Proxying) and Web filtering.
A remark to make here is 192.168.1.5 is (a internal server with Squid Proxy, configured to pass traffic to the internet ).

Here is a way to make tunnel between your remote SSH server and local Desktop PC to make Google queries without being logged by your ISP or company where you're working:

plink.exe "-v -x -a -T -C -noagent -ssh -L 127.
0.0.1:1234:95.158.130.242:443 -pw SecretPassword123 -l hipo www.pc-freak.net"

95.158.130.242 – is one of the IPs www.google.com resolves to
www.pc-freak.net – is name of my home router.

To make tunneled queries to Google then open in browser https://127.0.0.1:1234

using_localhost_ssh_tunnel_to_anonymously-search-in-google-plink-tunnel-for-security

 

If you're a lazy windows user and don't want to bother to make your tunnels from command line and you want need nice gui way to manage multiple tunnels – check out putty-tunnel-manager.

I myself am not a SSH Tunnel expert and thus Iwould be happy to learn further from people for some interesting cases and custom ways to do  SSH Tunnels. If you know of better advantageous ways to tunnel traffic, please share.

Happy tunneling ! 🙂

Use rsync to copy from files from destination host to source host (rsync reverse copy) / few words on rsync

Monday, January 9th, 2012

I've recently had to set up a backup system to synchronize backup archive files between two remote servers and as I do usually with this situation I just set up a crontab job to periodically execute rsync to copy data from source server to the destination server . Copying SRC to DEST is the default behaviour rsync uses, however in this case I had to copy from the destination server to the source server host (in other words sync files the reversely.

The usual way to copy with rsync via SSH (from SRC to DEST) is using a cmd line like:

debian:~$ /usr/bin/rsync -avz -e ssh backup-user@xxx.xxx.xxx.xxx:/home/backup-user/my-directory .

Where the xxx.xxx.xxx.xxx is my remote server IP with which files are synched.
According to rsync manual, the proposed docs SYNOPSIS is in the format;
Local: rsync [OPTION…] SRC… [DEST

Obviusly the default way to use rsync is to copy source to destination which I used until now, but in this case I had to the other way around and copy files from a destination host to the source server. It was logical that swapping the SRC and DEST would complete my required task. Anyways I consulted with some rsync gurus in irc.freenode.net , just to make sure it is proper to just swap the SRC, DEST arguments.
I was told this is possible, so I swapped args;

debian:~$ /usr/bin/rsync -avz -e ssh . backup-user@xxx.xxx.xxx.xxx:/home/backup-user/my-directory
...

Surprisingly this worked 😉 Anyways I was adviced by by a good guy nick named scheel , that putting -e ssh to command line is generally unnecessery except if there is no some uncommon used SSH port over which the data is transferred. An example case in which -e 'ssh is necessery would be if transferring via lets say SSH port 1234;

rsync -avz -e 'ssh -p1234' /source user@host:/dest

In all other cases omitting '-e ssh' is better as '-e ssh' is rsync default. Therefore my final swapped line I put in cron to copy from a destinatio to source host with rsync looked like so:

05 03 2 * * /usr/bin/ionice -c 3 /usr/bin/rsync -avz my-directory backup-user@xxx.xxx.xxx.xxx:/home/backup-user/ >/dev/null 2>&1