Posts Tagged ‘process’

How to disown a process once it is running on Linux – old but useful trick

Thursday, December 20th, 2018

how-to-disown-a-shell-running-process-on-linux-trick

There is one very old but  gold useful UNIX / Linux trick, I remembered which will be interesting to share it's called  it is called disowning.


Lets say you run execution of a job an rsync job or a simple copy job of a very large file, but in the middle of the copy you remembered you need to do something else and thus want to switch back to shell (without opening a new ssh if on remote server) or a new console if on a local machine.
Then how can you background the copy process and move the process to the rest of long running process system list e.g. "disown" it from yourself so the process continues its job in the background just like of the rest of the backgrounded running processes on the system.

Here is the basic syntax of the disown command:
 

help disown
disown: disown [-h] [-ar] [jobspec …]
    By default, removes each JOBSPEC argument from the table of active jobs.
    If the -h option is given, the job is not removed from the table, but is
    marked so that SIGHUP is not sent to the job if the shell receives a
    SIGHUP.  The -a option, when JOBSPEC is not supplied, means to remove all
    jobs from the job table; the -r option means to remove only running jobs.

 

Here is a live example of what I meant by above lines and actual situation where disown comes super useful.

The 'disown' command/builtin (this is in bash), which will disassociate the process from the shell and not send the HUP signal to the process on exit.

root@linux:~# cp -rpf SomeReallyLargeFile1 SomeReallylargeFile2

[1]+  Stopped                 cp -i -r SomeReallyLargeFile SomeReallylargeFile2
root@linux:~#  bg %1
[1]+ cp -i -r SomeReallyLargeFile SomeReallylargeFile2 &
root@linux:~#  jobs
[1]+  Running                 cp -i -r testLargeFile largeFile2 &
root@linux:~# disown -h %1
root@linux:~# ps -ef |grep largeFile2
root      5790  5577  1 10:04 pts/3    00:00:00 cp -i -rpf SomeReallyLargeFile SomeReallylargeFile2
root      5824  5577  0 10:05 pts/3    00:00:00 grep largeFile2
root@linux:~#


Of course you can always use something like GNU screen (VT100/ ANSI Terminal screen manager) or tmux (terminal multiplexer) to detach the process but you will have to have run the screen  / tmux session in advance which you might haven't  yet as well as it is  required one of the 2 to be present on a servers and on many servers in complex client environments this might be missing and hard to install (such as server is behind a firewall DMZ-ed (Demilitirezed Zoned) network and no way to install extra packages), the disown command makes sense.

Another useful old tip, that new Linux users might not konw is the nohup command (which runs a command immune to hangups with output to a non-tty), nohup's main use is if you want to run process in background with (ampersand) from bash / zsh / tcsh etc. and keep the backgrounded process running even once you've exited the active shell, to do so run the proc background as follows:
 

$ nohup command-to-exec &

 

Hope this helps someone, Enjoy!

 

Install postgresql on Debian Squeeze / How to install PostGreSQL on Obsolete Debian installation

Friday, June 10th, 2016

how-to-install-postgresql-on-obsolete-old-debian-squeeze-tutorial

If you're in position like me to be running an old version of Debian (Squeeze) and you need to install PostgreSQL you will notice that the Debian 6.0 standard repositories are no longer active and apt-get update && apt-get upgrade are returning errors, thus because this Debian release is already too old and even the LTS repositories are inactive it is impossible to install postgresql with the usual.

To get around the situation first thing I did was to try to add followin Debian  repositories. to /etc/apt/sources.list
 

deb http://ftp.debian.net/debian-backports squeeze-backports-sloppy main
deb http://archive.debian.org/debian-archive/debian/ squeeze main contrib non-free
deb http://archive.debian.org/debian-archive/debian/ squeeze-lts main contrib non-free

After adding it I continued getting missing package errors while trying:
 

# apt-get update && apt-get install postgresql postgresql-client
….
…..

 

E: Some index files failed to download. They have been ignored, or old ones used instead.


Thus I googled a bit and I found the following PostgreSQL instructions working Debian 7.0 Wheeze and decided to try it 1 in 1 just changing the repository package wheezy word with squeeze
in original tutorial postgre's deb repositories are:

 

deb http://apt.postgresql.org/pub/repos/apt/ wheezy-pgdg main


I've only changed that one with:

 

deb http://apt.postgresql.org/pub/repos/apt/ squeeze-pgdg main

 

I guess though this worked for Debian Squeeze installing current versions such as Debian 8.0 Jessis and newer wouldn't be a prolem if you just change the debian version keyword witht he distribution for which you need the postgresql package


Here is all the consequential steps I took to make the PostgreSQL 9.5 running on my old and unsupported Debian 6.0 Squeeze

Create /etc/apt/sources.list.d/pgdg.list. The distributions are called codename-pgdg. In the example, replace wheezy with the actual distribution you are using:

# vim /etc/apt/sources.list.d/pgdg.list

 

deb http://apt.postgresql.org/pub/repos/apt/ squeeze-pgdg main

debian:~# apt-get –yes install wget ca-certificates debian:~# wget –quiet -O – https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add – debian:~# apt-get update debian:~# apt-get upgrade debian:~# apt-get –yes install postgresql-9.5 pgadmin3

Next step is to connect to PostGreSQL and create database user and a database # su – postgres $ psql

Create a new database user and a database:

postgres=# CREATE USER mypguser WITH PASSWORD 'mypguserpass'; postgres=# CREATE DATABASE mypgdatabase OWNER mypguser;

 

or

# createuser mypguser #from regular shell # createdb -O mypguser mypgdatabase

Quit from the database

postgres=# q

Connect as user mypguser to new database

# su – mypguser $ psql mypgdatabase

or

# psql -d mypgdatabase -U mypguser

If you get errors like:

psql: FATAL: Ident authentication failed for user "mypguser"

edit pg_hba.conf in /etc/postgresql/9.5.Y/main/pg_hba.conf

 

local all all trust # replace ident or peer with trust

reload postgresql

/etc/init.d/postgresql reload …

 


To make sure that PostGreSQL is running on the system check the following processes are present on the server:

 

 

 

root@pcfreak:/var/www/images# ps axu|grep -i post postgres 9893 0.0 0.0 318696 16172 ? S 15:20 0:00 /usr/lib/postgresql/9.5/bin/postgres -D /var/lib/postgresql/9.5/main -c config_file=/etc/postgresql/9.5/main/postgresql.conf postgres 9895 0.0 0.0 318696 1768 ? Ss 15:20 0:00 postgres: checkpointer process postgres 9896 0.0 0.0 318696 2700 ? Ss 15:20 0:00 postgres: writer process postgres 9897 0.0 0.0 318696 1708 ? Ss 15:20 0:00 postgres: wal writer process postgres 9898 0.0 0.0 319132 2564 ? Ss 15:20 0:00 postgres: autovacuum launcher process postgres 9899 0.0 0.0 173680 1652 ? Ss 15:20 0:00 postgres: stats collector process root 14117 0.0 0.0 112404 924 pts/1 S+ 16:09 0:00 grep -i post

 

 


Well that's all folks now you will have the postgresql running on its default port 5433:

 

debian:/etc/postgresql/9.5/main# grep -i port postgresql.conf
port = 5433 # (change requires restart)
# supported by the operating system:
# supported by the operating system:
# ERROR REPORTING AND LOGGING # %r = remote host and port

 

 


Well that's it folks thanks The Lord Jesus Christ grace by the prayers of John The Baptist and Saint Sergij Radonezhki it works 🙂

 

 

Windows unable to delete file, file locked unlocking with Unlocker tiny tool

Wednesday, April 13th, 2016

Windows-Unlocker-program-show-what-program-or-software-is-locking-your-file-why-file-cant-be-deleted-file-locked

If you want to delete some file on a Windows server or Desktop but you get the a dialog with an error saying:

"This action can't be completed because the folder or a file in it is open in another program"

windows-unable-to-delete-file-file-locked-get-what-is-locking-it-and-unlock-the-file-with-Unlocker-tiny-desktop-graphic-tool-0

Then you need to find out which Program is preventing the file from deletion / locking the file, I've earlier blogged on how to check which process locks file with tasklist or wmic Windows commands

However some users might prefer to not bother with command line check what is locking a file and then killing the Process manually with taskmanager (taskmgr.exe) but do both file unlocking from one single gui interface, that's especially for lazy novice users, gamers and most of Desktop Windows users.

If you're one of those lazy users you will appriace

Unlocker – a useful utility for unlocking files, it will help you figure out which file is using a file what program is using the file you're prevented to delete.
Unlocker is the tool for you if you get any of below error messages, when you try to delete a file:

 

  • Cannot delete folder: It is being used by another person or program
  • Cannot delete file: Access is denied
  • There has been a sharing violation.
  • The source or destination file may be in use.
  • The file is in use by another program or user.
  • Make sure the disk is not full or write-protected and that the file is not currently in use.
     

windows-unable-to-delete-file-file-locked-get-what-is-locking-it-and-unlock-the-file-with-Unlocker-tiny-desktop-graphic-tool-1

If you stumble unto an locked file once you download and install Unlocker tool and launch the tool ( in case it disappers in future a mirror of Unlocker tool here ).
Once installed if you click properties over the file which is refused to be deleted you will get a new menu such as in below screenshot:

NB! Beware while installing Unlocker you might be offered to install a bunch of malware (make sure you deselect it). Also Unlocker's site is made in a way so the Download button could easily be confused with some Google Adsense

unlocker-windows-menu-added-to-properties-options-screenshot

Click on the file that is being locked and choose the Unlocker button, for example if you have a bunch of Videos installed and the video is being locked by VLC clicking on the file you will be shown VLC like in below screenshot

 

Unlocker-screenshot-locked-file-because-movie-opened-in-VLC

As you see you're shown the Process PID that is being used by the file the full path to the locking program and you have the option to quickly kill the process or unlock the file. Note that at some cases unlocking a file used by some critical program lets say Microsoft Word / Excel or OneNote could cause you a data loss, so before unlocking a file make sure you know what you're doing.

For more advanced users that still prefer GUI to find out what is Locking a file you can also check out Microsoft Process Explorer (advabced task manager) like tool.
If you haven't tried Process Explorer be sure to take a look at it as its a great tool for Win SysAdmins:

Process Explorer is very handy if you want to explore which .DLL (Windows Libraries) are used by a Process / Program

Windows-process-explorer-an-advanced-task-manager-for-windows-and-handy-tool-to-see-what-external-libraries-and-files-a-program-is-using.png

Windows-process-explorer-an-advanced-task-manager-for-windows-and-handy-tool-to-see-what-external-libraries-and-files-a-program-is-using-1

 

How to force logrorate process logs / Make logrotate changes take effect immediately

Sunday, April 10th, 2016

how-to-force-logrorate-to-process-logs-make-logrorate-changes-take-effect-immediately-log-rotate-300x299

Dealing with logrorate as admins we need to change or add new log-rorate configurations (on most Linux distributions configs are living uder
/etc/logrotate.d/
 

logrotate uses crontab to work. It's scheduled work, not as daemon, so usually no need to reload its configuration.
When the crontab executes logrotate, it will use your new config file automatically.

Most of the logrotate setups I've seen on various distros runs out of the /etc/cron.daily

$ ls -l /etc/cron.daily/logrotate 
-rwxr-xr-x 1 root root 180 May 18  2014 /etc/cron.daily/logrotate

Here is content of cron job scheduled script:

$ cat /etc/cron.daily/logrorate

#!/bin/sh /usr/sbin/logrotate /etc/logrotate.conf EXITVALUE=$? if [ $EXITVALUE != 0 ]; then /usr/bin/logger -t logrotate "ALERT exited abnormally with [$EXITVALUE]" fi exit 0

Configurations change to lograte configs takes effect on next crontab run,
but what If you need to test your config you can also execute logrotate
on your own with below ommand:

 

logrotate -vf /etc/logrotate.conf 

If you encounter some issues with just modified or newly added logrorate script to check out the status of last logrorate executing bunch of log-rotate scripts run on Debian  / Ubuntu etc. deb based Linux:

cat /var/lib/logrotate/status

Or on RHEL, Fedora, CentOS Linux


cat /var/lib/logrotate.status

logrotate state -- version 2

 

"/var/log/syslog" 2016-4-9
"/var/log/dpkg.log" 2016-4-1
"/var/log/unattended-upgrades/unattended-upgrades.log" 2012-9-20
"/var/log/unattended-upgrades/unattended-upgrades-shutdown.log" 2013-5-17
"/var/log/apache2/mailadmin.pc-freak.net-access.log" 2012-9-19
"/var/log/snort/portscan.log" 2012-9-12
"/var/log/apt/term.log" 2016-4-1
"/var/log/squid/access.log" 2015-3-21
"/var/log/mysql/mysql-slow.log" 2016-4-9
"/var/log/debug" 2016-4-3
"/var/log/mysql.log" 2016-4-9
"/var/log/squid/store.log" 2015-3-21
"/var/log/apache2/mailadmin.pc-freak.net-error.log" 2012-9-19
"/var/log/daemon.log" 2016-4-3
"/var/log/munin/munin-update.log" 2016-4-9
"/var/log/unattended-upgrades/unattended-upgrades*.log" 2013-5-16
"/var/log/razor-agent.log" 2015-2-19
"/var/log/btmp" 2016-4-1
"/var/log/squid/*.log" 2014-11-24
"/var/log/munin/munin-graph.log" 2016-4-9
"/var/log/mysql/mysql.log" 2012-9-12
"/var/log/munin/munin-html.log" 2016-4-9
"/var/log/clamav/freshclam.log" 2016-4-3
"/var/log/munin/munin-node.log" 2016-1-23
"/var/log/mail.info" 2016-4-3
"/var/log/apache2/other_vhosts_access.log" 2016-4-3
"/var/log/exim4/rejectlog" 2012-9-12
"/var/log/squid/cache.log" 2015-3-21
"/var/log/messages" 2016-4-3
"/var/log/stunnel4/stunnel.log" 2012-9-19
"/var/log/apache2/php_error.log" 2012-10-21
"/var/log/ConsoleKit/history" 2016-4-1
"/var/log/rsnapshot.log" 2013-4-15
"/var/log/iptraf/*.log" 2012-9-12
"/var/log/snort/alert" 2012-10-17
"/var/log/privoxy/logfile" 2016-4-3
"/var/log/auth.log" 2016-4-3
"/var/log/postgresql/postgresql-8.4-main.log" 2012-10-21
"/var/log/apt/history.log" 2016-4-1
"/var/log/pm-powersave.log" 2012-11-1
"/var/log/proftpd/proftpd.log" 2016-4-3
"/var/log/proftpd/xferlog" 2016-4-1
"/var/log/zabbix-agent/zabbix_agentd.log" 2016-3-25
"/var/log/alternatives.log" 2016-4-7
"/var/log/mail.log" 2016-4-3
"/var/log/kern.log" 2016-4-3
"/var/log/privoxy/errorfile" 2013-5-28
"/var/log/aptitude" 2015-5-6
"/var/log/apache2/access.log" 2016-4-3
"/var/log/wtmp" 2016-4-1
"/var/log/pm-suspend.log" 2012-9-20
"/var/log/snort/portscan2.log" 2012-9-12
"/var/log/mail.warn" 2016-4-3
"/var/log/bacula/log" 2013-5-1
"/var/log/lpr.log" 2012-12-12
"/var/log/mail.err" 2016-4-3
"/var/log/tor/log" 2016-4-9
"/var/log/fail2ban.log" 2016-4-3
"/var/log/exim4/paniclog" 2012-9-12
"/var/log/tinyproxy/tinyproxy.log" 2015-3-25
"/var/log/munin/munin-limits.log" 2016-4-9
"/var/log/proftpd/controls.log" 2012-9-19
"/var/log/proftpd/xferreport" 2012-9-19
"/var/spool/qscan/qmail-queue.log" 2013-5-15
"/var/log/user.log" 2016-4-3
"/var/log/apache2/error.log" 2016-4-3
"/var/log/exim4/mainlog" 2012-10-16
"/var/log/privoxy/jarfile" 2013-5-28
"/var/log/cron.log" 2016-4-3
"/var/log/clamav/clamav.log" 2016-4-3

 

The timestamp date next to each of the rotated service log is when the respective log was last rorated

It is also a handy thing to rorate only a certain service log, lets say clamav-server, mysql-server, apache2 and nginx
 


logrorate /etc/logrorate.d/clamav-server
logrorate /etc/logrorate.d/mysql-server
logrotate /etc/logrotate.d/nginx

Must have software on freshly installed windows – Essential Software after fresh Windows install

Friday, March 18th, 2016

Install-update-multiple-programs-applications-at-once-using-ninite

If you're into IT industry even if you don't like installing frequently Windows or you're completely Linux / BSD user, you will certainly have a lot of friends which will want help from you to re-install or fix their Windows 7 / 8 / 10 OS. At least this is the case with me every year, I'm kinda of obliged to install fresh windowses on new bought friends or relatives notebooks / desktop PCs.

Of course according to for whom the new Windows OS installed the preferrences of necessery software varies, however more or less there is sort of standard list of Windows Software which is used daily by most of Avarage Computer user, such as:
 

I tend to install on New Windows installs and thus I have more or less systematized the process.

I try to usually stick to free software where possible for each of the above categories as a Free Software enthusiast and luckily nowadays there is a lot of non-priprietary or at least free as in beer software available out there.

For Windows sysadmins or College and other public institutions networks including multiple of Windows Computers which are not inside a domain and also for people in computer repair shops where daily dozens of windows pre-installs or a set of software Automatic updates are  necessery make sure to take a look at Ninite

ninite-automate-windows-program-deploy-and-update-on-new-windows-os-openoffice-screenshot

As official website introduces Ninite:

Ninite – Install and Update All Your Programs at Once

Of course as Ninite is used by organizations as NASA, Harvard Medical School etc. it is likely the tool might reports your installed list of Windows software and various other Win PC statistical data to Ninite developers and most likely NSA, but this probably doesn't much matter as this is probably by the moment you choose to have installed a Windows OS on your PC.

ninite-choises-to-build-an-install-package-with-useful-essential-windows-software-screenshot
 

For Windows System Administrators managing small and middle sized network PCs that are not inside a Domain Controller, Ninite could definitely save hours and at cases even days of boring install and maintainance work. HP Enterprise or HP Inc. Employees or ex-employees would definitely love Ninite, because what Ninite does is pretty much like the well known HP Internal Tool PC COE.

Ninite could also prepare an installer containing multiple applications based on the choice on Ninite's website, so that's also a great thing especially if you need to deploy a different type of Users PCs (Scientific / Gamers / Working etc.)

Perhaps there are also other useful things to install on a new fresh Windows installations, if you're using something I'm missing let me know in comments.

How to Remove / Add SuSE Linux start service command

Thursday, July 2nd, 2015

opensuse-remove-add-new-service-geeko-suse-linux-mini-logo
If you happen to administer SUSE LINUX Enterprise Server 9 (x86_64) and you need to add or remove already existing /etc/init.d script or custom created Apache / Tomcat .. etc. service and you're already familiar with Fedora's / RHEL chkconfig, then the good news chkconfig is also available on SuSE and you can use in same way chkconfig to start / stop / enable / disable boot time services.

To list all available boot time init.d services use:
 

suse-linux:/etc # chkconfig –list

 

SuSEfirewall2_final       0:off  1:off  2:off  3:off  4:off  5:off  6:off
SuSEfirewall2_init        0:off  1:off  2:off  3:off  4:off  5:off  6:off
SuSEfirewall2_setup       0:off  1:off  2:off  3:off  4:off  5:off  6:off
Tivoli_lcfd1.bkp          0:off  1:off  2:off  3:off  4:off  5:off  6:off
activate_web_all          0:off  1:off  2:off  3:on   4:off  5:on   6:off
alsasound                 0:off  1:off  2:on   3:on   4:off  5:on   6:off
apache2                   0:off  1:off  2:off  3:off  4:off  5:off  6:off
apache2-eis               0:off  1:off  2:off  3:on   4:off  5:off  6:off
atd                       0:off  1:off  2:off  3:off  4:off  5:off  6:off
audit                     0:off  1:off  2:off  3:off  4:off  5:off  6:off
autofs                    0:off  1:off  2:off  3:off  4:off  5:off  6:off
autoyast                  0:off  1:off  2:off  3:off  4:off  5:off  6:off
boot.clock                0:off  1:off  2:off  3:off  4:off  5:off  6:off
boot.crypto               0:off  1:off  2:off  3:off  4:off  5:off  6:off
boot.device-mapper        0:off  1:off  2:off  3:off  4:off  5:off  6:off
boot.evms                 0:off  1:off  2:off  3:off  4:off  5:off  6:off
boot.idedma               0:off  1:off  2:off  3:off  4:off  5:off  6:off
boot.ipconfig             0:off  1:off  2:off  3:off  4:off  5:off  6:off
boot.isapnp               0:off  1:off  2:off  3:off  4:off  5:off  6:off
boot.klog                 0:off  1:off  2:off  3:off  4:off  5:off  6:off
boot.ldconfig             0:off  1:off  2:off  3:off  4:off  5:off  6:off
boot.loadmodules          0:off  1:off  2:off  3:off  4:off  5:off  6:off
boot.localfs              0:off  1:off  2:off  3:off  4:off  5:off  6:off
boot.localnet             0:off  1:off  2:off  3:off  4:off  5:off  6:off
boot.lvm                  0:off  1:off  2:off  3:off  4:off  5:off  6:off
boot.md                   0:off  1:off  2:off  3:off  4:off  5:off  6:off
boot.multipath            0:off  1:off  2:off  3:off  4:off  5:off  6:off
boot.proc                 0:off  1:off  2:off  3:off  4:off  5:off  6:off
boot.restore_permissions  0:off  1:off  2:off  3:off  4:off  5:off  6:off
boot.rootfsck             0:off  1:off  2:off  3:off  4:off  5:off  6:off
boot.sched                0:off  1:off  2:off  3:off  4:off  5:off  6:off
boot.scpm                 0:off  1:off  2:off  3:off  4:off  5:off  6:off
boot.scsidev              0:off  1:off  2:off  3:off  4:off  5:off  6:off
boot.shm                  0:off  1:off  2:off  3:off  4:off  5:off  6:off
boot.swap                 0:off  1:off  2:off  3:off  4:off  5:off  6:off
boot.sysctl               0:off  1:off  2:off  3:off  4:off  5:off  6:off
boot.udev                 0:off  1:off  2:off  3:off  4:off  5:off  6:off
coldplug                  0:off  1:on   2:on   3:on   4:off  5:on   6:off

 

To then stop the service:
 

suse-linux:/etc # chkconfig gtiweb off


If you prefer to do it the SuSE way and learn a bit more on SuSE boot time process check out:

 

suse-linux:/etc # man insserv


Removing already existing SuSE start-up script from init.d start up with insserv is done with:

suse-linux:/etc # cd /etc/init.d/
suse-linux:etc/init.d # insserv -r gtiweb
insserv: script ipmi.hp: service ipmidrv already provided!
insserv: script boot.multipath.2008-10-29: service boot.multipath already provided!


To install a new custom written and placed into /etc/inti.d/ on SuSE's server boot time with insserv:

 

suse-linux:/etc/init.d/ # insserv your_custom_script_name

Windows how to check which process locks file command – A M$ Windows equivalent of lsof command

Monday, February 23rd, 2015

windows-how-to-check-which-process-locks-file-command-a-ms-windows-equivalent-of-lsof-command

I've had a task today to deploy a new WAR (Web Application Archive) Tomcat file on Apache Tomcat server running  on Windows server 2008 R2 UAT environment.
The client Tomcat application within war is providing a frontend to an proprietary Risk Analysis application called Risiko Management (developed by a German vendor called Schleupen).
The update of WAR file was part of a version upgrade of application so, both "Risk Analysis" desktop standalone server RiskKit and the Web frontend was developed by Schleupen had to be updated.
In order to update I followed the usual .WAR Tomcat Javafile upadate Tomcat process.

1. Stopped Tomcat running service Instance via services.msc command e.g.
 

Start (menu) -> Run
 

services.msc

 

stopping-tomcat-application-howto-stop-service-ms-windows-screenshot
 


2. Move (by Renaming) old risk-analysis.war to risk-analysis_backup_2015.war

and also rename the automatically Tomcat extracted folder (named same name as the WAR archive file directory – D:\web\Apache-Tomcat-7.0.33\webapps\Risiko-Analysis\ to :\web\Apache-Tomcat-7.0.33\webapps\Risiko-Analysis_backup_2015, i.e. run:
 

C:\Users\risk-analysis> D:
D:\>
D:\> CD \Web\Apache-Tomcat-7.0.33\webapps\

D:\Web\Apache-Tomcat-7.0.33\webapps> move risk-analysis.war risk-analysis_2015.war
D:\Web\Apache-Tomcat-7.0.33\webapps> move  
Risiko-Analysis\  Risiko-Analysis_backup_2015\


But unfortunately I couldn't rename it and I got below error:

move-windows-command-access-is-denied-tiny-screenshot

Also I tried copying it using Windows Explorer Copy / Paste but this didn't worked either, and I got below error :

cant-move-risk-analysis-tomcat-java-application-error-ms-windows-screenshot

3. Finding what Locks a directory or File on M$ Windows


Obviously, the reason for unable to copy the directory was something was locking it. Actually there are plenty of locked files many running applications like Explorer do. A good example for all time locked file is Windows (swap file) pagefile.sys – this is Windows Linux equivalent of swap filesystem (enabled / disabled with spapon / swapoff commands)

Having the directory locked was a strange problem, because the Tomcat process was not running as I checked closely both in Windows taskmgr GUI interface and manually grepped for the process with tasklist command like so:

 

d:\>tasklist /m|find /i "tomcat"


tomcat7.exe                   4396 ntdll.dll, kernel32.dll, KERNELBASE.dll,

For people like me who use primary Linux , above command shows you very precious debugging information, it shows which Windows libraries (DLL) are loaded in memory and used by the process 

 

(Note that when Tomcat is running, it is visible with command)
 

D:\> wmic.exe process list brief | find /i "tomcat"
526          tomcat7.exe          8         4396       49           156569600


Just for those wondering the 156569600 number is number of bytes loaded in Windows memory used by Tomcat.

After tomcat was stopped above command returned empty string meaning obviously that tomcat is stopped ..

BTW, wmic command is very useful to get a list of process names (to list all running processes):

 D:> wmic.exe process list brief

get-all-process-names-in-command-line-with-windows-wmic-command-screenshot

Well obviously something was locking this directory (some of its subdirectories or a file name within the directory / folder), so I couldn't rename it just like that.
In Linux finding which daemon (service) is locking a file is pretty easy with lsof command (for those new to lsof check my previous article how to how to check what process listens on network port in Linux), however it was unknown to me how I can check which running service is locking a file and did a quick google search which pointed me to the famous handle part of SysInternals tools.
The command tool Handle.exe was exactly what I was looking for. 

handle-sysinternals-tool-to-windows-see-all-locked-files-and-what-is-locking-them-ms-windows-screenshot

To get list of all opened (locked) files and see which application has opened it just exec command without arguments, you will get
plenty of useful info which will help you to better understand what Windows OS is doing invisible in the background and what app uses what.

handle-command-part-of-sysinternals-witout-any-arguments-display-opened-locked-files-in-windows

handle is pretty much Windows equivalent command of Linux lsof

To get which file was locked by Tomcat I used handle in conjuntion with find /i command which is pretty much like Linux's grep equivalent

 

C:\TEMP> Handle.exe | FIND /I "Tomcat"
   1C: File  (RW-)   D:\Web\Apache-Tomcat-7.0.33\webapps\Risk-Analysis\images\app


Alternatively if you have sysinternals and prefer GUI environment you can use SysInternals Process Explorer (press CTRL + F) and look for a string:

process-explorer-toolbar-find-what-is-locking-a-file-or-directory-windows

Next to handle I found also another GUI program (Internet Explorer extension) WhoLockMe, that can be used to show you all running programs and locked files by this programs.
WhoLockMe is pretty straight forward to use, though it shows GUI output you have to run the command from cmd line. Below is sample output screenshot of wholockme.


who-lock-me-windows-screenshot-see-which-files-running-programs-are-locking-on-ms-windows

 

To Install Wholockme 


Unzip "WhoLockMe.zip" in a directory (for exemple : "C:\Program Files\WhoLockMe")
Launch "Install.bat" or execute this Windows registry modification command :
 

regsvr32 "C:\Program Files\WhoLockMe\WhoLockMe.dll"


To Uninstall WhoLockMe – if you need to later:

 

Execute command :
 

regsvr32 /u "C:\Program Files\WhoLockMe\WhoLockMe.dll"


Reboot (Or Kill Explorer.exe).

Removes the "C:\Program Files\WhoLockMe" directory and its contents.

Probably there are other ways to find out what is locking a file or direcotry using powershell scripts or .bat (batch) scripting. If you know of other way using default Windows embedded commands, please share in comments.

 

How much memory users uses in GNU / Linux and FreeBSD – Commands and Scripts to find user memory usage on Linux

Tuesday, February 17th, 2015

 

how-much-memory-users-use-in-gnu-linux-freebsd-command-to-find-and-show-ascending-descending-usage-of-system-memory-tux-memory-logo

 


If you have to administrate a heterogenous network with Linux and FreeBSD or other UNIX like OSes you should sooner or later need for scripting purposes to have a way to list how much memory separate users take up on your system. Listing memory usage per user is very helpful for admins who manager free-shells or for companies where you have developers, developing software directly on the server via ssh. Being able to check which process eats up most memory is essential for every UNIX / Linux sysadmin, because often we as admins setup (daemons) on servers and we forgot about their existence, just to remember they exist 2 years later and see the server is crashing because of memory exhaustion. Tracking server bottlenecks where RAM memory and Swapping is the bottleneck is among the main swiss amry knives of admins. Checking which user occupies all server memory is among the routine tasks we're forced to do as admins, but because nowdays servers have a lot of memory and we put on servers often much more memory than ever will be used many admins forget to routinely track users / daemons memory consumption or even many probably doesn't know how.  Probably all are aware of the easiest wy to get list of all users memory in console non interactively with free command, e.g.:
 

free -m
             total       used       free     shared    buffers     cached
Mem:         32236      26226       6010          0        983       8430
-/+ buffers/cache:      16812      15424
Swap:        62959        234      62725

 

but unfortunately free command only shows overall situation with memory and doesn't divide memory usage by user

Thus probably to track memory users the only known way for most pepole is to (interactively) use good old top command or if you like modern (colorful) visualization with htop:

debian:~# top

 

linux-check_memory_usage_by_logged-in-user-with-top-process-command-gnu-linux-freebsd-screenshot

Once top runs interactive press 'm' to get ordered list of processes which occupy most system memory on Linux server.Top process use status statistics will refresh by default every '3.0' seconds to change that behavior to '1' second press  s and type '1.0'. To get Sort by Memory Use in htop also press 'm'
 

[root@mail-server ~]# htop


htop_show_users_memory_usage_order_ascending-gnu-linux-screenshot

 

However if you need to be involved in scripting and setting as a cron job tasks to be performed in case if high memroy consumption by a service you will need to use few lines of code. Below are few examples on how Linux user memory usage can be shown with ps cmd.

Probably the most universal way to see memory usage by users on Debian / Ubuntu / CentOS / RHEL and BSDs (FreeBSD / NetBSD) is with below one liner:

 

server:~# ps hax -o rss,user | awk '{a[$2]+=$1;}END{for(i in a)print i” “int(a[i]/1024+0.5);}' | sort -rnk2
daemon 0
debian-tor 63
dnscache 1
dnslog 0
hipo 21
messagebus 1
mysql 268
ntp 2
privoxy 1
proftpd 1
qmaill 0
qmailq 0
qmailr 0
qmails 0
qscand 291
root 94
shellinabox 1
snmp 1
statd 1
vpopmail 80
www-data 6765

 

Output is in MBs

Below is output from machine where this blog is running, the system runs ( Apache + PHP + MySQL Webserver + Qmail Mail server and Tor) on Debian GNU / Linux.

 To get more human readable (but obscure to type – useful for scripting) output list of which user takes how much memory use on deb / rpm etc. based Linux :

 

server:~# echo "USER                 RSS      PROCS" ; echo "——————– ——– —–" ; \
ps hax -o rss,user | awk '{rss[$2]+=$1;procs[$2]+=1;}END{for(user in rss) printf “%-20s %8.0f %5.0f\n”, user, rss[user]/1024, procs[user];}' | sort -rnk2

 

USER                 RSS      PROCS
——————– ——– —–
www-data                 6918   100
qscand                    291     2
mysql                     273     1
root                       95   120
vpopmail                   81     4
debian-tor                 63     1
hipo                       21    15
ntp                         2     1
statd                       1     1
snmp                        1     1
shellinabox                 1     2
proftpd                     1     1
privoxy                     1     1
messagebus                  1     1
dnscache                    1     1
qmails                      0     2
qmailr                      0     1
qmailq                      0     2
qmaill                      0     4
dnslog                      0     1
daemon                      0     2

 

It is possible to get the list of memory usage listed in percentage proportion, with a tiny for bash loop and some awk + process list command
 

TOTAL=$(free | awk '/Mem:/ { print $2 }')
for USER in $(ps haux | awk '{print $1}' | sort -u)
do
    ps hux -U $USER | awk -v user=$USER -v total=$TOTAL '{ sum += $6 } END { printf "%s %.2f\n", user, sum / total * 100; }'
done

107 1.34
115 2.10
119 1.34
daemon 1.32
dnscache 1.34
dnslog 1.32
hipo 1.59
mysql 4.79
ntp 1.34
privoxy 1.33
proftpd 1.32
qmaill 1.33
qmailq 1.33
qmailr 1.32
qmails 1.33
qscand 4.98
root 1.33
snmp 1.33
statd 1.33
vpopmail 2.35
www-data 86.48

Also a raw script which can be easily extended to give you some custom information on memory use by user list_memory_use_by_user.sh is here.
You can also want to debug further how much memory a certain users (lets say user mysql and my username hipo) is allocating, this can easily be achieved ps like so:
 

root@pcfreak:~# ps -o size,pid,user,command -u mysql –sort -size
 SIZE   PID USER     COMMAND
796924 14857 mysql   /usr/sbin/mysqld –basedir=/usr –datadir=/var/lib/mysql –plugin-dir=/usr/lib/mysql/plugin –user=mysql –pid-file=/var/run/mysqld/mysqld.pid –socket=/var/run/mysqld/mysqld.sock –port=3306

 

root@pcfreak~# ps -o size,pid,user,command -u hipo –sort -size|less
 SIZE   PID USER     COMMAND
13408 19063 hipo     irssi
 3168 19020 hipo     SCREEN
 2940  2490 hipo     -bash
 1844 19021 hipo     /bin/bash
 1844 19028 hipo     /bin/bash
 1844 19035 hipo     /bin/bash
 1844 19042 hipo     /bin/bash
 1844 19491 hipo     /bin/bash
 1844 22952 hipo     /bin/bash
  744  2487 hipo     sshd: hipo@pts/0
  744  2516 hipo     sshd: hipo@notty
  524  2519 hipo     screen -r
  412  2518 hipo     /usr/lib/openssh/sftp-server

You see from below output user running with www-data (this is Apache Webserver user in Debian) is eating 86.48% of overall system memory and MySQL server user is using only 4.79% of available memory

Output is shown in Megabytes per username memory usage, and user memory usage is ordered (stepping-down / descentive) from top to bottom

Getting more thoroughful and easier to read reporting without beeing a 31337 bash coder you can install and use on Linux smem – memory reporting tool .

SMEM can provide you with following memory info:

  • system overview listing
  • listings by process, mapping, user
  • filtering by process, mapping, or user
  • configurable columns from multiple data sources
  • configurable output units and percentages
  • configurable headers and totals
  • reading live data from /proc
  • reading data snapshots from directory mirrors or compressed tarballs
  • lightweight capture tool for embedded systems
  • built-in chart generation


Installing smem on Debian 6 / 7 / Ubuntu 14.04 / Turnkey Linux etc. servers is done with standard:

 

debian:~# apt-get install –yes smem
….

 

 

To install smem on CentOS 6 / 7:

 

[root@centos ~ ]# yum -y install smem
….


On Slackware and other Linux-es where smem is not available as a package you can install it easily from binary archive with:

 

cd /tmp/
wget http://www.selenic.com/smem/download/smem-1.3.tar.gz
tar xvf smem-1.3.tar.gz
sudo cp /tmp/smem-1.3/smem /usr/local/bin/
sudo chmod +x /usr/local/bin/smem

 


Two most common smem uses are:

 

root@mail:~# smem -u
User     Count     Swap      USS      PSS      RSS
dnslog       1       44       48       54      148
qmaill       4      232      124      145      464
hipo        11    13552     8596     9171    13160
qscand       2     4500   295336   295602   297508
root       188   217312  4521080  4568699  7712776

 

Below command shows (-u – Report memory usage by user, -t – show totals, -k – show unix suffixes)

root@mail:~# smem -u -t -k
User     Count     Swap      USS      PSS      RSS
dnslog       1    44.0K    48.0K    54.0K   148.0K
qmaill       4   232.0K   124.0K   145.0K   464.0K
hipo        11    13.2M     8.4M     9.0M    12.9M
qscand       2     4.4M   288.4M   288.7M   290.5M
root       188   212.2M     4.3G     4.4G     7.4G
—————————————————
           206   230.1M     4.6G     4.6G     7.7G


To get users memory use by percentage with smem:
 

root@mail:~# smem -u -p
User     Count     Swap      USS      PSS      RSS
dnslog       1    0.00%    0.00%    0.00%    0.00%
qmaill       4    0.00%    0.00%    0.00%    0.01%
hipo        11    0.17%    0.11%    0.11%    0.16%
qscand       2    0.05%    3.63%    3.63%    3.66%
root       194    2.64%   56.18%   56.77%   95.56%

It is also useful sometimes when you want to debug system overloads caused by external hardware drivers loaded into kernel causing issues to get list of system wide memory use sorted by user

 

 root@mail:~# smem -w -p
Area                           Used      Cache   Noncache
firmware/hardware             0.00%      0.00%      0.00%
kernel image                  0.00%      0.00%      0.00%
kernel dynamic memory        38.30%     36.01%      2.28%
userspace memory             60.50%      0.98%     59.53%
free memory                   1.20%      1.20%      0.00%


smem is very nice as if you're running it on a Desktop Linux system with Xserver installed you can see also graphical output of memory use by application:
 

root@desktop-pc:~# smem –bar pid -c "pss uss"


smem_graphical_representation-of-which-user-application-is-consuming-most-memory-gnu-linux-kde-screenshot-smem-command-line-tool

smem can even generate graphical pie charts to visualize better memory use
 

root@desktop-pc:~# smem -P '^k' –pie=name

 

generate-graphical-staticstics-linux-memory-use-by-pie-chart

If there is a high percentage shown in firmware/hardware this means some buggy module is loaded in kernel eating up memory, to fix it debug further and remove the problematic module.
userspace memory actually shows the percantage of memory out of all server available RAM that is being consumed by applications (non kernel and other system processes which make the system move). You see in above example the kernel itself is consuming about 40% of system overall available memory. 

We all know the SWAP field stands for hard disk drive used as a memory when system is out, but there are 3 fields which smem will report which will be probably unclear for most here is also explanation on what USS / PSS / RSS means?

RSS is the Resident Set Size and is used to show how much memory is allocated to that process and is in RAM. It does not include memory that is swapped out. It does include memory from shared libraries as long as the pages from those libraries are actually in memory. It does include all stack and heap memory too.

There is also PSS (proportional set size). This is a newer measure which tracks the shared memory as a proportion used by the current process. So if there were two processes using the same shared library from before.

USS stands for Unique set size, USS is just the unshared page count, i.e. memory returned when process is killed 

PSS = Proportional set size, (PSS),  is a more meaningful representation of the amount of memory used by libraries and applications in a virtual memory system.  
Because large portions of physical memory are typically shared among multiple applications, the standard measure of memory usage known as resident set size (RSS) will significantly overestimate memory usage. The parameter PSS instead measures each application’s “fair share” of each shared area to give a realistic measure. For most admins checking out the output from RSS (output) should be enough, it will indicate which user and therefore which daemon is eating up all your memory and will help you to catch problematic services which are cause your server to run out of RAM and start swapping to disk.

xkill for Windows – Kill hanged programs with one click like in UNIX

Thursday, September 18th, 2014

Windows Xkill by Solo Dev logo kill easily hanged, crashed windows programs - Linux xkill for Windows alternative

I’ve used Linux as I’ve used it over the last 10 years and thus I’m so used to xkill that I consider it as a normal tool every modern operating system should have.

Since I’m forced to work on a Windows platform over the last 1 year, every now and then I have crashing / hanged apps Window which sometimes is hard to kill using Task Manager or command line tools like tasklist, taskkill or pstools – Windows Sysinternal tools cause you don’t know the exact process name of the Windows crashed application.

Thanksfully a good hearted guy SuprVillain made Windows Xkill program. Windows Xkill is portable app, so you don’t need to install it but simply download and run it.

windows-7-system-tray-xkill-screenshot-xkill-for-windows

Win-Xkill runs in system tray and has “kill mode” of operation, when Kill mode it is running Windowns Xkill operates exactly like UNIX’s xkill. The mouse pointer turns into Skull and Cross Bones, you point at a Window app you wanted to kill and it gets terminated.
Windows Xkill kill mode enabled killing a windows notepad app
Win Xkill kill mode, can be invoked using also a key press Control+Alt+Backspace as well as there is an option to leave xkill running in the background background, but disabling the tiny skull icon in systray from its interface.
Enjoy 🙂

Cannot Delete file on Desktop on Windows XP – The process cannot access the file because it is being used by another process.

Wednesday, August 13th, 2014

delete-locked-file-in-windows-xp-7-cannot-delete-file-on-desktop-windows-xp-file-used-by-another-process-solution-fix
I had to fix recently another Windows XP infected with plenty of malicious malware. I've used tips from http://malwaretips.com/blogs/ and launch all suggested tools to cleanup the PC. Some few thousands of infections were cleaned and the PC started working much better than earlier, however still there was one weird issue on the Windows XP desktop there was an .exe file hanging with 0 bytes size and whenever I tried to delete the file either from GUI or command line with (del command) it refused to delete with error:
 

The process cannot access the file because it is being used by another process.


Well the message clearly shows, the file can't be deleted because some program was using it however I couldn't see any program that has locked the .exe file. I've checked that in (Windows Task Manager) run from

Start -> Run:

taskmgr


I've done a quick search online to see, whether someone has fixed the issues, and saw many threads talking about the issue suggesting that the issue got solved Windows XP OS re-install, as this was not an option for me because the PC's reinstall would be at least a 2 day time work, plus it contains some programs which will probably be hard to freshly install, because they're no longed supported.

Finally I've seen in this thread suggestion to try to kill Windows Explorer after you have opened windows with windows cmd.exe because this will prevent the .exe file being locked by Windows Explorer and will allow it to be deleted with del command. This actually worked. I've launched

Start -> Run

taskmgr

windows-kill-explorer-process-to-delete-explorer-locked-file-on-windows-xp-desktop

and

cmd.exe

Found explorer.exe process and killed it, this makes the whole desktop (icons, start menu etc. disappear) – as Windows Explorer is main managing Windows app. Then using Alt+Tab to switch to Command Prompt window deleted it with:
 

del  Process_that_cannot_be_killed.exe


And finally succesfully deleted it 🙂