Posts Tagged ‘need’

Howto check if mod_rewrite is installed and loaded in Apache 2.x and serving directories correctly

Wednesday, February 10th, 2010

During my professional experience as a system administrator, it was a common misfortune in newly
configured unix servers mod_rewrite not to be serving .htaccess files.
Everytime I had to fix issue in which mod_rewrite was not working I loosed a lot of my precious time
The same story happened once again I had to check why mod_rewrite is not configured correctly and I cannot
apply some redirection rules in WordPress necessery for me
to exercise some SEO on my wordpress blog .
Therefore this time I decided to blog a step by step check on in order to:

1. Determine if mod_rewrite is installed & loaded correctly in Apache
2. Find out if mod_rewrite is configured to serve .htaccess files correctly
Going through this two scenarios I would be able to determine why I cannot get wordpress SEO optimization
mod_rewrite redirection rules to work as they should.
Okay, so let’s go through the two scenarios:

1. Here is a tiny script in PHP to determine if mod_rewrite is installed and loaded in Apache correctly
To use the script save it somewhere in your Domain document root with a .php extension
Here is the code of the script as well:

< ?phpprint_r(apache_get_modules());?>

If your mod_rewrite module is configured correctly you’ll see it in the php array containing
all your apache loaded modules.

Without PHP from the shell in order to get installed Apache modules on Linux, the following command applies:

apache2ctl -t -D DUMP_MODULES

For apache 2.2.x
httpd (or apache.exe) -M will show static and shared modules loaded via LoadModule.
httpd -l will show only static (compiled in) modules

For apache 2.0.x
httpd (apache.exe) -l is available but -M is not.
You will need to manually check the LoadModule directives and the files.

2. Now we continue, further on with another script that has to be installed somewhere in Apache’s DocumentRoot
I decided to install it in a directory /test/ so in my case it’s installed in www.pc-freak.net/test/
Here is a link to the script you need to find out if mod_rewrite is configured to serve .htaccess files for your preferred domain name’s DocumentRoot.

Now save this file and again make sure it has a .php extension.

Now you need to create an .htaccess file in the directory where you have rewrite.php file stored.
The content of the .htaccess file should be the following:

RewriteEngine OnRewriteOptions InheritRewriteRule ^link([^/]*).html$ rewrite.php?link=$1 [L]

Make sure the permissions of the .htaccess are correct, for example the file should have at least read
permissions for your webserver.
So let’s execute:

chmod a+r .htaccess to ensure the newly created file has proper permissions
So let’s check if mod_rewrite is enabled for my domain DocumentRoot by visiting:
this link
Hopefully if all is okay you’ll see:
Congratulations!! You are using Apache mod_rewrite whenever you press the TEST2 link
on the upmentioned webpage.

In case if you get a 404 error message, then most probably you don’t have mod_rewrite configured
to serve .htaccess files for your domain.
This often happens because of missing:
AllowOverride All in your main Directory directives in your webserver configuration file.
In my case I had a problem where mod_rewrite rules were not red from the .htaccess file:
To solve my issue I had to change:

Directory /
AllowOverride None
Order Deny,Allow
Deny from all
/Directory
in my httpd.conf to

Directory /
AllowOverride All
Order Deny,Allow
Deny from all
/Directory

So well, That’s all Folks!

12 must have Joomla extension plugins / Essential modules for new Joomla CMS install

Thursday, June 16th, 2011

Joomla bundle of must have extensions picture

These days very often I have to install, plain new Joomla based websites. I’ve realized that since there is no structured guide to follow describing the most essential plugins that every new fresh new joomla installation is required to have.
Thus I took the time and wrote this post, as it will be useful to myself in my future new joomla based websites establishment, I also believe these guide will be useful to other Joomla enthusiasts or administrators in their daily work.

Below I will describe in short the installation, configuration and oddities I’ve faced during installment of the above described bundle of plugins on a plan Joomla 1.5 install.These article will walk through 12 joomla essential plugins that I believe every fresh Joomla installation should be equipped with.
Hope this guide will be helpful to you. Now let’s start it up:

1. JoomlaXplorer – A sophisticated web file explorer for Joomla

One of the basic modules, beneficial with a new joomla CMS install is Joomla Xptplorer . This module enables the joomla admin to browse files in a web file explorer, on the server where the joomla CMS is installed. Below you see how handy the joomla web explorer provided by the plugin is:

Joomla file explorer extplorer module

Installing and using the plugin is a piece of cake. To install the plugin:

a. download Joomla file Xplorer from here or from the official plugin website.

b. Install the plugin through the admin joomla menu:

Extensions -> Install/Uninstall

c. Start using the newly installed plugin by following to menus:

Components -> eXtplorer

2. JCrawler Generate easily sitemap.xml to aim the overall Joomla website SEO optimization

JCrawler logo plugin joomla

I have previously written a very through tutorial on how to install configure and generate website sitemap with Jcrawler module, You can read my article titled: How to build website sitemap.xml in Joomla here

3. sh404SEF – Make your Joomla links and content more user friendly

sh404sef Joomla Search Engine Optimization plugin

sh404SEF is a great Joomla plugin, which will seriously improve SEO and could contribute well for a website to be better indexed with major search engines.

I have previously written an article describing thoroughfully the install and use procedures for the module.
You can read the article Making your Joomla URLS Google friendly with sh404sef plugin / Simple Joomla link SEO here

4. Akeeba Backup Joomla solution

Akeeba backup Joomla Module

Installing a joomla backup solution is very essential if you does take care about your data, it often happens that server hard disk got crashes or a RAID massives got corrupted or some kind of other unexpected disaster hits the server. In these terrible times, having a website backup will save you nerves and data recovery funds, not to mention that in many cases data recovery is impossible.

Joomla has a very easy to use software for creating full website backup called Akeeba Backup

To start using the software one must:

a. Download Akeeba Backup and install it via:

Extensions -> Install/Uninstall

After the Akeeba Backup installation is over, to create your first backup, one needs to navigate to:

Components -> Akeeba Backup -> Backup Now

Each Akeeba backup (a version of the website’s files data and sql info) will be created in an archive file with the extension .jpa
The backup files are created under joomla’s website (main) root directory in directory location administrator/components/com_akeeba/backup

The Akeeba plugin also has capabilities to recover a (.jpa) backup restore point easily.
To recover a backup with the Akeeba plugin one needs to do it once again, via the plugin joomla web interface.
5. Google Analytics (place easy tracking code) in Joomla

Google Analytics Tracking Module for Joomla

a. Download the Google Analytics Tracking Module
The module is available from Joomla Extensions on joomla.org

At the current time of writting you need to download the analytics_tracking15,zip file

b. Install the Google Analytics Tracking Module;
Login as joomla admin and navigate to;

Extensions -> Install

Place the analytics_tracking15.zip url to the Install URL: field.
Again as of time of writting you need to place https://www.pc-freak.net/files/analytics_tracking15.zip;

c. Open the Module Manager

Extensions -> Module Manager

d. Click over Google Analytics Tracking Module
On the right pane you will notice in the Module Parameters the Analytics_uacct_code field. In the uacct_code field you need to paste your UA obtained from your created google-analytics account.
This code is usually something like UA-2101595-10
Now place your code their and press the save green button located near the right top of the screen. You should see the text in blue Item Saved which would indicate your UA code is stored already in the Google Analytics Tracking Module, now press the Cancel button located again on the right top.
As a last step before the Google analytics is set-up on the Joomla you need to enable the plugin to do you need to press over the tick left sided from the text reading Google Analytics Tracking Module in the Module Manager
e. Click over the Enabled button in Module Manager;
Google Analytics Tracking Module -> Enabled

6. Itprism (Facebook, Twitter etc.) Social Network share buttons Joomla plugin

ITPShare Large Social Buttons Joomla

a. to add the itpsocialbuttons to joomla Download the Itpsocialbuttons latest module files from http://itprism.com/free-joomla-extensions/social-buttons-plugin or use my mirrored module files below:

Download mod_itpsocialbuttons .zip file
Download plg_itpsocialbuttons .zip

After having the two files necessery to be installed to make the ITPSocialButtons appear on website, installation is done like with any other installation:

Extensions -> Install/Uninstall -> Upload Package File (Choose File)

Next its necessery to configure the plugin to do so, follow to menus:

Extensions -> Module Manager -> Share

You will notice the Share dialog in the list of Module Name column in Module Manager

Here is a screenshot on how the settings options for ITpsocialbuttons will look like:

Mod ITpSocialButtons settings screen

The options which I personally changed was:

Show Title – I set this one to No as I wanted to omit the plugin title text to appear on my website.
Further on I’ve set the Enabled option to Yes to enable the plugin and choose the buttons Style option to be of a Small buttons type.
I’ve also found that the most suitable position for the Social Share buttons were to be of a right – Position .

Here is how the social network itprism share buttons looks like:

ITprism Joomla Social Share Plugin various button types

The plugin supports sharing of joomla pages to the following list of social networks:
 

  • Delicios.com
  • Digg.com
  • Facebook
  • Google Bookmarks
  • StumbleUpon
  • Technorati
  • Twitter
  • LinkedIn

The module configuration, also allows the user to configure the type of social network buttons, one wants to display on the website.

7. Joomla JCE Content Editor

This content editor is really awesome compared to the default editor TinyMCE. If you want to have an options rich content editor for Joomla, then this is the one for you 🙂
download JCE Content Editor here

After installing the JCE Joomla content editor in order to enable it as a default editor you need to go to the following location:

Site -> Global Configuration -> Default WYSIWYG Editor

There place on Editor TinyMCE and change it with Editor – JCE

Now go to edit some article, and you will see the difference in the editor 🙂

8. Add gallery Joomla capabilities with sigplus (Image Gallery Plus) and Very Simple Image Gallery

Image Gallery Plus sigplus Joomla Screenshot

Image Gallery Plus plugin (sigplus) gallery review screenshot

sigplus Image Gallery Plus is a straightforward way to add image or photo galleries to a Joomla article with a simple syntax. It takes a matter of minutes to set up a gallery but those who are looking for a powerful gallery solution will not be disappointed either: sigplus is suitable for both beginner and advanced users.

Download Sigplus Joomla Image Gallery plugin here
Using sigllus is quite easy all one has to do is use Joomla Media Manager from links:

Site -> Media Manager

Create new folder in the stories folder, let’s say New Pictures and further on use Media Manager to upload all desired pictures to be later displayed.

Being done with uploading the images you want to display, go to Article Manager :

Content -> Article Manager

From there choose your article where new uploaded pictures you want to display and type in the article:

{gallery}New Pictures{/gallery}.

Note that New Pictures is the directory just recently created as stated below, it’s important that there is no spacing between {gallery}and New Pictures, if one tries {gallery} New Pictures {/gallery} instead of {gallery}New Pictures{/gallery} an error will occur instead of the pictures being displayed in a scrolled gallery.

Sigplus Image Gallery has also a number of configuration options, which might make it look a bit more decent.
I have to say in my view the default way sigplus displays pictures is awful!

Another alternative if you don’t like Sigplus ‘s way of creating new galleries is to use Very Simple Image Gallery

Very Simple Image Gallery joomla screenshot

Here is a screenshot on a sample gallery created with Very Simple Image Gallery Joomla Plugin

You can download Very simple image gallery here

After installing the plugin. It’s use is analogous to the Sigplus . To use it likewise sigplus create new directory through Media Manager and in stories and upload your files in let’s say New Pictures1 . Later on in your article place, the code

{vsig}New Pictures1{/vsig}

Gallery will be generated automatically by the plugin. I think Simple Image Gallery is a bit more advanced and gives a better outlook to Galleries, though it’s configuration settings are much less than with SigPlus image gallery.

To add pictures comments e.g. img link alt=” and title=” tib you need to place a code within the Article manager similar to:

{vsig_c}0|Picture_1.JPG|Some sample text|Some other text{/vsig_c}
{vsig_c}0|Picture_2.JPG|Some example text|Some text{/vsig_c}
etc. ..

Note that the 0 in above example specifies the gallery number if you for instance are using a couple of galleries with Simple Image Gallery , the first one you used would be call 0 . The text specified as comments to the picture will also appear after you preview the gallery right below the picture when clicked on as a picture description in a really nice way.
9. Install Google maps plugin for Joomla

Google Maps plugin for Joomla screenshot

It’s a wise idea that every website has a location map on it’s website, for that reason Google maps is just great.
To install Google maps capability to joomla one can use a plugin called Google Maps .

You can straighly download Joomla’s Googlemap plugin from here

Afterwards use Extension Manager to install the plugin e.g. follow:

Extensions -&gr; Install/Uninstall (Choose File)

and click on Upload File & Install button.

To further enable and configure the Joomla Googlemap plugin you will have to go to the location:

Extensions -> Plugin Manager

Therein you will have to find and enable the Google Maps plugin which is to be found in the column named Plugin Manager
On my Joomla installation the plugin was located in the second page with modules, so if you don’t find the module on the listing with modules on the first page, make sure you scroll to the bottom of the page and click on Next button.

Therein in the list you will most likely notice Google Maps use the Enable button to enable it.

Next step is to configure the plugin, to do so press on the plugin name Google Maps
All configuration necessery here is to place Googlemaps API Key in the respective field (you will see it among config options).

Issuing a new Google Maps api key takes just few seconds, if you already have a gmail account just go to http://code.google.com/apis/maps/signup.html and take few seconds to issue the key.

You will get the key right on your gmail account after being issued (to repeat myself issuing takes few seconds so no worrier here).

One moreOnce having the key place it in the Googlemaps API Key field and configuring Address (which is one of the list of many options the plugin provides) you will be done with configuration.

To display a google map the location you just configured go to the Article Manager , select the article where you want the google maps location picture of your address to appear and type in the Article:

{mosmap|text='Exact street address location'|zoom='15'|zoomType='Large'|zoomNew='0'}

After you save the article a very nice Google map showing you the location’s streets will appear.
You can further conifgure a number of things related to the google map to appear, one thing you might want to play with is the zoom option which as you see in below’s code is equal to 15, e.g. zoom=’15’
Set it to another one if you want to regulate your googlemaps zoom level. For more thoroughful options take a look at the extensive plugin documentation.

10. Joomla Xmap (generating static HTML sitemap) Download Xmap from here , install it the usual plugin way.

Right after installation on the plugin succesful install screen you will notice the link component menu .
Clicking on the component menu you will be leaded to a page showing you few links Sitemap’s URL :
 

  • XML Sitemap:
  • HTML Sitemap:
  • News Sitemap:
  • Images Sitemap:

11. Add Joomla donate Paypal capabilities with Joomla PAYPAL DONATION MODULE

Paypal Donation Module Joomla Screenshot

Just recently I’ve written a a post on how to add a paypal donation capabilities to joomla, you can read my previous post here

12. Install Joomla RSForms Module (Advanced Joomla Forms Support)

Simple Joomla RsForm contact form

If you’re planning to add a complicated form support for Joomla, there are plenty of plugins, however one that was suggested by a friend of mine which is deep in Joomla world and moreover works good on my joomla installations is RSForms

Joomla – RSForms! is free to download and has great and easy interface to create new joomla forms.

At the time of writting I use these three RSForms components on new Joomla installations:

RSform Pro 1.1.0 com
Mod RsForm for Joomla 1.5
Mod Rsform list for Joomla

For latest release of RSForms! use the link http://www.rsjoomla.com/joomla-components/rsform.html

Installation is like any other module and is done through Extensions -> Install/Uninstall menu.

After installation, setting up a new form is available from the Joomla Menus:

Components -> RSform!Pro -> Manage Forms

I would not enter in details on how to edit the default RSForms or create a new RSForm. Just take some time and learn it by trying 😉

After the rsform is ready, to enable the new form, navigate to Joomla menus:

Menus -> Main Menu

Press the New button located in the buttons bar nearby the page header in the list of options in Select Menu item Type you will notice the RSForm!Pro as an option, press on it to establish the new form in the menus.

A follow up window will appear where one can set a Title: and Alias: for the new form as well as few other options.
After finalizing the settings press on Apply button to save the settings and the new form should appear in Joomla.

Probably there are many more handy plugins, which I’m missing here thus I’ll be glad if readers suggest some more helpful essential (must have plugins) for Joomla.
Feedback on this tutorial is very welcome!
Looking forward to hear for your opinions if my article was helpful to you 😉

Adding Comments & Google search plugins to nanoblogger

Thursday, July 30th, 2009

Ever wondered how you can add comments option to nanoblogger just like any normal blogging software like wordpress do support? Cause I did, Today I wandered around in google a bit until I can make it.In order to make all this workable I choose the nbcom nanoblogger plugin.Here’s what led me to success.First prepare yourself to loose a couple of hours! It’s more complicated or at least it was more complicated than I expected to add a simple plugin as this.Second, You’ll need the following nbcom 1.1 archive .
Right after that you’ll need to untar it and rename hdrs.tmpl and consts.tmpl to *.php, edit them and remove all the unneeded “/” back slashes. Copy the content of nbcom-1.1 dir to the document root of your nanoblogger. Make sure to edit blog.conf and have all the necessary like BLOG_URL, BLOG_ADMIN=”session”, DB_PASS=”password_of_sql_user_session”, yes I forgot to mention that you’ll need to read the INSTALL file contained in the nbcom archive and follow the instructions for creation of user and database that nbcom would use in the future. There are described like 7+ files more you need to edit and if you’re lucky like I was you’ll have nanoblogger+comments support up and running! Hooray! Wish you Good luck.
Secondly probably you have wondered how to make nanoblogger has a normal search field just like any normal php based blogger software out there. It took me like an hour before I came to the enlightenment. First I had downloaded the google.sh script mirrored for you to make your life easier here .
Next copy the file to your nanoblogger plugins directory.
And last you have to edit the templates/main_index.htm file and add the content that is as commented in the google.sh file. If you’re lucky again you’ll have nanoblogger with this two valuable plugins working. EnjoyEND—–

How much memory users uses in GNU / Linux and FreeBSD – Commands and Scripts to find user memory usage on Linux

Tuesday, February 17th, 2015

 

how-much-memory-users-use-in-gnu-linux-freebsd-command-to-find-and-show-ascending-descending-usage-of-system-memory-tux-memory-logo

 


If you have to administrate a heterogenous network with Linux and FreeBSD or other UNIX like OSes you should sooner or later need for scripting purposes to have a way to list how much memory separate users take up on your system. Listing memory usage per user is very helpful for admins who manager free-shells or for companies where you have developers, developing software directly on the server via ssh. Being able to check which process eats up most memory is essential for every UNIX / Linux sysadmin, because often we as admins setup (daemons) on servers and we forgot about their existence, just to remember they exist 2 years later and see the server is crashing because of memory exhaustion. Tracking server bottlenecks where RAM memory and Swapping is the bottleneck is among the main swiss amry knives of admins. Checking which user occupies all server memory is among the routine tasks we're forced to do as admins, but because nowdays servers have a lot of memory and we put on servers often much more memory than ever will be used many admins forget to routinely track users / daemons memory consumption or even many probably doesn't know how.  Probably all are aware of the easiest wy to get list of all users memory in console non interactively with free command, e.g.:
 

free -m
             total       used       free     shared    buffers     cached
Mem:         32236      26226       6010          0        983       8430
-/+ buffers/cache:      16812      15424
Swap:        62959        234      62725

 

but unfortunately free command only shows overall situation with memory and doesn't divide memory usage by user

Thus probably to track memory users the only known way for most pepole is to (interactively) use good old top command or if you like modern (colorful) visualization with htop:

debian:~# top

 

linux-check_memory_usage_by_logged-in-user-with-top-process-command-gnu-linux-freebsd-screenshot

Once top runs interactive press 'm' to get ordered list of processes which occupy most system memory on Linux server.Top process use status statistics will refresh by default every '3.0' seconds to change that behavior to '1' second press  s and type '1.0'. To get Sort by Memory Use in htop also press 'm'
 

[root@mail-server ~]# htop


htop_show_users_memory_usage_order_ascending-gnu-linux-screenshot

 

However if you need to be involved in scripting and setting as a cron job tasks to be performed in case if high memroy consumption by a service you will need to use few lines of code. Below are few examples on how Linux user memory usage can be shown with ps cmd.

Probably the most universal way to see memory usage by users on Debian / Ubuntu / CentOS / RHEL and BSDs (FreeBSD / NetBSD) is with below one liner:

 

server:~# ps hax -o rss,user | awk '{a[$2]+=$1;}END{for(i in a)print i” “int(a[i]/1024+0.5);}' | sort -rnk2
daemon 0
debian-tor 63
dnscache 1
dnslog 0
hipo 21
messagebus 1
mysql 268
ntp 2
privoxy 1
proftpd 1
qmaill 0
qmailq 0
qmailr 0
qmails 0
qscand 291
root 94
shellinabox 1
snmp 1
statd 1
vpopmail 80
www-data 6765

 

Output is in MBs

Below is output from machine where this blog is running, the system runs ( Apache + PHP + MySQL Webserver + Qmail Mail server and Tor) on Debian GNU / Linux.

 To get more human readable (but obscure to type – useful for scripting) output list of which user takes how much memory use on deb / rpm etc. based Linux :

 

server:~# echo "USER                 RSS      PROCS" ; echo "——————– ——– —–" ; \
ps hax -o rss,user | awk '{rss[$2]+=$1;procs[$2]+=1;}END{for(user in rss) printf “%-20s %8.0f %5.0f\n”, user, rss[user]/1024, procs[user];}' | sort -rnk2

 

USER                 RSS      PROCS
——————– ——– —–
www-data                 6918   100
qscand                    291     2
mysql                     273     1
root                       95   120
vpopmail                   81     4
debian-tor                 63     1
hipo                       21    15
ntp                         2     1
statd                       1     1
snmp                        1     1
shellinabox                 1     2
proftpd                     1     1
privoxy                     1     1
messagebus                  1     1
dnscache                    1     1
qmails                      0     2
qmailr                      0     1
qmailq                      0     2
qmaill                      0     4
dnslog                      0     1
daemon                      0     2

 

It is possible to get the list of memory usage listed in percentage proportion, with a tiny for bash loop and some awk + process list command
 

TOTAL=$(free | awk '/Mem:/ { print $2 }')
for USER in $(ps haux | awk '{print $1}' | sort -u)
do
    ps hux -U $USER | awk -v user=$USER -v total=$TOTAL '{ sum += $6 } END { printf "%s %.2f\n", user, sum / total * 100; }'
done

107 1.34
115 2.10
119 1.34
daemon 1.32
dnscache 1.34
dnslog 1.32
hipo 1.59
mysql 4.79
ntp 1.34
privoxy 1.33
proftpd 1.32
qmaill 1.33
qmailq 1.33
qmailr 1.32
qmails 1.33
qscand 4.98
root 1.33
snmp 1.33
statd 1.33
vpopmail 2.35
www-data 86.48

Also a raw script which can be easily extended to give you some custom information on memory use by user list_memory_use_by_user.sh is here.
You can also want to debug further how much memory a certain users (lets say user mysql and my username hipo) is allocating, this can easily be achieved ps like so:
 

root@pcfreak:~# ps -o size,pid,user,command -u mysql –sort -size
 SIZE   PID USER     COMMAND
796924 14857 mysql   /usr/sbin/mysqld –basedir=/usr –datadir=/var/lib/mysql –plugin-dir=/usr/lib/mysql/plugin –user=mysql –pid-file=/var/run/mysqld/mysqld.pid –socket=/var/run/mysqld/mysqld.sock –port=3306

 

root@pcfreak~# ps -o size,pid,user,command -u hipo –sort -size|less
 SIZE   PID USER     COMMAND
13408 19063 hipo     irssi
 3168 19020 hipo     SCREEN
 2940  2490 hipo     -bash
 1844 19021 hipo     /bin/bash
 1844 19028 hipo     /bin/bash
 1844 19035 hipo     /bin/bash
 1844 19042 hipo     /bin/bash
 1844 19491 hipo     /bin/bash
 1844 22952 hipo     /bin/bash
  744  2487 hipo     sshd: hipo@pts/0
  744  2516 hipo     sshd: hipo@notty
  524  2519 hipo     screen -r
  412  2518 hipo     /usr/lib/openssh/sftp-server

You see from below output user running with www-data (this is Apache Webserver user in Debian) is eating 86.48% of overall system memory and MySQL server user is using only 4.79% of available memory

Output is shown in Megabytes per username memory usage, and user memory usage is ordered (stepping-down / descentive) from top to bottom

Getting more thoroughful and easier to read reporting without beeing a 31337 bash coder you can install and use on Linux smem – memory reporting tool .

SMEM can provide you with following memory info:

  • system overview listing
  • listings by process, mapping, user
  • filtering by process, mapping, or user
  • configurable columns from multiple data sources
  • configurable output units and percentages
  • configurable headers and totals
  • reading live data from /proc
  • reading data snapshots from directory mirrors or compressed tarballs
  • lightweight capture tool for embedded systems
  • built-in chart generation


Installing smem on Debian 6 / 7 / Ubuntu 14.04 / Turnkey Linux etc. servers is done with standard:

 

debian:~# apt-get install –yes smem
….

 

 

To install smem on CentOS 6 / 7:

 

[root@centos ~ ]# yum -y install smem
….


On Slackware and other Linux-es where smem is not available as a package you can install it easily from binary archive with:

 

cd /tmp/
wget http://www.selenic.com/smem/download/smem-1.3.tar.gz
tar xvf smem-1.3.tar.gz
sudo cp /tmp/smem-1.3/smem /usr/local/bin/
sudo chmod +x /usr/local/bin/smem

 


Two most common smem uses are:

 

root@mail:~# smem -u
User     Count     Swap      USS      PSS      RSS
dnslog       1       44       48       54      148
qmaill       4      232      124      145      464
hipo        11    13552     8596     9171    13160
qscand       2     4500   295336   295602   297508
root       188   217312  4521080  4568699  7712776

 

Below command shows (-u – Report memory usage by user, -t – show totals, -k – show unix suffixes)

root@mail:~# smem -u -t -k
User     Count     Swap      USS      PSS      RSS
dnslog       1    44.0K    48.0K    54.0K   148.0K
qmaill       4   232.0K   124.0K   145.0K   464.0K
hipo        11    13.2M     8.4M     9.0M    12.9M
qscand       2     4.4M   288.4M   288.7M   290.5M
root       188   212.2M     4.3G     4.4G     7.4G
—————————————————
           206   230.1M     4.6G     4.6G     7.7G


To get users memory use by percentage with smem:
 

root@mail:~# smem -u -p
User     Count     Swap      USS      PSS      RSS
dnslog       1    0.00%    0.00%    0.00%    0.00%
qmaill       4    0.00%    0.00%    0.00%    0.01%
hipo        11    0.17%    0.11%    0.11%    0.16%
qscand       2    0.05%    3.63%    3.63%    3.66%
root       194    2.64%   56.18%   56.77%   95.56%

It is also useful sometimes when you want to debug system overloads caused by external hardware drivers loaded into kernel causing issues to get list of system wide memory use sorted by user

 

 root@mail:~# smem -w -p
Area                           Used      Cache   Noncache
firmware/hardware             0.00%      0.00%      0.00%
kernel image                  0.00%      0.00%      0.00%
kernel dynamic memory        38.30%     36.01%      2.28%
userspace memory             60.50%      0.98%     59.53%
free memory                   1.20%      1.20%      0.00%


smem is very nice as if you're running it on a Desktop Linux system with Xserver installed you can see also graphical output of memory use by application:
 

root@desktop-pc:~# smem –bar pid -c "pss uss"


smem_graphical_representation-of-which-user-application-is-consuming-most-memory-gnu-linux-kde-screenshot-smem-command-line-tool

smem can even generate graphical pie charts to visualize better memory use
 

root@desktop-pc:~# smem -P '^k' –pie=name

 

generate-graphical-staticstics-linux-memory-use-by-pie-chart

If there is a high percentage shown in firmware/hardware this means some buggy module is loaded in kernel eating up memory, to fix it debug further and remove the problematic module.
userspace memory actually shows the percantage of memory out of all server available RAM that is being consumed by applications (non kernel and other system processes which make the system move). You see in above example the kernel itself is consuming about 40% of system overall available memory. 

We all know the SWAP field stands for hard disk drive used as a memory when system is out, but there are 3 fields which smem will report which will be probably unclear for most here is also explanation on what USS / PSS / RSS means?

RSS is the Resident Set Size and is used to show how much memory is allocated to that process and is in RAM. It does not include memory that is swapped out. It does include memory from shared libraries as long as the pages from those libraries are actually in memory. It does include all stack and heap memory too.

There is also PSS (proportional set size). This is a newer measure which tracks the shared memory as a proportion used by the current process. So if there were two processes using the same shared library from before.

USS stands for Unique set size, USS is just the unshared page count, i.e. memory returned when process is killed 

PSS = Proportional set size, (PSS),  is a more meaningful representation of the amount of memory used by libraries and applications in a virtual memory system.  
Because large portions of physical memory are typically shared among multiple applications, the standard measure of memory usage known as resident set size (RSS) will significantly overestimate memory usage. The parameter PSS instead measures each application’s “fair share” of each shared area to give a realistic measure. For most admins checking out the output from RSS (output) should be enough, it will indicate which user and therefore which daemon is eating up all your memory and will help you to catch problematic services which are cause your server to run out of RAM and start swapping to disk.

how to archive with Windows default zip (compress) algorithm

Tuesday, April 1st, 2014

windows-sent-to-compressed-file
I'm working on a decomissioning project (for those who hear decomissioning for a first time – in corporate world this means removal of service/s or assigned resources of a server or a physical server hardware that is not to be used in future or is obsolete). The decomissioning includes removal of Apache Tomcat (Software Configuration Item CI – in HP terms) from Microsoft Windows 2007 – Service Pack 2.
Part of decomissioning is of course to create backup of Tomcat Application server and for that I needed to create compressed archive of Tomcat instances. Usually I do archives on Windows using Winrar or Winzip but this time as the server productive server has the minimum installed there was no any external vendor produced archiving software available.

My memories from past were that there is a native compressing program embedded into Windows as I've unzipped compressed archives on Win hosts with no need for external WinZip. However until so far I never did .ZIP archive with no available external uncompress software.

Using Winzip or Winrar so far to make archive from a number of files I used to select files to enter Archive press right mouse button and select Create Archive (Add To Archive) so I expected this will work whenever no Winrar, however there was no obvious button like this, so I googled a bit to find out how is that possible ending up on Article from Microsoft titled "Compress and uncompress files (zip files)", there is a dumb proof video teaching Compressing files with Microsoft default ZIP program is done by the the weird "Send To" menu 🙂 

Selecting files to enter Archive;
> (Click Right Mouse Button) -> (Send To Compressed Zipped Folder)

compress_zipped_folder_with_windows_default_archive_algorithm how to archive with windows default compress archive

Honestly If I didn't checked the net probably I will never think of looking it there.

Finding spam sending php scripts on multiple sites servers – Tracing and stopping spammer PHP scripts

Monday, April 14th, 2014

stop_php_mail-spam-find-spammer-and-stop-php-spammer-websites
Spam has become a severe issue for administrators, not only for mail server admins but also for webshosting adms. Even the most secure spam protected mail server can get affected by spam due to fact it is configured to relay mail from other servers acting as web hosting sites.

Webhosting companies almost always suffer seriously from spam issues and often their mail servers gets blocked (enter spam blacklists), because of their irresponsible clients uploading lets say old vulnerable Joomla, WordPress without Akismet or proper spam handling plugin,a CMS which is not frequently supported / updated or custom client insecure php code.

What I mean is Shared server A is often configured to sent mail via (mail) server B. And often some of the many websites / scripts hosted on server A gets hacked and a spam form is uploaded and tons of spam start being shipped via mail server B.

Of course on mail server level it is possible to configure delay between mail sent and adopt a couple of policies to reduce spam, but the spam protection issue can't be completely solved thus admin of such server is forced to periodically keep an eye on what mail is sent from hosting server to mail server.
 


If you happen to be one of those Linux (Unix) webhosting admins who find few thousand of spammer emails into mail server logs or your eMail server queue and you can't seem to find what is causing it, cause there are multiple websites shared hosting using mainly PHP + SQL and you can't identify what php script is spamming by reviewing  Apache log / PHP files. What you can do is get use of:

PHP mail.log directive

Precious tool in tracking spam issues is a PHP Mail.log parameter, mail log paramater is available since PHP version >= 5.3.0 and above.
PHP Mail.log parameter records all calls to the PHP mail() function including exact PHP headers, line numbers and path to script initiating mail sent.

Here is how it is used:
 

1. Create empty PHP Mail.log file

touch /var/log/phpmail.log

File has to be writtable to same user with which Apache is running in case of Apache with SuPHP running file has to be writtable by all users.

On Debian, Ubunut Linux:

chown www:data:www-data /var/log/phpmail.log

On CentOS, RHEL, SuSE phpmail.log has to be owned by httpd:

chown httpd:httpd /var/log/phpmail.log

On some other distros it might be chown nobody:nobody etc. depending on the user with which Apache server is running.

 

2. Add to php.ini configuration following lines

mail.add_x_header = On
mail.log = /var/log/phpmail.log

PHP directive instructs PHP to log complete outbund Mail header sent by mail() function, containing the UID of the web server or PHP process and the name of the script that sent the email;
 

(X-PHP-Originating-Script: 33:mailer.php)


i.e. it will make php start logging to phpmail.log stuff like:
 

 

mail() on [/var/www/pomoriemonasteryorg/components/com_xmap/2ktdz2.php:1]: To: info@globalremarketing.com.au — Headers: From: "Priority Mail" <status_93@pomoriemon
astery.org> X-Mailer: MailMagic2.0 Reply-To: "Priority Mail" <status_93@pomoriemonastery.com> Mime-Version: 1.0 Content-Type: multipart/alternative;boundary="——
—-13972215105347E886BADB5"
mail() on [/var/www/pomoriemonasteryorg/components/com_xmap/2ktdz2.php:1]: To: demil7167@yahoo.com — Headers: From: "One Day Shipping" <status_44@pomoriemonastery.
org> X-Mailer: CSMTPConnectionv1.3 Reply-To: "One Day Shipping" <status_44@pomoriemonastery.com> Mime-Version: 1.0 Content-Type: multipart/alternative;boundary="—
——-13972215105347E886BD344"
mail() on [/var/www/pomoriemonasteryorg/components/com_xmap/2ktdz2.php:1]: To: domainmanager@nadenranshepovser.biz — Headers: From: "Logistics Services" <customer.
id86@pomoriemonastery.com> X-Mailer: TheBat!(v3.99.27)UNREG Reply-To: "Logistics Services" <customer.id86@pomoriemonastery.com> Mime-Version: 1.0 Content-Type: mult
ipart/alternative;boundary="———-13972215105347E886BF43E"
mail() on [/var/www/pomoriemonasteryorg/components/com_xmap/2ktdz2.php:1]: To: bluesapphire89@yahoo.com — Headers: From: "Priority Mail" <status_73@pomoriemonaster
y.org> X-Mailer: FastMailer/Webmail(versionSM/1.2.6) Reply-To: "Priority Mail" <status_73@pomoriemonastery.com> Mime-Version: 1.0 Content-Type: multipart/alternativ
e;boundary="———-13972215105347E886C13F2"

 

On Debian / Ubuntu Linux to enable this logging, exec:

echo 'mail.add_x_header = On' >> /etc/php5/apache2/php.ini
echo 'mail.log = /var/log/phpmail.log' >> /etc/php5/apache2/php.ini


I find it useful to symlink /etc/php5/apache2/php.ini to /etc/php.ini its much easier to remember php location plus it is a standard location for many RPM based distros.

ln -sf /etc/php5/apache2/php.ini /etc/php.ini

Or another "Debian recommended way" to enable mail.add_x_header logging on Debian is via:

echo 'mail.add_x_header = On' >> /etc/php5/conf.d/mail.ini
echo 'mail.log = /var/log/phpmail.log' >> /etc/php5/conf.d/mail.ini

On Redhats (RHEL, CentOS, SuSE) Linux issue:

echo 'mail.add_x_header = On' >> /etc/php.ini
echo 'mail.log = /var/log/phpmail.log' >> /etc/php.ini

3. Restart Apache

On Debian / Ubuntu based linuces:

/etc/init.d/apache2 restart

P.S. Normally to restart Apache without interrupting client connections graceful option can be used, i.e. instead of restarting do:

/etc/init.d/apache2 graceful

On RPM baed CentOS, Fedora etc.:

/sbin/service httpd restart

or

apachectl graceful
 

4. Reading the log

To review in real time exact PHP scripts sending tons of spam tail it:

tail -f /var/log/phpmail.log

 

mail() on [/var/www/remote-admin/wp-includes/class-phpmailer.php:489]: To: theosfp813@hotmail.com — Headers: Date: Mon, 14 Apr 2014 03:27:23 +0000 Return-Path: wordpress@remotesystemadministration.com From: WordPress Message-ID: X-Priority: 3 X-Mailer: PHPMailer (phpmailer.sourceforge.net) [version 2.0.4] MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="UTF-8"
mail() on [/var/www/pomoriemonasteryorg/media/rsinstall_4de38d919da01/admin/js/tiny_mce/plugins/inlinepopups/skins/.3a1a1c.php:1]: To: 2070ccrabb@kiakom.net — Headers: From: "Manager Elijah Castillo" <elijah_castillo32@pomoriemonastery.com> X-Mailer: Mozilla/5.0 (Windows; U; Windows NT 5.0; es-ES; rv:1.9.1.7) Gecko/20100111 Thunderbird/3.0.1 Reply-To: "Manager Elijah Castillo" <elijah_castillo32@pomoriemonastery.com> Mime-Version: 1.0 Content-Type: multipart/alternative;boundary="———-1397463670534B9A76017CC"
mail() on [/var/www/pomoriemonasteryorg/media/rsinstall_4de38d919da01/admin/js/tiny_mce/plugins/inlinepopups/skins/.3a1a1c.php:1]: To: 20wmwebinfo@schools.bedfordshire.gov.uk — Headers: From: "Manager Justin Murphy" <justin_murphy16@pomoriemonastery.com> X-Mailer: Opera Mail/10.62 (Win32) Reply-To: "Manager Justin Murphy" <justin_murphy16@pomoriemonastery.com> Mime-Version: 1.0 Content-Type: multipart/alternative;boundary="———-1397463670534B9A7603ED6"
mail() on [/var/www/pomoriemonasteryorg/media/rsinstall_4de38d919da01/admin/js/tiny_mce/plugins/inlinepopups/skins/.3a1a1c.php:1]: To: tynyrilak@yahoo.com — Headers: From: "Manager Elijah Castillo" <elijah_castillo83@pomoriemonastery.com> X-Mailer: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; pl; rv:1.9.1.9) Gecko/20100317 Thunderbird/3.0.4 Reply-To: "Manager Elijah Castillo" <elijah_castillo83@pomoriemonastery.com> Mime-Version: 1.0 Content-Type: multipart/alternative;boundary="———-1397463670534B9A7606308"
mail() on [/var/www/pomoriemonasteryorg/media/rsinstall_4de38d919da01/admin/js/tiny_mce/plugins/inlinepopups/skins/.3a1a1c.php:1]: To: 2112macdo1@armymail.mod.uk — Headers: From: "Manager Justin Murphy" <justin_murphy41@pomoriemonastery.com> X-Mailer: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; pl; rv:1.9.1.9) Gecko/20100317 Thunderbird/3.0.4 Reply-To: "Manager Justin Murphy" <justin_murphy41@pomoriemonastery.com> Mime-Version: 1.0 Content-Type: multipart/alternative;boundary="———-1397463670534B9A76086D1"

 

As you can see there is a junky spam mails sent via some spammer script uploaded under name .3a1a1c.php, so to stop the dirty bastard, deleted the script:

rm -f /var/www/pomoriemonasteryorg/media/rsinstall_4de38d919da01/admin/js/tiny_mce/plugins/inlinepopups/skins/.3a1a1c.php

It is generally useful to also check (search) for all hidden .php files inside directoring storing multiple virtualhost websites, as often a weirdly named hidden .php is sure indicator of either a PHP Shell script kiddie tool or a spammer form.

Here is how to Find all Hidden Perl / PHP scripts inside /var/www:

find . -iname '.*.php'
./blog/wp-content/plugins/fckeditor-for-wordpress-plugin/ckeditor/plugins/selection/.0b1910.php
./blog/wp-content/plugins/fckeditor-for-wordpress-plugin/filemanager/browser/default/.497a0c.php
./blog/wp-content/plugins/__MACOSX/feedburner_feedsmith_plugin_2.3/._FeedBurner_FeedSmith_Plugin.php

find . -iname '.*.pl*'

….

Reviewing complete list of all hidden files is also often useful to determine shitty cracker stuff

 find . -iname ".*"

Debugging via  /var/log/phpmail.log enablement is useful but is more recommended on development and staging (QA) environments. Having it enable on productive server with high amounts of mail sent via PHP scripts or just on dedicated shared site server could cause both performance issues, hard disk could quickly get and most importantly could be a severe security hole as information from PHP scripts could be potentially exposed to external parties.

Make Viber calls with no smartphone from Mobile to PC and from PC to Mobile – Bluestacks install android mobile apps on PC

Thursday, April 24th, 2014

Viber-for-smartphones-connect-freely-through-internet-voip-on-your-mobile

Since I've bought ZTE smat phone and I have Android on it, decided to install   Viber – iOS, Android and Desktop PC – Free Calls, Text and Picture sharing through the internet app. Viber is used by a lot of my people including many friends already so I installed it as well to be possible to speak for free with close friends …

Why Viber?


What makes this nifty app so great is its capability to make free calls over mobile phones through the Internet Viber.
Viber saves you a lot of money as calls are handled only through the Internet (you need Wifi on your mobile or Mobile 3G Internet access on phone) and you don't need to pay to your mobile operator 0.10 – 0.15 euro / cents per minute. Besides being Free another advantage of Viber is conversations sound quality which is much better than a regular phone call

Viber doesn't need a special registration, but as (login) identificator uses your mobile phone number – you just need to have a working Mobile operator phone num. Once registered under a number even if you change your mobile sim card to other operator (for example moving from country to country) still the Viber account will continue work. Another good reason to use Viber is it makes possible price free calls between different countries (for example if you travel a lot and you want to regularly speak with your wife) – in my case right now I'm in Bulgaria and my wife is in Belarus, so to save money and keep talking daily we use Viber daily.

What Devices and Operating System Viber Supports and what is Viber advantages / disadvantages ?


Another reason why Viber is so great is its multi-platform support it works on iPhone, Blackberry, Windows Phone, Nokia (Symbian), Windows, Mac OS and even (Korean own OS-ed) Bada devices. Some might argue that Viber is inferior to Skype and interms of Voice and Video quality its better because of its enhanced HD voice enhanced codecs, besides that Viber's video is still in Beta. However Viber has one big advantage it makes easy possible to reach people using just their Mobile Phone numbers where in Skype it takes time and effort to register in Skype install application on your Mobile keep yourself logged in in Skype and have all contacts previously added, all this happens automatically in Viber in time of installation of Viber App on your mobile.
 

Which Is Cheaper Viber or Skype?

Skype_VS_Viber-VOIP-Prices-which-is-cheaper-skype-or-viber


Once installed Viber could integrate itself with rest of your Mobile OS Call Manager and in time of call a friend number you have the opportunity to make it free Viber call. Viber are also selling Viber Credits so if you want to use your Viber Voice Over IP you can call external mobile operator numbers on a very very cheap price. Viber Calls to landline or mobile phones could be up to 400% cheaper than Skype! Whether you own a Smartphone it will be nice to give Viber a try.

Viber – How to make Phone calls between Desktop PC and Smarphone Mobile

 

One not so standard Viber use is to make Viber calls with no smartphone (at hand) from PC to another Viber equipped Mobile and vice versa.
I needed to make Viber calls from my ZTE Android running mobile to my wife's MacBook Air PC because her mobile is an old Nokia running obscure Symbian version which is not supporting Viber + she doesn't have an Internet access tariff switched on her mobile.

Here is what I had to do to make Phone calls between my Mobile Viber App and my wife's MacbookAir Notebook PC:
 

  • Install BlueStacks Web App Player

     

     

     

    BlueStacks_emulate-google-appstore-on-Windows-and-Mac-OS-android-emulator_Logo
    BlueStacks App Player is a software designed to enable Android applications to run on Windows PC, Apple Macintosh Computers and Windows tablets. BlueStacks is something like (VMware, Qemu) Virtual Machine which allows you to install and run any Android App on your Desktop PC.
    Its curious that app was created by Rosen Sharma in 2008 an ex CTO (Chief Technology Officer) of McAfee
     

  •  A mobile phone with a working SIM card (Nokia 6310 or any old mobile no need to be a smartphone
     
  • Desktop PC with Windows 7, 8 or PC with Mac OS


Install Bluestacks

BlueStacks is needed in order to emulate a smartphone on your PC, therefore once setupped Bluestacks. Launch it and  inside its necessary to login with your Gmail (Google Account) in order to allow access to Google Play Appstore on your PC.
viber with no mobile phone bluestacks
 

Installing and Verifying Viber

This is the most crucial and tricky part in order to make Viber working on any device you need to receive a special Viber verification code, you need to fill in this code to confirm Viber installation on PC. Here I assume you have BlueStacks running with Viber Application installed.

viber-running-under-bluestacks-on-windows-7-8-screenshot

First will be prompted to Agree with Terms and Conditions and provide Mobile Phone number for verification. Tell the Viber app that you have a smartphone with Viber already when prompted. After receving Viber Verification Code you need to fill in this code into BlueStacks Window (inside Viber should be running), go further to next step and you should be done with Desktop PC Viber number registration.

N.B. ! One brackets to open here is you need to have a working Mobile Phone number where you will receive the verification code as SMS, otherwise you cannot get the verification. On your filled in mobile phone number you will get the verification code as SMS.

Making Viber Calls to (Windows Mac) PC without Smartphone

There is no more further need for BlueStacks so you can uninstall it, however I preferred to keep it as its useful to be able to install Android Applications straight on your Desktop PC. To start using Viber on Desktop, just launch Viber application (not through BlueStacks) but the direct install.

Use Viber dial pad to dial your desired remote Smartphone number with Viber equipped.
Enjoy the free Internet calls ! 🙂
 

Ditaa convert ASCII diagrams into bitmap graphic (pictures)

Monday, May 12th, 2014

ditta_convert-ascii-art-diagram-to-png-jpg-picture
As part of my passion for ASCII art, I've found another interesting tool useful to ASCII art maniacs like me, the tool is called ditta and is able to convert manually drawn ASCII art diagrams to graphics, below is tool description from my debian apt-cache as well as a screenshot:

 apt-cache show ditaa|grep -i ditaa -A 4

Package: ditaa
Priority: optional
Section: graphics
Installed-Size: 164
Maintainer: David Paleino <dapal@debian.org>

Filename: pool/main/d/ditaa/ditaa_0.9+ds1-2_all.deb
Size: 107270
MD5sum: 05ec52d9274b954b053f1835ca5d7a7f
SHA1: 792d91d05fff2a2a19c0ebce317351d138436c18
SHA256: c4319d32e7918aab782e2f38cdad745bc9023f9f09a999033d983095ee4f70d5

 DiTAA is a small command-line utility that can convert diagrams drawn using
 ASCII art ("drawings" that contain characters that resemble lines, like | /
 and -), into proper bitmap graphics.
 .
 DiTAA also uses special markup syntax to increase the possibilities of shapes
 and symbols that can be rendered.
Homepage: http://ditaa.org

 

To install ditaa on Debian and Ubuntu Linux:

debian:~# apt-get install --yes ditaa
...

Ditaa text diagram to Graphics converter is also available in Fedora Linux and in Source RPMs to be used on Redhat Based RPM distributions.
To install in most of RPM based Linuxes:

[root@fedora:~]# yum install -y ditaa

For most people probably Ditta will not be of any value except as a PoC and of a Hack value just like Ditaa's home page suggests. Nomatter that Ditta is cool but has just 2 drawback it doesn't understand non-latin characters i.e. Cyrillic and requires Java Virtual Machine .. but if you're a real geek you will do  the sacrifice to install a whole bunch of the heavy java for the sake of some oldschool fun 🙂 Being written in Java makes Ditta multi-platform, but you will need a Java VM version of at least 1.6 (it doesn't work with Java 1.5).

The format Ditta understands is close to HTML

<ditaa [optional parameters]>
 
... (some ditaa-code) ...
 
</ditaa>


There are also special tags understood by Ditta which are automatically turned into shaped graphical buttons and forms.
 

Possible tags

Not all shape selector tags are documented on the ditaa site. A quick source scan revealed:

tag Description
{c} decision(Choice)
{d} document
{io} input/output, parallelogram
{mo} manual operation
{o} ellipse, circle
{s} storage
{tr} trapezoid (looks like an inverted {mo} )

Here is an example Ditta code
 

<ditaa round noedgesep right>
+--------+   +-------+    +-------+
|        | --+ ditaa +--> |       |
|  Text  |   +-------+    |diagram|
|Document|   |!magic!|    |       |
|     {d}|   |  c478 |    |       |
+---+----+   +-------+    +-------+
:                         ^
|       Lots of work      :
+-------------------------+
</ditaa>

This Ditta code will generate following picture:

ditaa_convert-ascii-art-picture-to-graphic-png-jpg-bmp


To learn more on ditta please check Ditaa's Project homepage on Sourceforge
Many thanks to Cybercity's 30 Cool Open Source Software of 2013 for inspiring this post.

Install VMWare tools on Debian and Ubuntu Linux – Enable VMWare Fullscreen and copy paste between OS host and Virtual machine

Wednesday, May 28th, 2014

install-vmware-tools-on-debian-gnu-linux-and-ubuntu-howto

If you need to use Virtual Machine to run some testing on heterogenous Operating Systems and you have chosen VMWare as a Virtual Machine. You will soon notice some of Virtual Machines functionality like copy between host operating system and Virtual Machine, true fullscreen mode and most importantly Copy paste between your host operating system and VMWare is not working. I'm not too much into Virtualization these days so for me it was truely shocking that a proprietary software like VMWare, claimed to be the best and most efficient Virtual Machine nowadays is not supporting copy / paste, fullscreen and copy between host and guest OS.  For those arguing why I'm using VMWare at all as it is proprietary and there is already free software Virtual Machines like QEMU and Oracle's VirtualBox its simply because now I have the chance to install and use VMWare 9 Enterprise on my work place at HP with a free Corporate license – in other words I'm using VMWare just for the sake of educating myself and would always recommend VirtualBox for those looking for good substitute free alternative to VMWare.

Before trying out VMWare, I tried Virtualbox to emulate Linux on my HP work PC running Windows with VirtualBox I was having issues with keyboard not working (because of lack of support of USB, no full screen support and lack of copy / paste between OS-es), I've just recently understood this is not because Virtualbox is bad Virtualization solution but because I forgot to install VirtualBox Oracle VM VirtualBox Extension Pack which allows support for USB, enables copy paste and full screen support. The equivalent to Virtualbox Oracle VM VirtualBox in VMWare world is called VMWare-Tools and once the guest operating system is installed inside VMWare VM, its necessary to install vmware-tools to enable better screen resolution and copy paste.
 

In Windows Virtual Machine installation of vmware-tools is pretty straight forward you go through VMWare's menus

 

VM -> Install Vmware-tools

install-vmware-tools-on-linux-guest-host-os-debian-redhat-screenshot

follow the instructions and you're done, however as always installing VMWare-tools on Linux is little bit more complicated you need to run few commands from Linux installed inside the Virtual Machine to install vmware-tools. Here is how vmware-tools is installed on Debian / Ubuntu / Linux Mint and rest of Debian based operating systems:

  1. Install Build essentials and gcc You need to have this installed some developer tools as well as GCC compiler in order for the vmware-tools to compile a special Linux kernel module which enables extra support (integration) between the VMWare VM and the installed inside VM Linux distro

apt-get install --yes build-essential gcc
...

2. Install appropriate Linux headers corresponding to current Linux OS installed kernel

apt-get install --yes linux-headers-$(uname -r)
....

3. Mount CD (Virtual) Content to obtain the vmware-tools version for your Linux

Be sure to have first checked from VMWare menus on menus VM -> Intall Vmware Tools
This step is a little bit strange but just do it without too much questioning …


mount /dev/cdrom /mnt/
umount /media/cdrom0/
mount /media/cdrom
mount /dev/sr0 /mnt/cdrom/
mount /dev/sr0 /mnt/

 

Note that /dev/sr0, might already be mounted and sometimes it might be necessary to unmount it first (don't remember exactly if I unmounted it or not)

4. Copy and Untar VMwareTools-9.2.0-799703.tar.gz

cp -rpf /media/cdrom/VMwareTools-9.2.0-799703.tar.gz /tmp/
cd /tmp/
tar -zxvvf VMwareTools-9.2.0-799703.tar.gz
...

5. Run vmware-tools installer

cd vmware-tools-distrib/
./vmware-install.pl

You will be asked multiple questions you can safely press enter to answer with default settings to all settings, hopefully if all runs okay this will make VMWare Tools installed
 

Creating a new VMware Tools installer database using the tar4 format.
Installing VMware Tools.
In which directory do you want to install the binary files?
[/usr/bin]
What is the directory that contains the init directories (rc0.d/ to rc6.d/)?
[/etc]
What is the directory that contains the init scripts?
[/etc/init.d]
In which directory do you want to install the daemon files?
[/usr/sbin]
In which directory do you want to install the library files?
[/usr/lib/vmware-tools]
The path "/usr/lib/vmware-tools" does not exist currently. This program is
going to create it, including needed parent directories. Is this what you want?
[yes]
In which directory do you want to install the documentation files?
[/usr/share/doc/vmware-tools]
The path "/usr/share/doc/vmware-tools" does not exist currently. This program
is going to create it, including needed parent directories. Is this what you
want? [yes]
The installation of VMware Tools 9.2.0 build-799703 for Linux completed
successfully. You can decide to remove this software from your system at any
time by invoking the following command: "/usr/bin/vmware-uninstall-tools.pl".
Before running VMware Tools for the first time, you need to configure it by
invoking the following command: "/usr/bin/vmware-config-tools.pl". Do you want
this program to invoke the command for you now? [yes]
Initializing…
Making sure services for VMware Tools are stopped.
Stopping VMware Tools services in the virtual machine:
Guest operating system daemon: done
Unmounting HGFS shares: done
Guest filesystem driver: done
[EXPERIMENTAL] The VMware FileSystem Sync Driver (vmsync) is a new feature that creates backups of virtual machines. Please refer to the VMware Knowledge Base for more details on this capability. Do you wish to enable this feature?
[no]
Before you can compile modules, you need to have the following installed…
make
gcc
kernel headers of the running kernel
Searching for GCC…
Detected GCC binary at "/usr/bin/gcc-4.6".
The path "/usr/bin/gcc-4.6" appears to be a valid path to the gcc binary.
Would you like to change it? [no]
Searching for a valid kernel header path…
Detected the kernel headers at "/lib/modules/3.2.0-4-amd64/build/include".
The path "/lib/modules/3.2.0-4-amd64/build/include" appears to be a valid path
to the 3.2.0-4-amd64 kernel headers.
Would you like to change it? [no]
The vmblock enables dragging or copying files between host and guest in a
Fusion or Workstation virtual environment. Do you wish to enable this feature?
[no] yes
make: Leaving directory `/tmp/vmware-root/modules/vmblock-only'

No X install found.
Creating a new initrd boot image for the kernel.
update-initramfs: Generating /boot/initrd.img-3.2.0-4-amd64
Checking acpi hot plug done
Starting VMware Tools services in the virtual machine:
Switching to guest configuration: done
VM communication interface: done
VM communication interface socket family: done
File system sync driver: done
Guest operating system daemon: done
The configuration of VMware Tools 8.6.10 build-913593 for Linux for this
running kernel completed successfully.
You must restart your X session before any mouse or graphics changes take
effect.
You can now run VMware Tools by invoking "/usr/bin/vmware-toolbox-cmd" from the
command line or by invoking "/usr/bin/vmware-toolbox" from the command line
during an X server session.
To enable advanced X features (e.g., guest resolution fit, drag and drop, and
file and text copy/paste), you will need to do one (or more) of the following:
1. Manually start /usr/bin/vmware-user
2. Log out and log back into your desktop session; and,
3. Restart your X session.
Enjoy,
–the VMware team
Found VMware Tools CDROM mounted at /mnt. Ejecting device /dev/sr0 …

.To make sure vmware-tools compiled modules are loaded into Linux kernel inside VM, restart the Virtual Machine. Once Linux boots again and you login to gnome-terminal to check what is vmware-tools status (e.g. if properly loaded) run:

service vmware-tools status
vmtoolsd is running

install-vmware-tools-on-debian-gnu-linux-and-ubuntu-virtual-machine-screenshot

This method of installing works on Debian 7 (Wheezy) but same steps should work on any Ubuntu and rest of Debian derivatives. For Redhat (RPM) based Linux distributions to install vmware-tools after mounting cdrom drive following above instructions you will have an rpm package instead of .tar.gz archive so all you have to do is install the rpm, e.g. launch smth. like:

rpm -Uhv /mnt/cdrom/VMwareTools-9.2.0-799703.i386.rpm
Cheers 😉

Linux / BSD: Check if Apache web server is listening on port 80 and 443

Tuesday, June 3rd, 2014

apache_check_if_web_server_running_port-80-and-port-443-logo-linux-and-bsd-check-apache-running
If you're configuring a new Webserver or adding a new VirtualHost to an existing Apache configuration you will need to restart Apache with or without graceful option once Apache is restarted to assure Apache is continuously running on server (depending on Linux distribution) issue:

1. On Debian Linux / Ubuntu servers

# ps axuwf|grep -i apache|grep -v grep

root 23280 0.0 0.2 388744 16812 ? Ss May29 0:13 /usr/sbin/apache2 -k start
www-data 10815 0.0 0.0 559560 3616 ? S May30 2:25 _ /usr/sbin/apache2 -k start
www-data 10829 0.0 0.0 561340 3600 ? S May30 2:31 _ /usr/sbin/apache2 -k start
www-data 10906 0.0 0.0 554256 3580 ? S May30 0:20 _ /usr/sbin/apache2 -k start
www-data 10913 0.0 0.0 562488 3612 ? S May30 2:32 _ /usr/sbin/apache2 -k start
www-data 10915 0.0 0.0 555524 3588 ? S May30 0:19 _ /usr/sbin/apache2 -k start
www-data 10935 0.0 0.0 553760 3588 ? S May30 0:29 _ /usr/sbin/apache2 -k start

 


2. On CentOS, Fedora, RHEL and SuSE Linux and FreeBSD

ps ax | grep httpd | grep -v grep

 

7661 ? Ss 0:00 /usr/sbin/httpd
7664 ? S 0:00 /usr/sbin/httpd
7665 ? S 0:00 /usr/sbin/httpd
7666 ? S 0:00 /usr/sbin/httpd
7667 ? S 0:00 /usr/sbin/httpd
7668 ? S 0:00 /usr/sbin/httpd
7669 ? S 0:00 /usr/sbin/httpd
7670 ? S 0:00 /usr/sbin/httpd
7671 ? S 0:00 /usr/sbin/httpd

 

Whether a new Apache IP Based VirtualHosts are added to already existing Apache and you have added new

Listen 1.1.1.1:80
Listen 1.1.1.1:443

directives, after Apache is restarted to check whether Apache is listening on port :80 and :443
 

netstat -ln | grep -E ':80|443'

tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:443            0.0.0.0:*               LISTEN


Meaning of 0.0.0.0 is that Apache is configured to Listen on Any Virtualhost IPs and interfaces. This output is usually returned whether in Apache config httpd.conf / apache2.conf webserver is configured with directive.

Listen *:80
 

If in netstat output there is some IP poping up for example  "192.168.1.1:http", this means that only connections to the "192.168.1.1" IP address will be accepted by Apache.

Another way to look for Apache in netstat (in case Apache is configured to listen on some non-standard port number) is with:

netstat -l |grep -E 'http|www'

tcp        0      0 *:www                   *:*                     LISTEN


As sometimes it might be possible that Apache is listening but its processes are in in defunct (Zommbie) state it is always a good idea, also to check if pages server by Apache are opening in browser (check it with elinks, lynx or curl)

To get more thorough information on Apache listened ports, protocol, user with which Apache is running nomatter of Linux distribution use lsof command:
 

/usr/bin/lsof -i|grep -E 'httpd|http|www'

httpd     6982 nobody    3u  IPv4  29388359      0t0  TCP www.pc-freak.net:https (LISTEN)
httpd    18071 nobody    3u  IPv4 702790659      0t0  TCP www.pc-freak.net:http (LISTEN)
httpd    18071 nobody    4u  IPv4 702790661      0t0  TCP www.pc-freak.net.net:https (LISTEN)


If Apache is not showing up even though restarted check what is going wrong in the error logs:

– on Debian standard error log is /var/log/apache2/error.log
– On RHEL, CentOS, SuSE std. error log is in /var/log/httpd/error.log
– on FeeBSD /var/log/httpd-error.log