Posts Tagged ‘key’

WordPress Plugins to monitor and debug WP enabled plugins – Find Errors / Warnings and Remove WP problematic plugins slowing down your Website (blog) database

Thursday, February 19th, 2015

plugins-to-monitor-debug-wordpress-enabled-plugins-how-to-track-find-errors-and-warnings-and-remove-problematic-wp-extensions-that-slow-down-your-website

Recent days, I'm spending a lot of time again trying to optimize my wordpress blog. Optimizing WP for better efficiency is becoming harder and harder task day by day as the website file content data is growing along with SQL databases. Moreover situation gets even worse because the number of plugins enabled on my blog is incrementally growing with time because, there is more and more goodies I'd like to add.
Optimizing WordPress to run for Speed on a server is a whole a lot of art and its a small universe in itself, because as of time of writting this post the count (number) of WordPress available PLUGINS is 36,197 ! 

1. Manually Tracking WordPress  Plugins causing Slow SQL Queries (MySQL bottleneck) issues directly using console / SSH

Because of its open source development and its nice modular design wordpress has turned into a standard for building small, middle sized and large websites (some WordPress based blogs and sites have from 50 000 to 100 000 unique pages!). My blog is still a small WordPress site with only 1676 posts, so I still haven't reached the high volume traffic optimization requirements but still even though I have a relatively good server hardware  8GB RAM / (2×2.70 Ghz Intel CPU) / 500 GB (7400 RPM HDD) at times I see Apache Webservers is unable to properly serve coming requests because of MySQL database (LEFT JOIN) requests being slow to serve (taking up to few seconds to complete) and creating a MySQL table lock, putting all the rest SQL queries to stay in a long unserved queues line, I've realized about this performance issue by using a a mysql cli (command) client and few commands and console command (tool) called mytop (also known as mtop). MyTop refreshes every 3 seconds, so the slow query will immediately stay on screen to view moer info about it press "f" and type the  in query ID.

mysql-top-running-on-gnu-linux-server-tracking-sql-queries-in-console-screenshot.png

mysql-top-running-on-gnu-linux-server-tracking-sql-queries-in-console-screenshot2

Finally it is very useful to run  for a while MySQL server logging to /var/log/mysql/slow-query.log:
Slow query is enabled (on my Debian 7 Wheezy host) by adding to /etc/mysql/my.cnf
after conf section

 

vim /etc/mysql/my.cnf
#
# * Logging and Replication
#
# Both location gets rotated by the cronjob.
# Be aware that this log type is a performance killer.
# As of 5.1 you can enable the log at runtime!
#general_log_file        = /var/log/mysql/mysql.log
#general_log             = 1
#
# Error logging goes to syslog due to /etc/mysql/conf.d/mysqld_safe_syslog.cnf.
#
# Here you can see queries with especially long duration

 

Paste:

 

slow_query_log = 1
slow_query_log_file = /var/log/mysql/slow-query.log
long_query_time = 2
log-queries-not-using-indexes

 

And then to make new mysql configuration load restarted mysql server:

 

debian-server:~# /etc/init.d/mysql restart
Stopping MySQL database server: mysqld.
Starting MySQL database server: mysqld ..
Checking for tables which need an upgrade, are corrupt or were
not closed cleanly..

 

Leaving mysql-slow.log to be enabled for 30 minutes to an 1 hrs is a good time to track most problematic slow queries and based on this queries, I took parts of  SQL UPDATE / SELECT / INSERT etc. Db queries which was problematic and grepped throughout /var/www/blog/wp-content/plugin files in order to determine which WordPress Plugin is triggering the slow query, causing blog to hang when too many clients try to see it in browser.

My main problematic SQL query having long execution time  (about 2 to 3 seconds!!!) most commonly occuring in slow-query.log was:

 

SELECT DISTINCT post_title, ID, post_type, post_name FROM wp_posts wposts LEFT JOIN wp_postmeta wpostmeta ON wposts.ID = wpostmeta.post_id LEFT JOIN wp_term_relationships ON (wposts.ID = wp_term_relationships.object_id) LEFT JOIN wp_term_taxonomy ON (wp_term_relationships.term_taxonomy_id = wp_term_taxonomy.term_taxonomy_id) WHERE (post_type='page' OR (wp_term_taxonomy.taxonomy = 'category' AND wp_term_taxonomy.term_id IN(11))) AND post_status = 'publish' AND LENGTH(post_title)>=5 ORDER BY LENGTH(post_title) ASC LIMIT 500

Because above query uses SQL Column names and Tables which are not hard coded in PHP code, to find out which plugins is most probably to launch this complex LEFT JOIN query, I used a quick bash one-liner:

 

# cd /var/www/blog/wp-content/plugins

 

# for i in $(grep -rli 'SELECT DISTINCT' *); do grep -rli 'LEFT JOIN' $i; done 
./seo-automatic-links/seo-links.php
./wp-postviews/wp-postviews.php
./yet-another-related-posts-plugin/classes/YARPP_Cache_Tables.php

 

I wanted to put less load on CPU during grep so looked for string only in .PHP extensioned files with:

 

 # for i in $(find . -iname '*.php' -exec grep -rli 'SELECT DISTINCT' '{}' \;); do grep -rli 'LEFT JOIN' $i; done
./seo-automatic-links/seo-links.php
./wp-postviews/wp-postviews.php
./yet-another-related-posts-plugin/classes/YARPP_Cache_Tables.php


As you can see the complex query is being called from PHP file belonging to one of 3 plugins

  • SEO Automatic Links – this is SEO Smart Links WP plugin (Does internal bliog interlinking in order to boast SEA)
  • WP PostViews – WordPress Post Views plugin (Which allows me to show how many times an article was read in WP Widget menu)
  • Yet Another Related Posts – Which is WP plugin I installed / enabled to show Related posts down on each blog post


2. Basic way to optimize MySQL slow queries (EXPLAIN / SHOW CREATE TABLE)

Now as I have a basic clue on plugins locking my Database, I disabled them one by one while keeping enabled mysql slow query log and viewing queries in mytop and I figure out that actually all of the plugins were causing a short time overheat (lock) on server Database because of LEFT JOINs. Though I really like what this plugins are doing, as they boast SEO and attract prefer to disable them for now and have my blog all the time responsible light fast instead of having a little bit better Search Engine Optimization (Ranking) and loosing many of my visitors because they're annoyed to wait until my articles open

Before disabling I tried to optimize the queries using MySQL EXPLAIN command + SHOW CREATE TABLE (2 commands often used to debug slow SQL queries and find out whether a Column needs to have added INDEX-ing to boast MySQL query).

Just in case if you decide to give them a try here is example on how they're used to debug problematic SQL query:
 

  1. mysql> explain SELECT DISTINCT post_title, ID, post_type, post_name
  2.     -> FROM wp_posts wposts LEFT JOIN wp_postmeta wpostmeta
  3.     -> ON wposts.ID = wpostmeta.post_id LEFT JOIN wp_term_relationships
  4.     -> ON (wposts.ID = wp_term_relationships.object_id) LEFT JOIN wp_term_taxonomy
  5.     -> ON (wp_term_relationships.term_taxonomy_id = wp_term_taxonomy.term_taxonomy_id)
  6.     -> WHERE (post_type='page'
  7.     -> OR (wp_term_taxonomy.taxonomy = 'category'
  8.     -> AND wp_term_taxonomy.term_id IN(11,15,17)))
  9.     -> AND post_status = 'publish'
  10.     -> AND LENGTH(post_title)>=5
  11.     -> ORDER BY LENGTH(post_title) ASC
  12.     -> LIMIT 500;
  13. +—-+————-+———————–+——–+——————+———+———+———————————————+——+———————————————-+
  14. | id | select_type | table                 | type   | possible_keys    | key     | key_len | ref                                         | rows | Extra                                        |
  15. +—-+————-+———————–+——–+——————+———+———+———————————————+——+———————————————-+
  16. |  1 | SIMPLE      | wposts                | ALL    | type_status_date | NULL    | NULL    | NULL                                        | 1715 | Using where; Using temporary; Using filesort |
  17. |  1 | SIMPLE      | wpostmeta             | ref    | post_id          | post_id | 8       | blog.wposts.ID                              |   11 | Using index; Distinct                        |
  18. |  1 | SIMPLE      | wp_term_relationships | ref    | PRIMARY          | PRIMARY | 8       | blog.wposts.ID                              |   19 | Using index; Distinct                        |
  19. |  1 | SIMPLE      | wp_term_taxonomy      | eq_ref | PRIMARY          | PRIMARY | 8       | blog.wp_term_relationships.term_taxonomy_id |    1 | Using where; Distinct                        |
  20. +—-+————-+———————–+——–+——————+———+———+———————————————+——+———————————————-+
  21. 4 rows in set (0.02 sec)
  22.  
  23. mysql>
  24.  

     

     

  1. mysql> show create table wp_posts;
  2. +———-+————————–+
  3. | Table    | Create Table                                                                                                                                                                                                                                                                                                                                                                                                                                 |
  4. +———-+————————–+
  5. | wp_posts | CREATE TABLE `wp_posts` (
  6.   `ID` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
  7.   `post_author` bigint(20) unsigned NOT NULL DEFAULT '0',
  8.   `post_date` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
  9.   `post_date_gmt` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
  10.   `post_content` longtext NOT NULL,
  11.   `post_title` text NOT NULL,
  12.   `post_excerpt` text NOT NULL,
  13.   `post_status` varchar(20) NOT NULL DEFAULT 'publish',
  14.   `comment_status` varchar(20) NOT NULL DEFAULT 'open',
  15.   `ping_status` varchar(20) NOT NULL DEFAULT 'open',
  16.   `post_password` varchar(20) NOT NULL DEFAULT '',
  17.   `post_name` varchar(200) NOT NULL DEFAULT '',
  18.   `to_ping` text NOT NULL,
  19.   `pinged` text NOT NULL,
  20.   `post_modified` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
  21.   `post_modified_gmt` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
  22.   `post_content_filtered` longtext NOT NULL,
  23.   `post_parent` bigint(20) unsigned NOT NULL DEFAULT '0',
  24.   `guid` varchar(255) NOT NULL DEFAULT '',
  25.   `menu_order` int(11) NOT NULL DEFAULT '0',
  26.   `post_type` varchar(20) NOT NULL DEFAULT 'post',
  27.   `post_mime_type` varchar(100) NOT NULL DEFAULT '',
  28.   `comment_count` bigint(20) NOT NULL DEFAULT '0',
  29.   PRIMARY KEY (`ID`),
  30.   KEY `post_name` (`post_name`),
  31.   KEY `type_status_date` (`post_type`,`post_status`,`post_date`,`ID`),
  32.   KEY `post_parent` (`post_parent`),
  33.   KEY `post_author` (`post_author`),
  34.   FULLTEXT KEY `post_related` (`post_title`,`post_content`)
  35. ) ENGINE=MyISAM AUTO_INCREMENT=12033 DEFAULT CHARSET=utf8 |
  36. +———-+———————-+
  37. 1 row in set (0.00 sec)
  38.  
  39. mysql>
  40.  


By the way above output is a paste from the the new PasteBin Open Source (Stikked powered) service I started on pc-freak.net – paste.pc-freak.net (p.pc-freak.net) 🙂

Before I took final decision to disable slow WP plugins, I've experimented a bit trying to add INDEX to Table Column (wposts) in hope that this would speed up SQL queries with:

 

mysql> ALTER TABLE TABLE_NAME ADD INDEX (wposts);

 

But this didn't improve query speed even on the contrary it make execution time worse.

3. Tracking WordPress Plugin PHP Code Execution time and Plugins causing Slow SQL Queries (MySQL bottleneck) issues through WP itself

Well fine, I'm running my own hosted Blog and WordPress sites, but for people who have wordpress sites on shared hosting, there is usually no SSH (Terminal) Access to server, those people will be happy to hear there are 2 Free easy installable WordPress plugins which can be used to Debug Slow WordPress Plugins SQL Queries as well as plugin to Track which plugin takes most time to execute, this are:
 

 

a) P3 Plugin Performance Profiler  

runs a scan over your site to determine what resources your plugins are using, and when, during a standard page request. P3 PPP Can even create reports in a beatiful Excel like Pie chart sheet.

p3-plugin-performance-profiler-godaddy-screenshot-debian-gnu-linux-wordpress-website

Another useful thing to see with P3 PPP is Detailed Timeline it shows when the plugins are being loaded during new page request so you can see if there is a certain sequence in time when a plugin slows down the website.

detailed_timeline-wordpress-p3-performance-plugin-on-website-screenshot

The pictures says it all as P3 PPP is Godaddy's work, congrats to GoDaddy, they've done great job.

 

b) WordPress memory Viewer WP plugins

Is useful to check how much memory each of WordPress plugin is taking on user (visitor) request.
Memory Viewer is allows you to view WordPress’ memory utilization at several hooks during WordPress’ execution. It also shows a summary of MySQL Queries that have ran as well as CPU time.
To use it download it to plugins/ folder as usual enable it from:

Installed Plugins -> (Inactive) -> Memory Viewer (Enable)

To see statistics from Memory Viewer open any post from your blog website and scroll down to the bottom you will notice the statistics, showing up there, like on below screenshot.

wordpress-memory-viewer-plugin-debian-gnu-linux-hosted-website-show-which-plugin-component-eats-most-memory-in-wordprses-blog
 

Though WP Memory Viewer is said to work only up to WP version 3.2.1, I've tested it and it works fine on my latest stable WordPress 4.1 based blog.

c) WordPress Query Monitor

wordpress-query-monitor-plugin-to-monitor-track-and-optimize-problems-with-sql-caused-by-wp-plugins.png
 

Query Monitor is a debugging plugin for anyone developing with WordPress but also very helpful for anyone who want to track issues with plugins who use the database unefficient.
It has some advanced features not available in other debugging plugins, including automatic AJAX debugging and the ability to narrow down things by plugin or theme.
You can view plenty of precious statistics on how enabled plugins query the database server, here is a short overview on its Database Queries capabilities:

  • Shows all database queries performed on the current page
  • Shows affected rows and time for all queries
  • Show notifications for slow queries and queries with errors
  • Filter queries by query type (SELECT, UPDATE, DELETE, etc)
  • Filter queries by component (WordPress core, Plugin X, Plugin Y, theme)
  • Filter queries by calling function
  • View aggregate query information grouped by component, calling function, and type
  • Super advanced: Supports multiple instances of wpdb on one page
  • Once enabled from Plugins you will see it appear as a new menu on bottom Admin raw.

An important note to make here is latest Query Monitor extension fails when loaded on current latest Wordpress 4.1, to use it you will have to download and useolder Query Monitor plugin version 2.6.8 you can download it from here

d) Debug Bar

If you want you want a Memory Viewer like plugin for more complex used components memory debugging, reporting if (WP_DEBUG is set in wp-config.php) also check out Debug Bar .
For me Debug Bar was very useful because it show me depreciated functions some plugins used, so I substituted the obsoleted function with new one.

 

debug-bar-debug-wordpress-plugins-memory-use-screenshot-website


4. Server Hardware hungry (slow) WordPress plugins that you better not use

While spending time to Google for some fixes to WP slow query plugins – I've stumbled upon this post giving a good list with WordPress Plugins better off not to use because they will slow down your site
This is a publicly well known list of WP plugins every WordPress based site adminstrator should avoid, but until today I didn't know so my assumption is you don't know either ..

Below plugins are extremely database intensive mentioned in article that we should better (in all cases!) avoid:

  • Dynamic Related Posts
  • SEO Auto Links & Related Posts
  • Yet Another Related Posts Plugin
  • Similar Posts
  • Contextual Related Posts
  • Broken Link Checker — Overwhelms even our robust caching layer with an inordinate amount of HTTP requests.
  • MyReviewPlugin — Slams the database with a fairly significant amount of writes.
  • LinkMan — Much like the MyReviewPlugin above, LinkMan utilizes an unscalable amount of database writes.
  • Fuzzy SEO Booster — Causes MySQL issues as a site becomes more popular.
  • WP PostViews — Inefficiently writes to the database on every page load. To track traffic in a more scalable manner, both the stats module in Automattic’s Jetpack plugin and Google Analytics work wonderfully.
  • Tweet Blender — Does not play nicely with our caching layer and can cause increased server load.


A good Complete list of known WordPress slow plugins that will hammer down your wordpress performance is here

There are few alternatives to this plugins and when I have some free time I will download and test their alternatives but for now I plan the plugins to stay disabled.
 

For the absolute WP Performance Optimization Freaks, its good to check out the native way to Debug a wordpress installation through using few embedded
variables

 

define('WP_DEBUG', true);
define('WP_DEBUG', false);
define('WP_DEBUG_LOG', true);
define('WP_DEBUG_DISPLAY', false);
define('SAVEQUERIES', true);

 

An article describing how you can use native WP debug variables is here


Happy Optimizing  ! 🙂

Share this on

Improve Website Apache Webserver SEO without Website source code moficitations with Google PageSpeed module on Debian, Ubuntu, CentOS, Fedora and SuSE Linux

Thursday, December 18th, 2014

Improve-website-apache-webserver-seo-without-website-source-code-modifications-with-Google-PageSpeed-Apache-module

For hosting companies and even personal website speed performance becomes increasingly important factor that gives higher and higher weight on overall PageRank and is one of the key things for Successful Site Search Engine Optimization (positioning) in Search Engines of a not specially SEO friendly crafted website.

Virtually all Google / Yahoo / Bing,  Yahoo  etc. Search Engines give better pagerank to websites which load faster and has little or no downtimes, for the reason a faster loading time of a website pages means better user experience and is indicator that the website is well maintained. 

Often websites deployed written for purpose of a business-es or just community CMS / Blog Website Open Source systems such as Joomla, Drupal and WordPress by default are not made to provide fantastic speed right after deploy without install of custom plugins and website tuning, i.e.:

  • Content size optimization (gzipping)
  • More efficient way to deliver CSS / Javascript (MinifyJS / CSS files into single ones
  • HTML optimization
  • Stripping (useful) page Comments
  • Adding <head> if missing on pages etc.

. Therefore as I said in many of my previous LAMP Optimization articles page  (opening) speed could make really Bad Users / Clients experience when the site grows too big or is badly optimized it gives degraded page speed times (often page loads 20 / 30 seconds waiting for the page to load!). Having Pages lagging on big information sites or EShos has both Ruining Company's Image on the market and quickly convinces the user to use another service from the already thosands available and thus drives out (potential) customers.

As Programming code maintainance and improvement is usually very costly, companies that want to save money or can't afford it (because of the shrinking budgets dictacted by the global economic crisis), the best thing to do is to ask your sysadmin to Squeeze the Best out of the WebService and Servers without major (Backend Code) infrastructural changes.

To  Speed up Apache and create Proper Page Caching without installing on server external PHP Caching modules such as Eaccelerator  / PHP APC caching and without
extra CMS modules
such as lets say WordPress W3 Total Cache there is Google Develop Apache Webserver external module – PageSpeed.

Here is Google Pagespeed Module overview :
 

PageSpeed speeds up your site and reduces page load time. This open-source webserver module automatically applies web performance best practices to pages and associated assets (CSS, JavaScript, images) without requiring that you modify your existing content or workflow.


What does Apache Google PageSpeed actually does?
 

  • Automatic website and asset optimization
  • Latest web optimization techniques
  • 40+ configurable optimization filters
  • Free, open-source, and frequently updated
  • Deployed by individual sites, hosting providers, CDNs


1. Install PageSpeed on Debian / Ubuntu, deb derivatives) Linux

a) Download and install module 

On 64 bit deb based Linux:

cd /usr/local/src
wget https://dl-ssl.google.com/dl/linux/direct/mod-pagespeed-stable_current_amd64.deb 
dpkg -i mod-pagespeed-stable_current_amd64.deb
apt-get -f install


On 32 bit Linux:

cd /usr/local/src
wget https://dl-ssl.google.com/dl/linux/direct/mod-pagespeed-stable_current_i386.deb
dpkg -i 
direct/mod-pagespeed-stable_current_i386.deb
apt-get -f install


b) Restart Apache
 

sudo /etc/init.d/apache2 restart

Important files and folders placed on server by deb installer are:

/usr/bin/pagespeed_js_minify – binary that does Javascript minification
/etc/apache2/mods-available/pagespeed.conf – Pagespeed config
/etc/apache2/mods-available/pagespeed.load – Load module directives in Apache
/etc/cron.daily/mod-pagespeed – mod_pagespeed cron script for checking and installing latest updates.
/var/cache/mod_pagespeed – Mod Pagespeed cahing folder (useful to install memcached to increase even further caching performance)
/var/log/pagespeed – Directory to store pagespeed log files

 

2. Install PageSpeed on (RPM based CentOS, Fedora, RHEL / SuSE Linux)


RPM 64 bit package install:
 

rpm -Uvh https://dl-ssl.google.com/dl/linux/direct/mod-pagespeed-beta_current_x86_64.rpm

 


32 bit pack version:
 

rpm -Uvh https://dl-ssl.google.com/dl/linux/direct/mod-pagespeed-stable_current_i386.rpm


Modify pagespeed mod config 

Restart Apache

sudo /etc/init.d/httpd restart


Important config files and folders created during RPM install are:

  • /etc/cron.daily/mod-pagespeed : mod_pagespeed cron script for checking and installing latest updates.
  • /etc/httpd/conf.d/pagespeed.conf : The main configuration file for Apache.
  • /usr/lib/httpd/modules/mod_pagespeed.so : mod_pagespeed module for Apache.
  • /var/www/mod_pagespeed/cache : File caching direcotry for web sites.
  • /var/www/mod_pagespeed/files : File generate prefix for web sites.

3. Configuring Google PageSpeed module

 

To configure PageSpeed you can either edit the package installed bundled pagespeed.conf (/etc/apache2/mods-available/pagspeed.conf,  /etc/httpd/conf.d/pagespeed.conf) or insert configuration items inside Apache VirtualHosts config files or even if you need flexibility and you don't have straight access to Apache config files (on shared hosting servers where module is available) through .htaccess.
Anyways try to avoid adding pagespeed directives to .htaccess as it will be too slow and inefficient.

Configuration is managed by setting different so-called "Rewrite Levels". Default behavior is to use Level of "Corefilters.", a set of filters (module behavior configs) which according to Google is safe for use. PageSpeed Filters is a set of actions applied to Web Delivered files.

Default config setting is hence:
 

ModPagespeedRewriteLevel CoreFilters

Disabling default set of filters is done with:
 

ModPagespeedRewriteLevel PassThrough

"Corefilters" default filter set as of time of writting this article:
 

add_head
combine_css
convert_jpeg_to_progressive
convert_meta_tags
extend_cache
flatten_css_imports
inline_css
inline_import_to_link
inline_javascript
rewrite_css
rewrite_images
rewrite_javascript
rewrite_style_attributes_with_url

Complete documentation on Configuring PageSpeed Filters is here.

If caching is turned on, default PageSped caching is configured in /var/cache/mod_pagespeed/
Enabling someof the non-Corefilters that sometimes are useful for SEO (reduce of served / returned pagesize) are:
 

ModPagespeedEnableFilters pedantic,remove_comments

By default pagespeed does some things (such as inline_css, inline_javascript and rewrite_images (Optimize, removing Excess pixels).  My litle experience with pagespeed shows in some cases this could break websites), so I found for my case useful to disable some of the filters:

 

vim /etc/apache2/mods-available/pagespeed.conf

 

ModPagespeedDisableFilters rewrite_images,convert_jpeg_to_progressive,inline_css,inline_javascript

 

4. Testing if PageSpeed is Enabled pagespeed_admin

By default PageSpeed has Admin which by default is only allowed to be accessed from server localhost (127.0.0.1) to get basic statistics either install text browser like lynx / elinks or add more access IPs again in pagespeed config / vhosts pagespeed.conf include more Allow lines like below:

 

    <Location /pagespeed_admin>
        Order allow,deny
        Allow from localhost
        Allow from 127.0.0.1
        Allow from 192.168.1.1
        Allow from xxx.xxx.xxx.xxx

        #Allow from All
        SetHandler pagespeed_admin
    </Location>
    <Location /pagespeed_global_admin>
        Order allow,deny
        Allow from localhost
        Allow from 127.0.0.1

        Allow from 192.168.1.1
        Allow from xxx.xxx.xxx.xxx
        SetHandler pagespeed_global_admin
    </Location>

 

Once configured pagespeed_admin access it with favourite browser on:

http://127.0.0.1/pagespeed_admin
http://127.0.0.1/pagespeed_global_admin

improve-website-apache-webserver-seo-without-source-code-modifications-google-pagespeed_admin_panel

Other way to test it is enabled is by creating php file with good old <? phpinfo(); ?> – PHP stats enabled / disabled features code:

pagespeed-in-phpinfo-x-mod-pagespeed-output-screenshot-apache-webserver

I've also tested also pagespeed unstable release, but experienced some segmentation faults in both error.log and access.log so finally decided to keep using stable release.

PageSpeed is a great way to boost your server sites performance, however it comes on certain costs as expect your server CPU Load to jump drastically, (in my case it jumped more than twice), there are Linux servers where enabling the module could totally stone the servers, so before implementing the module on a Production system environment, always first test thouroughfully with loaded pagespeed on UAT (testing) environment with AB or Siege (Apache Benchmarking Tools).

Share this on

xkill for Windows – Kill hanged programs with one click like in UNIX

Thursday, September 18th, 2014

Windows Xkill by Solo Dev logo kill easily hanged, crashed windows programs - Linux xkill for Windows alternative

I’ve used Linux as I’ve used it over the last 10 years and thus I’m so used to xkill that I consider it as a normal tool every modern operating system should have.

Since I’m forced to work on a Windows platform over the last 1 year, every now and then I have crashing / hanged apps Window which sometimes is hard to kill using Task Manager or command line tools like tasklist, taskkill or pstools – Windows Sysinternal tools cause you don’t know the exact process name of the Windows crashed application.

Thanksfully a good hearted guy SuprVillain made Windows Xkill program. Windows Xkill is portable app, so you don’t need to install it but simply download and run it.

windows-7-system-tray-xkill-screenshot-xkill-for-windows

Win-Xkill runs in system tray and has “kill mode” of operation, when Kill mode it is running Windowns Xkill operates exactly like UNIX’s xkill. The mouse pointer turns into Skull and Cross Bones, you point at a Window app you wanted to kill and it gets terminated.
Windows Xkill kill mode enabled killing a windows notepad app
Win Xkill kill mode, can be invoked using also a key press Control+Alt+Backspace as well as there is an option to leave xkill running in the background background, but disabling the tiny skull icon in systray from its interface.
Enjoy 🙂

Share this on

mod_rewrite redirect rule 80 to 443 on Apache webserver

Wednesday, April 2nd, 2014

A classic sysadmin scenario is to configure new Apache webserver with requirement to have an SSL ceriticate installed and working on port 443 and all requests coming on port 80 to be redirected to https://.
On Apache this is done with simple mod_rewrite rule:

RewriteEngine On
RewriteCond %{HTTPS} off
RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}

Before applying the rule don't forget to have Apache mod_rewrite enabled usually it is not enabled on default most Linux distributions by default.
On shared hostings if you don't have access to directly modify Apache configuration but have .htaccess enabled you can add above rules also to .htaccess

Add this to respective VirtualHost configuration and restart Apache and that's it. If after configuring it for some reason it is not working debug mod_rewrite issues by enabling mod_rewrite's rewrite.log

Other useful Apache mod_rewrite redirect rule is redirect a single landing page from HTTP to HTTP

RewriteEngine On
RewriteRule ^apache-redirect-http-to-https.html$ https://www.site-url.com/apache-redirect-http-to-https.html [R=301,L]

!Note! that in case where performance is a key requirement for a website it might be better to use the standard way to redirect HTTP to HTTPS protocol in Apache through:

ServerName www.site-url.com Redirect / https://www.site-url.com/

To learn more on mod_rewrite redirecting  check out this official documentation on Apache's official site.

Share this on

Apache SSLCertificateChainFile adding SSL with Certificate Chain / What is Certificate Chain

Friday, January 31st, 2014

configure-apache-ssl-certificate-chain-ssl-certificate-keychain-each-signing-each-other

If you work in a big company with large network infrastructure who has to deal with SSL Certificates you will sooner or later will have to learn about existence of SSL Certificate Chains.
Its worthy thus to know what is SSL Certificate Chains and how such a chain is configured in Apache?

Personal SSL certificates (certificates issued to an individual or a company) can be used by clients to uniquely identify themselves when they are involved in starting an SSL connection.
SSL Certificate file contains X.509 certificate, which, in turn, contains a public key used for encryption.
Each personal certificate has zero or more certificate chains of certification authority certificates that extend back to the root certification authority.
 

Certificate R (Root Certification Authority)
    |
    | represents issuer of
    V
Certificate I1 (Intermediate Certification Authority)
    |
    | represents issuer of
    V
Certificate I2 (A subsidiary Intermediate Certification Authority)
    |
    | represents issuer of
    V
Certificate I3 (A further subsidiary Intermediate Certification Authority)
    |
    | represents issuer of
    V
Certificate P (A personal certificate that is used to identify its owner
               on an SSL handshake)

Certificate chains are used to verify the authenticity of each certificate in that chain, including the personal certificate. Each certificate in the chain is validated using its 'parent' certificate, which in turn is validated using the next certificate up the chain, and so on, from the personal certificate up to the root certification authority certificate.

Now after explaining thoroughfully what is SSL Certificate Chain, here is how to configure a SSL Certificate in Apache Webserver.

Open apache2.conf or httpd.conf (depending on GNU / Linux distribution) and add to it;

  SSLEngine On
   SSLCertificateFile conf/cert/webserver-host.crt
   SSLCertificateKeyFile conf/cert/webserver-host.key
   SSLCertificateChainFile conf/cert/internet-v4.crt
   # SSLCertificateChainFile conf/cert/intranet-v3.crt
   SSLOptions +StdEnvVars +OptRenegotiate +ExportCertData

SSLCertificateChainFile conf/cert/chain-cert.crt
loads a chain of separate Personal SSL certificates each signing each other on different levels, chain is leading to top ROOT CA (Certificate Authority).

Share this on

Adding Listing and Deleting SSL Certificates in keystore Tomcat Application server / What is keystore

Thursday, December 5th, 2013

Apache Tomcat keystore delete import list logo

 I work on ongoing project where Tomtat Application servers configured to run Clustered located behind Apache with mod_proxy configured to use ReverseProxy are used. One of customers which required a java application deployment experienced issues with application's capability to connect to SAP database.

After some investigation I figured out, the application is unable to connect to the SAP db server becuse remote host webserver running some SAP related stuff was not connecting due to expired certificate in Tomcat Keystore known also as JKS / Java Keystore– (.keystore) – which is a file containing multiple remote hosts imported certificates.

The best and shortest definition of keystore is:

Keystore entry = private + public key pair = identified by an alias

The keystore protects each private key with its individual password, and also protects the integrity of the entire keystore with a (possibly different) password.

Managing Java imported certificates later used by Tomcat is done with a command line tool part of JDK (Java Development Kit) called keystore. Keystore is usually located under /opt/java/jdk/bin/keytool. My Java VM is installed in /opt/ anyways usual location of keytool is $JAVA_HOME/bin/

Keytool has capabilities to create / modify / delete or import new SSL certificates and then Java applications can access remote applications which requires Secure Socket Layer handshake . Each certificate kept in .keystore file (usually located somewhere under Tomcat web app server directory tree), lets say – /opt/tomcat/current/conf/.keystore

1. List current existing imported SSL certificates into Java's Virtual Machine

tomcat-server:~# /opt/java/jdk/bin/keytool -list -keystore /opt/tomcat/current/conf/.keystore
password:
Command returns output similar to;

Entry type: trustedCertEntry

Owner: CN=www.yourhost.com, OU=MEMBER OF E.ON GROUP, OU=DEVICES, O=E.GP AG, C=DE
Issuer: CN=E.ON Internal Devices Sub CA V2, OU=CA, O=EGP, C=DE
Serial number: 67460001001c6aa51fd25c0e8320
Valid from: Mon Dec 27 07:05:33 GMT 2010 until: Fri Dec 27 07:05:22 GMT 2013
Certificate fingerprints:
         MD5:  D1:AA:D5:A9:A3:D2:95:28:F1:79:57:25:D3:6A:16:5E
         SHA1: 73:CE:ED:EC:CA:18:E4:E4:2E:AA:25:58:E0:2B:E4:D4:E7:6E:AD:BF
         Signature algorithm name: SHA1withRSA
         Version: 3

Extensions:

#1: ObjectId: 2.5.29.15 Criticality=true
KeyUsage [
  DigitalSignature
  Key_Encipherment
  Key_Agreement
]

#2: ObjectId: 2.5.29.19 Criticality=true
BasicConstraints:[
  CA:false
  PathLen: undefined
]

#3: ObjectId: 1.3.6.1.5.5.7.1.1 Criticality=false
AuthorityInfoAccess [
  [
   accessMethod: 1.3.6.1.5.5.7.48.2
   accessLocation: URIName: http://yourhost.com/cacerts/egp_internal_devices_sub_ca_v2.crt,
   accessMethod: 1.3.6.1.5.5.7.48.2
   accessLocation: URIName: http://www.yourhost1.com/certservices/cacerts/egp_internal_devices_sub_ca_v2.crt]
]

#4: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: D3 52 C7 63 0F 98 BF 6E   FE 00 56 5C DF 35 62 22  .R.c…n..V\.5b"
0010: F2 B9 5B 8F                                        ..[.
]

Note that password that will be promtped has is by default changeit (in case if you don't have explicitly changed it from Tomcat's default config server.xml).

2. Delete Old expired SSL host Certificate from Java Keystore
It is good practice to always make backup of old .keystore before modifying, so I ran:

tomcat-server:~# cp -rpf /opt/tomcat/current/conf/.keystore /opt/tomcat/current/conf/.keystore-05-12-2013

In my case first I had to delete old expired SSL certificate with:

tomcat-server:~# /opt/java/jdk/bin/keytool -delete -alias "your-hostname" -v -keystore /opt/tomcat/current/conf/.keystore

Then to check certificate is no longer existent in keystore chain;
tomcat-server:~# /opt/java/jdk/bin/keytool -list -keystore /opt/tomcat/current/conf/.keystore

-keystore – option is obligitory it does specify where keystore file is located
-list – does list the certificate
-v – stands for verbose

 

3. Finally to import new SSL from already expored via a browser url in keystore

tomcat-server:~# /opt/java/jdk/bin/keytool -importcert -file /tmp/your-hostname.cer -alias your-hostname.com -keystore /opt/tomcat/current/conf/.keystore

More complete information on how to deal with keystore is available from Apache Tomcat's SSL Howto – a must read documentation for anyone managing Tomcat.

Share this on

How to generate self signed SSL certificate with openssl on Linux / BSD

Wednesday, November 6th, 2013

How to generate self signed SSL certificate with openssl on Linux / BSD

It is common solution for personal use to generate SSL certificates which are self-signed. Self-signed certificates are dangerous as no authority or company guarantees that remote site is trustable. However for private use having encrypted connection whether you need to transfer personal data is better than not having such. There are plenty of tutorials online pointing how to set-up Apache webserver to provide access via SSL port 443 with self-signed certifacate, but anyways I decided to blog here a one-liner command way, which makes generating self-signed certificate a piece of cake. Self-signed certificates on UNIX are generated with openssl command part of openssl (Secure Socket Layer cryptocgraphic tools).

On Debian Linux to install openssl (if you still don't have it):

apt-get install --yes openssl

On Fedora, RHEL, CentOS etc. same install should be done with:

yum install -y openssl

On FreeBSD to install it use ports;

cd /usr/ports/security/openssl
make install clean

Once openssl is available, here is command to generate self signed SSL certitifacate;
 

openssl req -x509 -nodes -days 3650 -newkey rsa:2048 -keyout my-sitename.key -out my-sitename.crt
Generating a 2048 bit RSA private key
............................................+++
..............................................................+++
writing new private key to 'key.pem'
Enter PEM pass phrase:
Verifying - Enter PEM pass phrase:
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:BG
State or Province Name (full name) [Some-State]:Sofia
Locality Name (eg, city) []:Sofia
Organization Name (eg, company) [Internet Widgits Pty Ltd]:Pc Freak
Organizational Unit Name (eg, section) []:Pc Freak
Common Name (eg, YOUR name) []:www.pc-freak.net
Email Address []:testing@pc-freak.net

The generated certificate Private Key file is placed in my-sitename.key
, actual certificate is located in my-sitename.crt -days option tells for how long period certificate will be valid. Regenerating certificate every year (360 days) is good security practice but it is pretty annoying to see your certificate has expered thus for private self signed SSL certificate it is more confortable to generate cert for 10 years time.
To use my-sitename.key and my-sitename.crt copy them to /etc/ssl/crt/

cp -rpf my-sitename.crt /etc/ssl/crt/
cp -rpf my-sitename.key /etc/ssl/crt/

Next what's left is to configure Apache to use new generated certs. Quickest way is to add it inside virtual host. Adding to Apache also depends on Linux distribution where SSL has to be added but in general, what should work is something like:

SSLEngine on
SSLCertificateFile /etc/ssl/crt/my-sitename.crt
SSLCertificateKeyFile /etc/ssl/crt/my-sitename.key

Note that if SSL has to be enabled for specific Virtual Host you will have to place above Apache directives into the Vhost. Though certifiate will only be trusted by your authority RSA 2048 bit encryption in transfer data between your Webserver and Browser should guarantee you extra security of data, not that it is impossible for data to be still sniffed by a skilled hacker but makes data securily transferred in probably 99% of cases 🙂

Share this on

12 Lessons Steve Jobs Taught Guy Kawasaki – SEO Summit Guy Kawasaki speech

Monday, July 30th, 2012

I'm not a big fan of Steve Jobs, neither I like the cult that is nowdays to his personality. After his recent death the cult to Jobs and his works has bloomed once again. From philosophical point of view I don't like Jobs ideas that there is no good and bad but only success matters. However I should admit as a SEO and as a business enterpreneur his achievements are significant. Hence I decided to share with you a video of SEO Summit ex-employee Guy Kawasaki, who "worked for jobs twice and survived". Jobs is famous for not being loved too much by his employees. Also it is not a secret historically he screw up, Steve Wozniak and a number of other people who were either employeed or worked in any way with him.

The points his ex-employee KAWASAKI shares on this SEO Summit presentation are quite interesting and are things, beginning business enterpreneurs like me could learn tremendously from. One key point that is hardly underlined in the presentation is the importance of simplicity.

* Simplicity in everything is essential for the success. It was quite curious to me Steve Jobs presentations were consisting often with a slides of just one word. Obviously this means Jobs was a simplicity freak.

Some of other concepts of Jobs was:
 

Either it works or it doesn't work.

You see again his trend to be simplifying things. In business we all know the SMART (Keep it Simple And Stupid). It seems Jobs SMART was only SAS (Simple and Stupid) 🙂 ….

Some other things Kawasaki learned from Jobs was:
 

Never believe Experts

– If someone tells you he is an expert in something he is definitely not ….

Another belief (understanding) of Jobs and probably many other succesful enterpreneurs is that DESIGN COUNTS. Design is one of the most crucial points in any product, so one has to be extra-careful here. A fail in design is a fail in product line ….

There is plenty of other interesting stuff in the video but the key point is SIMPLICITY. Enjoy Kawasaki Speech …

12 Lessons Steve Jobs Taught Guy Kawasaki

 

Share this on

How to check MASTER / SLAVE MySQL nodes status – Check MySQL Replication Status

Thursday, April 19th, 2012

I'm doing replication for one server. Its not the first time I do configure replication between two MySQL database nodes, however since I haven't done it for a few years, my "know how" has mostly vanished so I had some troubles in setting it up. Once I followed some steps to configure replication I had to check if the two MASTER / Slave MySQL db nodes communicate properly. Hence I decided to drop a short post on that just in case if someone has to do the same or if I myself forget how I did it so I can check later on:

1. Check if MASTER MySQL server node is configured properly

The standard way to check a MySQL master node status info is with:
 

mysql> show master status;
+——————+———-+———————————————————+——————+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+——————+———-+———————————————————+——————+
| mysql-bin.000007 | 106 | database1,database2,database3 | |
+——————+———-+———————————————————+——————+
1 row in set (0.00 sec)

By putting \G some extra status info is provided:
 

mysql> show master status\G;
*************************** 1. row ***************************
File: mysql-bin.000007
Position: 106
Binlog_Do_DB: database1,database2,database3
Binlog_Ignore_DB:
1 row in set (0.00 sec)

ERROR:
No query specified

2. Check if Slave MySQL node is configured properly

To check status of the slave the cmd is:
 

mysql> show slave status;

The command returns an output like:
 

mysql> show slave status;+———————————-+————-+————-+————-+—————+——————+———————+————————-+—————+———————–+——————+——————-+——————————————————-+———————+——————–+————————+————————-+—————————–+————+————+————–+———————+—————–+—————–+—————-+—————+——————–+——————–+——————–+—————–+——————-+—————-+———————–+——————————-+—————+—————+—————-+—————-+| Slave_IO_State | Master_Host | Master_User | Master_Port | Connect_Retry | Master_Log_File | Read_Master_Log_Pos | Relay_Log_File | Relay_Log_Pos | Relay_Master_Log_File | Slave_IO_Running | Slave_SQL_Running | Replicate_Do_DB | Replicate_Ignore_DB | Replicate_Do_Table | Replicate_Ignore_Table | Replicate_Wild_Do_Table | Replicate_Wild_Ignore_Table | Last_Errno | Last_Error | Skip_Counter | Exec_Master_Log_Pos | Relay_Log_Space | Until_Condition | Until_Log_File | Until_Log_Pos | Master_SSL_Allowed | Master_SSL_CA_File | Master_SSL_CA_Path | Master_SSL_Cert | Master_SSL_Cipher | Master_SSL_Key | Seconds_Behind_Master | Master_SSL_Verify_Server_Cert | Last_IO_Errno | Last_IO_Error | Last_SQL_Errno | Last_SQL_Error |+———————————-+————-+————-+————-+—————+——————+———————+————————-+—————+———————–+——————+——————-+——————————————————-+———————+——————–+————————+————————-+—————————–+————+————+————–+———————+—————–+—————–+—————-+—————+——————–+——————–+——————–+—————–+——————-+—————-+———————–+——————————-+—————+—————+—————-+—————-+| Waiting for master to send event | HOST_NAME.COM | slave_user | 3306 | 10 | mysql-bin.000007 | 106 | mysqld-relay-bin.000002 | 251 | mysql-bin.000007 | Yes | Yes | database1,database2,database3 | | | | | | 0 | | 0 | 106 | 407 | None | | 0 | No | | | | | | 0 | No | 0 | | 0 | |+———————————-+————-+————-+————-+—————+——————+———————+————————-+—————+———————–+——————+——————-+——————————————————-+———————+——————–+————————+————————-+—————————–+————+————+————–+———————+—————–+—————–+—————-+—————+——————–+——————–+——————–+—————–+——————-+—————-+———————–+——————————-+—————+—————+—————-+—————-+

As you can see the output is not too readable, as there are too many columns and data to be displayed and this doesn't fit neither a text console nor a graphical terminal emulator.

To get more readable (more verbose) status for the SQL SLAVE, its better to use command:
 

mysql> show slave status\G;

Here is a sample returned output:
 

mysql> show slave status\G;*************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: HOST_NAME.COM Master_User: slave_user Master_Port: 3306 Connect_Retry: 10 Master_Log_File: mysql-bin.000007 Read_Master_Log_Pos: 106 Relay_Log_File: mysqld-relay-bin.000002 Relay_Log_Pos: 251 Relay_Master_Log_File: mysql-bin.000007 Slave_IO_Running: Yes Slave_SQL_Running: Yes Replicate_Do_DB: database1,database2,database3 Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 106 Relay_Log_Space: 407 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: 0Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: 1 row in set (0.00 sec)ERROR: No query specified

If show master status or shwo slave status commands didn't reveal replication issue, one needs to stare at the mysql log for more info.

Share this on

Don’t revoke GoDaddy SSL certificate. (Expired) Revoked SSL is impossible to revert

Thursday, April 12th, 2012

godaddy_logo
One of our company SSL (https) Certificates recently expired so I needed to renew the SSL certificate.

I was in a hurry doing plenty of other stuffs so it seemed logical for me to Revoke the Certificate. I thought revoking the certificate will simply cancel it and afterwards, in Godaddy's SSL (Manager Certificates) interface the Revoked – Cancelled certificate will re-appear in the menu, ready to be generated in the same way as earlier I initially generated the Godaddy's bought SSL certificate

Hence I proceeded and used Revoke button:

Godaddy SSL certificate manager browser certificate Screenshot

Well guess what my calculations, were wrong.
 Revoking, just cancel it. The  revoked domain SSL certificate did not show up again in Godaddy's Cert Manager and I have no way from their interface to revert the changes.

To deal with the situation, I contacted Godaddy Support immediately with the following inquiry:
 

Other : Revoked SSL Certificate
Issue :
Hello we have revoked the SSL certificate for our domain our.domain-name.com.
 Can we revert back the certificate as it was.
If not how to generate a new key for our domain https://our.domain-name.com
Thanks in Advance.
Kindest Regards

"My-Company-name" Tech Support

In 5 hours time I received the following tech support answer:
 

Dear Tech Support,

Thank you for contacting Online Support. It is not possible to reinstate a canceled certificate. You will need to purchase a new certificate. I have requested that a refund be applied to your account. Once the credit appears in your account, please allow 5-7 business days to see the funds applied to the associated payment method. Thank you for your patience and understanding in this matter.

Please let us know if we can help you in any other way.

Sincerely,

Christian P.
Online Support Team
Customer Inquiry
Name : Cadia Tech Support
Domain Name : our.domain-name.com
ShopperID : xxxxxxxxx
Phone : xxxxxxxx
Shopper Validated : Yes
Browser : Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3

Apparently Godaddy, can work out a bit on their tech support answering time 5 hours for a simple reply is quite long.

Now taking in consideration, above reply from Godady, my only options are to either wait for 5 to 7 (business days) or buy a new credit for SSL certificate.

Buying a new credit will probably not happen as our company is experiencing some financial troubles because of the crisis. So I guess we will have to wait for this 7 days at worst. So again if you wonder to REVOKE or not an SSL certificate. Think again …

Just a small note to make here, that Godaddy has a very straight forward way to just renew an expered certificate, which I succesfully later have done for 4 domains. Well, if only I knew earlier what REVOKE SSL cert really does I wouldn't have ended in this mess …

Share this on