Posts Tagged ‘Wordpress’

How to merge two or more wordpress sites from separete domains into one without impacting SEO

Monday, October 23rd, 2017

merge-multiple-wordpres-websites-howto-merge-two-or-more-websites-together-without-loosing-seo-credability

If you run multiple WordPress websites or blogs, but one of the websites is much more developed at a certain point you might decide that the best of the two worlds would be two merge two WordPress blogs into one.
 

So how can we do merge your multiple wordpress websites into single ?


1. Export content from each of the old websites you will be closing soon

First thing to do is to immediately create SQL data and WordPress sites backup, so if something goes wrong you can easily restore.

 


Next From wordpress admin area wp-admin

export-content-wordpress-website-screenshot-howto

Visit

Tools -> Export

Make sure, the All Content option is selected like shown in above screenshot and click on

Download Export File button

The downloaded files will have a copy of all yours

– Posts and pages
– Custom Post Types
– All included categories and tags

Once we have transferred the file to the Website with which we're going to merge old WP content, we need to install the Import plugin

install-wordpress-importer-to-merge-multiple-wordpress-sites

To do so navigate to:

Tools -> Import -> (Intall now)

wordpress-merge-two-or-more-websites-run-importer-screenshot.

Tools -> Run Importer

You will be asked for the file you just dumped into the second wordpress install which you'll be merging

export-and-import-wp-file-screenshot

Wordpress will start importing, your Export file and analyze it, then you will be redirected to

Import Settings Page
 

wordpress-import-settings-page-screenshot-howto-merge-sites

Assign Authors (that should be your account or the account with which the primary blog has most posts and with which you'll be merging).

Hopefully all will be fine with import and you will get the success page:

wordpress-success-page-importing-two-websites

One last step is to set properly redirects so that the transition merge from the Old Website to the New one does not impact your website Search Engine Optimization Ratings.

The redirects are made via .htaccess file it will be located in the Document Root (Or main directory) of your wordpress installed websites for example if your website is located under /var/www/wordpress-site your .htaccess file will be located under /var/www/wordpress-site/.htaccess

Open the file (SITUATED ON THE OLD WORDPRESS WEBSITE THAT YOU'LL BE MERGING) with editor directly on the server or if it is a shared hosting, download it and edit it at your side.

To the beginning of it add the following Apache ModRewrite code:
 

#Options +FollowSymLinks
RewriteEngine on
RewriteRule ^(.*)$ http://www.newsite.com/$1 [R=301,L]


Replace inside above example http://www.newsite.com with the name of the primary domain name to which we're merging (e.g. the merging domain and not the one we merge).

If you experience some problems with pictures or you like to Merge Bulk Edit categories  and tags  read import external images to wordpress as well as how to merge bulk edit categories in tags in WordPress

 

List of vulnerable wordpress plugins. Hacked, dangerous, vulnerable

Tuesday, October 17th, 2017

list-of-vulnerable-wordpress-pluginshacked-dangerous-vulnerable-wp

 

Have your wordpress has been hacked recently? Mine has Don't despair, below is a list of famous WordPress Plugins for its hackability.
Hope this helps you prevent your self on time and wipe out all the unnecessery plugins.
Double check the version number of Vulnerable plugins, and remove it only when you're sure its hackable. If you're sure you happen to run on your WordPress Blog or site one of the below plugins immediately deactivate and delete it.

 

Vulnerability types

A quick reminder of the most common security holes and issues WordPress plugins face. Please note that most problems are a combination of two or more types listed below.

Arbitrary file viewing
Instead of allowing only certain file source to be viewed (for example plugin templates) the lack of checks in the code allows the attacker to view the source of any file, including those with sensitive information such as wp-config.php

Arbitrary file upload
Lack of file type and content filtering allows for upload of arbitrary files that can contain executable code which, once run, can do pretty much anything on a site

Privilege escalation
Once the attacker has an account on the site, even if it’s only of the subscriber type, he can escalate his privileges to a higher level, including administrative ones.

SQL injection
By not escaping and filtering data that goes into SQL queries, malicious code can be injected into queries and data deleted, updated or inserted into the database. This is one of the most common vulnerabilities.

Remote code execution (RCE)
Instead of uploading and running malicious code, the attacker can run it from a remote location. The code can do anything, from hijacking the site to completely deleting it.

Plugin Name Vulnerability Type Min / Max Versions Affected
1 Flash Gallery arbitrary file upload 1.3.0 / 1.5.6
360 Product Rotation arbitrary file upload 1.1.3 / 1.2.0
Tevolution arbitrary file upload 2.0 / 2.2.9
Addblockblocker arbitrary file upload 0.0.1
Ads Widget remote code execution (RCE) 2.0 / n/a
Advanced Access Manager privilege escalation 3.0.4 / 3.2.1
Advanced Ajax Page Loader arbitrary file upload 2.5.7 / 2.7.6
Advanced Video Embed Embed Videos Or Playlists arbitrary file viewing n/a / 1.0
Analytic remote code execution (RCE) 1.8
Analytics Counter PHP object injection 1.0.0 / 3.4.1
Appointments PHP object injection 1.4.4 Beta / 2.2.0
Asgaros Forum settings change 1.0.0 / 1.5.7
Aspose Cloud Ebook Generator arbitrary file viewing 1.0
Aspose Doc Exporter arbitrary file viewing 1.0
Aspose Importer Exporter arbitrary file viewing 1.0
Aspose Pdf Exporter arbitrary file viewing 1.0
Attachment Manager arbitrary file upload 1.0.0 / 2.1.1
Auto Attachments arbitrary file upload 0.2.7 / 0.3
Bbpress Like Button SQL injection 1.0 / 1.5
Bepro Listings arbitrary file upload 2.0.54 / 2.2.0020
Blaze Slide Show For WordPress arbitrary file upload 2.0 / 2.7
Brandfolder local file inclusion (LFI) 2.3 / 3.0
Breadcrumbs Ez remote code execution (RCE) n/a
Candidate Application Form arbitrary file viewing 1.0
Category Grid View Gallery arbitrary file upload 0.1.0 / 0.1.1
Cherry Plugin arbitrary file upload 1.0 / 1.2.6
Chikuncount arbitrary file upload 1.3
Cip4 Folder Download Widget arbitrary file viewing 1.4 / 1.10
Cms Commander Client PHP object injection 2.02 / 2.21
Contus Video Gallery arbitrary file viewing 2.2 / 2.3
Cookie Eu remote code execution (RCE) 1.0
Cp Image Store arbitrary file viewing 1.0.1 / 1.0.5
Cross Rss arbitrary file viewing 0.5
Custom Content Type Manager remote code execution 0.9.8.8
Custom Lightbox possible remote code execution (RCE) 0.24
Cysteme Finder arbitrary file viewing 1.1 / 1.3
Db Backup arbitrary file viewing 1.0 / 4.5
Delete All Comments arbitrary file upload 2.0
Developer Tools arbitrary file upload 1.0.0 / 1.1.4
Disclosure Policy Plugin remote file inclusion (RFI) 1.0
Display Widgets remote code execution 2.6
Dop Slider arbitrary file upload 1.0
Download Zip Attachments arbitrary file viewing 1
Downloads Manager arbitrary file upload 1.0 Beta / 1.0 rc-1
Dp Thumbnail arbitrary file upload 1.0
Dropbox Backup PHP object injection 1.0 / 1.4.7.5
Dukapress arbitrary file viewing 2.3.7 / 2.5.3
Ebook Download arbitrary file viewing 1.1
Ecstatic arbitrary file upload 0.90 (x9) / 0.9933
Ecwid Shopping Cart PHP Object Injection 3.4.4 / 4.4.3
Enable Google Analytics remote code execution (RCE) n/a
Estatik arbitrary file upload 1.0.0 / 2.2.5
Event Commerce Wp Event Calendar persistent cross-site scripting (XSS) 1.0
Filedownload arbitrary file viewing 0.1
Flickr Gallery PHP object injection 1.2 / 1.5.2
Form Lightbox option update 1.1 / 2.1
Formidable information disclosure 1.07.5 / 2.0.07
Fresh Page arbitary file upload .11 / 1.1
Front End Upload arbitrary file upload 0.3.0 / 0.5.3
Front File Manager arbitrary file upload 0.1
Fs Real Estate Plugin SQL injection 1.1 / 2.06.03
G Translate remote code execution (RCE) 1.0 / 1.3
Gallery Objects SQL injection 0.2 / 0.4
Gallery Slider remote code execution (RCE) 2.0 / 2.1
Genesis Simple Defaults arbitrary file upload 1.0.0
Gi Media Library arbitrary file viewing 1.0.300 / 2.2.2
Google Analytics Analyze remote code execution (RCE) 1.0
Google Document Embedder SQL injection 2.5 / 2.5.16
Google Maps By Daniel Martyn remote code exection (RCE) 1.0
Google Mp3 Audio Player arbitrary file viewing 1.0.9 / 1.0.11
Grapefile arbitrary file upload 1.0 / 1.1
Gravityforms reflected cross-site scripting (XSS) 1.7 / 1.9.15.11
Hb Audio Gallery Lite arbitrary file viewing 1.0.0
History Collection arbitrary file viewing 1.1. / 1.1.1
Html5avmanager arbitrary file upload 0.1.0 / 0.2.7
I Dump Iphone To WordPress Photo Uploader arbitrary file upload 1.1.3 / 1.8
Ibs Mappro arbitrary file viewing 0.1 / 0.6
Image Export arbitrary file viewing 1.0.0 / 1.1.0
Image Symlinks arbitrary file upload 0.5 / 0.8.2
Imdb Widget arbitrary file viewing 1.0.1 / 1.0.8
Inboundio Marketing arbitrary file upload 1.0.0 / 2.0
Infusionsoft arbitrary file upload 1.5.3 / 1.5.10
Inpost Gallery local file inclusion (LFI) 2.0.9 / 2.1.2
Invit0r arbitrary file upload 0.2 / 0.22
Is Human remote code execution 1.3.3 / 1.4.2
Iwp Client PHP object injection 0.1.4 / 1.6.0
Jssor Slider arbitrary file upload 1.0 / 1.3
Like Dislike Counter For Posts Pages And Comments SQL injection 1.0 / 1.2.3
Mac Dock Gallery arbitrary file upload 1.0 / 2.7
Magic Fields arbitrary file upload 1.5 / 1.5.5
Mailchimp Integration remote code execution (RCE) 1.0.1 / 1.1
Mailpress local file inclusion (LFI) 5.2 / 5.4.6
Mdc Youtube Downloader arbitrary file viewing 2.1.0
Menu Image malicious JavaScript loading 2.6.5 / 2.6.9
Miwoftp arbitrary file viewing 1.0.0 / 1.0.4
Mm Forms Community arbitrary file upload 1.0 / 2.2.6
Mobile App Builder By Wappress arbitrary file upload n/a / 1.05
Mobile Friendly App Builder By Easytouch arbitrary file upload 3.0
Multi Plugin Installer arbitrary file viewing 1.0.0 / 1.1.0
Mypixs local file inclusion (LFI) 0.3
Nmedia User File Uploader arbitrary file upload 1.8
Option Seo remote code execution (RCE) 1.5
Page Google Maps remote code execution (RCE) 1.4
Party Hall Booking Management System SQL injection 1.0 / 1.1
Paypal Currency Converter Basic For Woocommerce arbitrary file viewing 1.0 / 1.3
Php Analytics arbitrary file upload n/a
Pica Photo Gallery arbitrary file viewing 1.0
Pitchprint arbitrary file upload 7.1 / 7.1.1
Plugin Newsletter arbitrary file viewing 1.3 / 1.5
Post Grid file deletion 2.0.6 / 2.0.12
Posts In Page authenticated local file inclusion (LFI) 1.0.0 / 1.2.4
Really Simple Guest Post local file inclusion (LFI) 1.0.1 / 1.0.6
Recent Backups arbitrary file viewing 0.1 / 0.7
Reflex Gallery arbitrary file upload 1.0 / 3.0
Resume Submissions Job Postings arbitrary file upload 2.0 / 2.5.3
Return To Top remote code execution (RCE) 1.8 / 5.0
Revslider arbitrary file viewing 1.0 / 4.1.4
S3bubble Amazon S3 Html 5 Video With Adverts arbitrary file viewing 0.5 / 0.7
Sam Pro Free local file inclusion (LFI) 1.4.1.23 / 1.9.6.67
Se Html5 Album Audio Player arbitrary file viewing 1.0.8 / 1.1.0
Sell Downloads arbitrary file viewing 1.0.1
Seo Keyword Page remote code execution (RCE) 2.0.5
Seo Spy Google WordPress Plugin arbitrary file upload 2.0 / 2.6
Seo Watcher arbitrary file upload 1.3.2 / 1.3.3
Sexy Contact Form arbitrary file upload 0.9.1 / 0.9.8
Share Buttons Wp remote code execution (RCE) 1.0
Showbiz arbitrary file viewing 1.0 / 1.5.2
Simple Ads Manager information disclosure 2.0.73 / 2.7.101
Simple Download Button Shortcode arbitrary file viewing 1.0
Simple Dropbox Upload Form arbitrary file upload 1.8.6 / 1.8.8
Simple Image Manipulator arbitrary file viewing 1.0
Simplr Registration Form privilege escalation 2.2.0 / 2.4.3
Site Import remote page inclusion 1.0.0 / 1.2.0
Slide Show Pro arbitrary file upload 2.0 / 2.4
Smart Slide Show arbitrary file upload 2.0 / 2.4
Smart Videos remote code execution (RCE) 1.0
Social Networking E Commerce 1 arbitrary file upload 0.0.32
Social Sharing possible arbitrary file upload 1.0
Social Sticky Animated remote code execution (RCE) 1.0
Spamtask arbitrary file upload 1.3 / 1.3.6
Spicy Blogroll local file inclusion (LFI) 0.1 / 1.0.0
Spotlightyour arbitrary file upload 1.0 / 4.5
Stats Counter PHP object injection 1.0 / 1.2.2.5
Stats Wp remote code execution 1.8
Store Locator Le unrestricted email sending 2.6 / 4.2.56
Tera Charts reflected cross-site scripting (XSS) 0.1 / 1.0
The Viddler WordPress Plugin cross-site request forgery (CSRF)/cross-site scripting (XSS) 1.2.3 / 2.0.0
Thecartpress local file inclusion (LFI) 1.1.0 / 1.1.5
Tinymce Thumbnail Gallery arbitrary file viewing v1.0.4 / v1.0.7
Ultimate Product Catalogue arbitrary file upload 1.0 / 3.1.1
User Role Editor privilege escalation 4.19 / 4.24
Web Tripwire arbitrary file upload 0.1.2
Webapp Builder arbitrary file upload 2.0
Website Contact Form With File Upload arbitrary file upload 1.1 / 1.3.4
Weever Apps 20 Mobile Web Apps arbitrary file upload 3.0.25 / 3.1.6
Woocommerce Catalog Enquiry arbitrary file upload 2.3.3 / 3.0.0
Woocommerce Product Addon arbitrary file upload 1.0 / 1.1
Woocommerce Products Filter authenticated persistent cross-site scripting (XSS) 1.1.4 / 1.1.4.2
Woopra arbitrary file upload 1.4.1 / 1.4.3.1
WordPress File Monitor persistent cross-site scripting (XSS) 2.0 / 2.3.3
Wp Appointment Schedule Booking System persistent cross-site scripting (XSS) 1.0
Wp Business Intelligence Lite arbitrary file upload 1.0 / 1.0.7
Wp Crm arbitrary file upload 0.15 / 0.31.0
Wp Custom Page arbitrary file viewing 0.5 / 0.5.0.1
Wp Dreamworkgallery arbitrary file upload 2.0 / 2.3
Wp Easybooking reflected cross-site scripting (XSS) 1.0.0 / 1.0.3
Wp Easycart authenticated arbitrary file upload 1.1.27 / 3.0.8
Wp Ecommerce Shop Styling authenticated arbitrary file viewing 1.0 / 2.5
Wp Editor authenticated arbitrary file upload 1.0.2 / 1.2.5.3
Wp Filemanager arbitrary file viewing 1.2.8 / 1.3.0
Wp Flipslideshow persistent cross-site scripting (XSS) 2.0 / 2.2
Wp Front End Repository arbitrary file upload 1.0.0 / 1.1
Wp Handy Lightbox remote code execution (RCE) 1.4.5
Wp Homepage Slideshow arbitrary file upload 2.0 / 2.3
Wp Image News Slider arbitrary file upload 3.0 / 3.5
Wp Levoslideshow arbitrary file upload 2.0 / 2.3
Wp Miniaudioplayer arbitrary file viewing 0.5 / 1.2.7
Wp Mobile Detector authenticated persistent cross-site scripting (XSS) 3.0 / 3.2
Wp Mon arbitrary file viewing 0.5 / 0.5.1
Wp Online Store arbitrary file viewing 1.2.5 / 1.3.1
Wp Piwik persistent cross-site scripting (XSS) 0.10.0.1 / 1.0.10
Wp Popup remote code execution (RCE) 2.0.0 / 2.1
Wp Post Frontend arbitrary file upload 1.0
Wp Property arbitrary file upload 1.20.0 / 1.35.0
Wp Quick Booking Manager persistent cross-site scripting (XSS) 1.0 / 1.1
Wp Royal Gallery persistent cross-site scripting (XSS) 2.0 / 2.3
Wp Seo Spy Google arbitrary file upload 3.0 / 3.1
Wp Simple Cart arbitrary file upload 0.9.0 / 1.0.15
Wp Slimstat Ex arbitrary file upload 2.1 / 2.1.2
Wp Superb Slideshow arbitrary file upload 2.0 / 2.4
Wp Swimteam arbitrary file viewing 1 / 1.44.1077
Wp Symposium arbitrary file upload 13.04 / 14.11
Wp Vertical Gallery arbitrary file upload 2.0 / 2.3
Wp Yasslideshow arbitrary file upload 3.0 / 3.4
Wp2android Turn Wp Site Into Android App arbitrary file upload 1.1.4
Wpeasystats local file inclusion (LFI) 1.8
Wpmarketplace arbitrary file viewing 2.2.0 / 2.4.0
Wpshop arbitrary file upload 1.3.1.6 / 1.3.9.5
Wpstorecart arbitrary file upload 2.0.0 / 2.5.29
Wptf Image Gallery arbitrary file viewing 1.0.1 / 1.0.3
Wsecure remote code execution (RCE) 2.3
Wysija Newsletters arbitrary file upload 1.1 / 2.6.7
Xdata Toolkit arbitrary file upload 1.6 / 1.9
Zen Mobile App Native arbitrary file upload 3.0
Zingiri Web Shop arbitrary file upload 2.3.6 / 2.4.3
Zip Attachments arbitrary file viewing 1.0 / 1.4

 

Have your WordPress site been hacked?

Don’t despair; it happens to the best of us. It’s tough to give generic advice without having a look at your site.

Speed up WordPress / Joomla CMS and MySQL server on Linux with tmpfs ram file system / Decrease Website pageload times with RAM caching

Wednesday, March 4th, 2015

speed-up-accelerate-wordpress-joomla-drupal-cms-and-mysql-server-with-tmpfs_ramfs_decrease-pageload-times-with-ram-caching
As a WordPress blog owner and an sys admin that has to deal with servers running a lot of WordPress / Joomla / Droopal and other custom CMS installed on servers, performoing slow or big enough to put a significant load on servers
and I love efficiency and hardware cost saving is essential for my daily job, I'm constantly trying to find new ways to optimize Customer Website (WordPress) and rest of sites in order to utilize better our servers and improve our clients sites speed (and hence satisfaction). 

There is plenty of little things to do on servers but probably among the most crucial ones which we use nowadays that save us a lot of money is tmpfs, and earlier (ramfs) – previously known as shmfs).
TMPFS is a (Temporary File Storage Facility) Linux kernel technology based on ramfs (used by Linux kernel initrd / initramfs on boot time in order to load and store the Linux kernel in memory, before system hard disk partition file systems are mounted) which is heavily used by virtually all modern popular Linux distributions. 

Using ramfs (cramfs variation – Compressed ROM filesystem) has been used to store different system environment kernel and Desktop components of many Linux environment / applications and used by a lot of the Linux BootCD such as the most famous (Klaus Knopper's) KNOPPIX LiveCD and Trinity Rescue Kit Linux (TRK uses /dev/shm which btw can be seen on most modern Linux distros and is actually just another mounted tmpfs).
If you haven't tried Live Linux yet try it out as me and a lot of sysadmins out there use some kind of LiveLinux at least few times on yearly basis  to Recover Unbootable Linux servers after some applied remote Updates as well as for Rescuing (Save) Data from Linux server failing to properly boot because of hard disk (bad blocks) failures. As I said earlier TMPFS is also used on almost any distribution for the /dev/ filesystem which is kept in memory.

You can see which tmpfs partitions is used on your Linux server with:

 

debian-server:~# mount |grep -i tmpfs
tmpfs on /lib/init/rw type tmpfs (rw,nosuid,mode=0755)
udev on /dev type tmpfs (rw,mode=0755)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)

 

Above is an output from a standard Debian Linux server. On CentOS 7 standard mounted tmpfs are as follows:

 

[root@centos ~]# mount |grep -i tmpfs
devtmpfs on /dev type devtmpfs (rw,nosuid,seclabel,size=1016332k,nr_inodes=254083,mode=755)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,seclabel)
tmpfs on /run type tmpfs (rw,nosuid,nodev,seclabel,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (rw,nosuid,nodev,noexec,seclabel,mode=755)

 

[root@centos ~]# df -h|grep -i tmpfs
devtmpfs                 993M     0  993M   0% /dev
tmpfs                   1002M   92K 1002M   1% /dev/shm
tmpfs                   1002M  8.8M  993M   1% /run
tmpfs                   1002M     0 1002M   0% /sys/fs/cgroup

The /run tmpfs mounted directory is also to be seen also on latest Ubuntus and Fedoras and is actually the good old /var/run ( where applications keep there pids and some small app related files) stored in tmpfs filesystem stored in memory.

If you're wondering what is /dev/shm and why it appears mounted on every single Linux Server / Desktop you've ever used this is a special filesystem shared memory which various running programs (processes) can use to transfer data quick and efficient between each other to preven the slow disk swapping. People using Linux for the rest 15 years should remember /dev/shm has been a target of a lot of kernel exploits as historically it had a lot of security issues.

While writting this article I've just checked about KNOPPIX developed amd just for info as of time of writting this distro has already 1000+ programs on CD version and 2600+ packages / application on DVD version.
Nowadays Knoppix is mostly used mostly as USB Live Flash drive as a lot of people are dropping CD / DVD use (many servers doesn't have a CD / DVD Drive) and for USB Live Flash Linux distros tmpfs is also key technology used as this gives the end user an amazing fast experience (Desktop applications run much fasten on Live USBs when tmpfs is used than when the slow 7200 RPM HDDs are used).

Loading big parts of the distribution within RAM (with tmpfs from Linux Kernel 2.4+ onwards) is also heavily used by a lot of Cluster vendors in most of Clustered (Cloud) Linux based environemnts, cause TMPFS gives often speeds up improvements to x30 times and decreases greatly I/O HDD. FreeBSD users will be happy to know that TMPFS is already ported and could be used on from FreeBSD 7.0+ onward.

In this small article I will give you example use on how I use tmpfs to speed up our WordPress Websites which use WP Caching plugins such as W3 Total Cache and WP Super Cache
and Hyper Cache / WP Super Cache disk caching and MySQL server as a Database backend.
Below example is wordpress specific but since it can be easily applied to JoomlaDrupal or any other CMS out there that uses mySQL server to make a lot of CPU expensive memory hungry (LEFT JOIN) queries which end up using a slow 7200 RPM hard disk.


 

1. Preparing tmpfs partitions for WordPress File Cache directory
 

If you want to give tmpfs a test drive, I recommend you try to create / mount a 20 Megabyte partition. To create a tmpfs partition you don't need to use a tool like mkfs.ext3 / mkfs.ext4 as TMPFS is in reality a virtual filesystem that is mapped in the server system physical RAM (volatile memory). TMPFS is very nice because if you run out of free RAM system starts a combination of RAM use + some Hard disk SWAP 
The great thing about TMPFS is it never uses all of the available RAM and SWAP, which would not halt your server if TMPFS partition gets filled, but instead you will start getting the usual "Insufficient Disk Space", just like with a physical HDD parititon. RAMFS cares much less about server compared to TMPFS, because if RAMFS is historically older.

ramfs file systems cannot be limited in size like a disk base file system which is limited by it’s capacity, thus ramfs will continue using memory storage until the system runs out of RAM and likely crashes or becomes unresponsive. This is a problem if the application writing to the file system cannot be limited in total size, so in my opinion you better stay away from RAMFS except you have a good idea what you're doing. Another disadvantage of RAMFS compared to TMPFS is you cannot see the size of the file system in df and it can only be estimated by looking at the cached entry in free.

Note that before proceeding to use TMPFS or RAMFS you should know besides having advantages, there are certain serious disadvantage that if the server using tmpfs (in RAM) to store files crashes the customer might loose his data, therefore using RAM filesystems on Production servers is best to be used just for caching folders which are regularly synchronized with (rsync) to some folder to assure no data will be lost on server reboot or crash.

Memory of fast storage areas are ideally suited for applications which need repetitively small data areas for caching or using as temporary space such as Jira (Issue and Proejct Tracking Software) Indexing  As the data is lost when the machine reboots the tmpfs stored data must not be data of high importance as even scheduling backups cannot guarantee that all the data will be replicated in the even of a system crash.

To test mounting a tmpfs virtual (memory stored) filesystem issue:
 

mount -t tmpfs tmpfs -o size=256m /mnt/tmpfs


If you want to test mount a ramfs instead:

 

 mount -t ramfs -o size=256m ramfs /mnt/ramfs

 

debian-server:~#  mount |grep -i -E "ramfs|tmpfs"
tmpfs on /lib/init/rw type tmpfs (rw,nosuid,mode=0755)
udev on /dev type tmpfs (rw,mode=0755)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
tmpfs on /mnt/tmpfs type tmpfs (rw,size=256m)
ramfs on /mnt/ramfs type tmpfs (rw,size=256m)

 

Once mounted tmpfs can be used in the same way as any ext4 / reiserfs filesystem. In the same way to make mounts permanent, its necessery to add a line to /etc/fstab

To illustrate better a tmpfs use case on my blog running WordPress with W3TotalCache (W3TC) plugin cache folder in /var/www/blog/wp-content/w3tc to get advantage of tmpfs to store w3tc files.

a) Stop Apache

On Debian
 

debian-server:~# /etc/init.d/apache stop


On CentOS 
 

[root@centos ~]# /etc/init.d/httpd stop


b) Move w3tc dir to w3tc-bak

 

debian-server:~# cd /var/www/blog/wp-content/
debian-server:~# mv w3tc w3tc-bak

 

c) Create w3tc directory
 

debian-server:/var/www/blog/wp-content# mkdir w3tc
debian-server:/var/www/blog/wp-content# chown -R www-data:www-data w3tc


d) Add tmpfs record to /etc/fstab

My W3TC Cache didn't grow bigger than 2Gigabytes so I create a 2Giga directory for it by adding following in /etc/fstab 
 

debian-server:~# vim /etc/fstab

 

tmpfs /var/www/blog/wp-content/w3tc tmpfs defaults,size=2g,noexec,nosuid,uid=33,gid=33,mode=1755 0 0


You might also want to add the nr_inodes (option) to tmpfs while mounting. nr_inodes is the maximum inode for instance. Default is half the number of your physical RAM pages, (on a machine with highmem) the number of lowmem RAM page, some common option that should work is nr_inodes=5k, if you're unsure what this option does you can safely skip it 🙂

e) Mount new added tmpfs folder

Then to mount the newly added filesystem issue:
 

mount -a


Or if you're on a CentOS / RHEL server use httpd Apache user instead and whenever you have docroot and wordpress installed.

 

[root@centos ~]# chown -R apache:apache: w3tc


If you're using Apache SuPHP use whatever the UID / GID is proper.

On CentOS you will need to set proper UID and GID (UserID / GroupID), to find out which ones to to use check in /etc/passwd:
 

[root@centos ~]# grep -i apache /etc/passwd
apache:x:48:48:Apache:/var/www:/sbin/nologin


f) Move old w3tc cache from w3tc-bak to w3tc

 

debian-server:/var/www/blog/wp-content# mv w3tc-bak/* w3tc/

 

g) Start again Apache

On Debian:

 

debian-server:~# /etc/init.d/apache2 start

 


On CentOS:
 

[root@centos~]# /etc/init.d/httpd start

h) Keeping w3tc cache site folder synced

As I said earlier the biggest problem with caching (the reason why many hosting providers) and site admins refuse to use it is they might loose some data, to prevent data loss or at least mitigate the data loss to few minutes intervals it is a good idea to synchronize tmpfs kept folders somewhere to disk with rsync.

To achieve that use a cronjob like this:
 

debian-server:~# crontab -u root -e
*/5 * * * * /usr/bin/ionice -c3 -n7 /usr/bin/nice -n 19 /usr/bin/rsync -ah –stats –delete /var/www/blog/wp-content/w3tc/ /backups/tmpfs/cache/ 1>/dev/null


Note that you will need to have the /backups/tmpfs/cache folder existing, create it with:

 

debian-server:~# mkdir -p /backups/tmpfs/cache


You will also need to add a rsync synchronization from backupped folder to tmpfs (in case if the server gets accidently rebooted because it hanged or power outage), place in

/etc/rc.local

 

ionice -c3 -n7 nice -n 19 rsync -ahv –stats –delete /backups/tmpfs/cache/ /var/www/blog/wp-content/w3tc/ 1>/dev/null


(somewhere before exit 0) line
 

0 05 * * * /usr/bin/ionice -c3 -n7 /bin/nice -n 19 /usr/bin/rsync -ah –stats –delete /var/www/blog/wp-content/w3tc/ /backups/tmpfs/cache/ 1>/dev/null

 

 

2. Preparing tmpfs partitions for MySQL server temp File Cache directory


Its common that MySQL servers had to serve a lot of long and heavy SQL JOIN Queries mostly by related posts WP plugins such as (Zemanta Related Posts) and Contextual Related posts though MySQLs are well optimized  to work as much as efficient using mysql tuner (tuning primer) still often SQL servers get a lot of temp tables created to disk (about 25% to 30%) of all SQL queries use somehow HDD to serve queries and as this is very slow and there is file lock created the overall MySQL performance becomes sluggish at times to fix (resolve) that without playing with SQL code to optimize the slow queries the best way I found is by using TMPFS as MySQL temp folder.

To do so I create a TMPFS usually the size of 256 MB because this is usually enough for us, but other hosting companies might want to add bigger virtual temp disk:

a) Add tmpfs new dir to /etc/fstab

In /etc/fstab add below record with vim editor:
 

debian-server:~# vim /etc/fstab

 

tmpfs /var/mysqltmp tmpfs rw,gid=111,uid=108,size=256M,nr_inodes=10k,mode=0700 0 0

 

Note that the uid / and gid 105 and 114 are taken again from /etc/passwd

On Debian

debian-server:~# grep -i mysql /etc/passwd
mysql:x:108:111:MySQL Server,,,:/var/lib/mysql:/bin/false


On CentOS
 

[root@centos ~]# grep -i mysql /etc/passwd
mysql:x:27:27:MySQL Server:/var/lib/mysql:/bin/bash


b) Create folder /var/mysqltmp or whenever you want to place the tmpfs memory kept SQL folder

 

debian-server:~# mkdir /var/mysqltmp
debian-server:~# chown mysql:mysql /var/mysqltmp

 

debian-server:~# mount|grep -i tmpfs
tmpfs on /lib/init/rw type tmpfs (rw,nosuid,mode=0755)
udev on /dev type tmpfs (rw,mode=0755)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
tmpfs on /var/www/blog/wp-content/w3tc type tmpfs (rw,noexec,nosuid,size=2g,uid=33,gid=33,mode=1755)
tmpfs on /var/mysqltmp type tmpfs (rw,gid=108,uid=111,size=256M,nr_inodes=10k,mode=0700)


c) Add new path to tmpfs created folder in my.cnf 

Then  edit /etc/mysql/my.cnf

 

debian-server:~# vim /etc/mysql/my.cnf

[mysqld]
#
# * Basic Settings
#
user        = mysql
pid-file    = /var/run/mysqld/mysqld.pid
socket      = /var/run/mysqld/mysqld.sock
port        = 3306
basedir     = /usr
datadir     = /var/lib/mysql
tmpdir      = /var/mysqltmp

 

On CentOS edit and change tmpdir in same way within /etc/my.cnf


d) Finally Restart Apache and MySQL to make mysql start using new set tmpfs memory kept folder

On Debian:
 

debian-server:~# /etc/init.d/apache2 stop; /etc/init.d/mysql restart; /etc/init.d/apache2 start

On CentOS:
 

[root@centos ~]# /etc/init.d/httpd stop; /etc/init.d/mysqld restart; /etc/initd/httpd start


Now monitor your server and check your pagespeed increase for me such an optimization usually improves site performance so site becomes +50% faster, to see the difference you can test your website before applying tmpfs caching for site and after that by using Google PageInsight (PageSpeed) Online Test. Though this example is for MySQL and WordPress you can easily adopt the same for Joomla if you have Joomla Caching enabled to some folder, same goes for any other CMS such as Drupal that can take use of Disk Caching. Actually its a small secret of many Hosting providers that allow clients to create sites via CPanel and Kloxo this tmpfs optimizations are already used for sites and by this the provider is able to offer better website service on lower prices. VPS hosting providers also use heavy caching. A lot of people are using TMPFS also to accelerate Sites that have enabled Google Pagespeed as Cacher and accelerator, as PageSpeed module puts a heavy HDD I/O load that can easily stone the server. Many admins also choose to use TMPFS for  /tmp, /var/run, and /var/lock directories as this leads often to significant overall server services operations improvement.
Once you have tmpfs enabled, It is a good idea to periodically monitor your SWAP used space with (df -h), because if you allocate bigger tmpfs partitions than your physical memory and tmpfs's full size starts to be used your machine will start swapping heavily and this could have a very negative performance affect.
 

debian-server:~# df -h|grep -i tmpfs
tmpfs            3,9G     0   3,9G   0% /lib/init/rw
tmpfs            3,9G     0   3,9G   0% /dev/shm
tmpfs            2,0G  1,4G   712M  66% /var/www/blog/wp-content/w3tc
tmpfs            256M     0   256M   0% /mnt/tmpfs
tmpfs            256M  236K   256M   1% /var/mysqltmp

The applications of tmpfs to accelerate services is up to your imagination, so I will be glad to hear from other admins on any interesting other application or problems faced while using TMPFS.

 Enjoy! 🙂

WordPress Plugins to monitor and debug WP enabled plugins – Find Errors / Warnings and Remove WP problematic plugins slowing down your Website (blog) database

Thursday, February 19th, 2015

plugins-to-monitor-debug-wordpress-enabled-plugins-how-to-track-find-errors-and-warnings-and-remove-problematic-wp-extensions-that-slow-down-your-website

Recent days, I'm spending a lot of time again trying to optimize my wordpress blog. Optimizing WP for better efficiency is becoming harder and harder task day by day as the website file content data is growing along with SQL databases. Moreover situation gets even worse because the number of plugins enabled on my blog is incrementally growing with time because, there is more and more goodies I'd like to add.
Optimizing WordPress to run for Speed on a server is a whole a lot of art and its a small universe in itself, because as of time of writting this post the count (number) of WordPress available PLUGINS is 36,197 ! 

1. Manually Tracking WordPress  Plugins causing Slow SQL Queries (MySQL bottleneck) issues directly using console / SSH

Because of its open source development and its nice modular design wordpress has turned into a standard for building small, middle sized and large websites (some WordPress based blogs and sites have from 50 000 to 100 000 unique pages!). My blog is still a small WordPress site with only 1676 posts, so I still haven't reached the high volume traffic optimization requirements but still even though I have a relatively good server hardware  8GB RAM / (2×2.70 Ghz Intel CPU) / 500 GB (7400 RPM HDD) at times I see Apache Webservers is unable to properly serve coming requests because of MySQL database (LEFT JOIN) requests being slow to serve (taking up to few seconds to complete) and creating a MySQL table lock, putting all the rest SQL queries to stay in a long unserved queues line, I've realized about this performance issue by using a a mysql cli (command) client and few commands and console command (tool) called mytop (also known as mtop). MyTop refreshes every 3 seconds, so the slow query will immediately stay on screen to view moer info about it press "f" and type the  in query ID.

mysql-top-running-on-gnu-linux-server-tracking-sql-queries-in-console-screenshot.png

mysql-top-running-on-gnu-linux-server-tracking-sql-queries-in-console-screenshot2

Finally it is very useful to run  for a while MySQL server logging to /var/log/mysql/slow-query.log:
Slow query is enabled (on my Debian 7 Wheezy host) by adding to /etc/mysql/my.cnf
after conf section

 

vim /etc/mysql/my.cnf
#
# * Logging and Replication
#
# Both location gets rotated by the cronjob.
# Be aware that this log type is a performance killer.
# As of 5.1 you can enable the log at runtime!
#general_log_file        = /var/log/mysql/mysql.log
#general_log             = 1
#
# Error logging goes to syslog due to /etc/mysql/conf.d/mysqld_safe_syslog.cnf.
#
# Here you can see queries with especially long duration

 

Paste:

 

slow_query_log = 1
slow_query_log_file = /var/log/mysql/slow-query.log
long_query_time = 2
log-queries-not-using-indexes

 

And then to make new mysql configuration load restarted mysql server:

 

debian-server:~# /etc/init.d/mysql restart
Stopping MySQL database server: mysqld.
Starting MySQL database server: mysqld ..
Checking for tables which need an upgrade, are corrupt or were
not closed cleanly..

 

Leaving mysql-slow.log to be enabled for 30 minutes to an 1 hrs is a good time to track most problematic slow queries and based on this queries, I took parts of  SQL UPDATE / SELECT / INSERT etc. Db queries which was problematic and grepped throughout /var/www/blog/wp-content/plugin files in order to determine which WordPress Plugin is triggering the slow query, causing blog to hang when too many clients try to see it in browser.

My main problematic SQL query having long execution time  (about 2 to 3 seconds!!!) most commonly occuring in slow-query.log was:

 

SELECT DISTINCT post_title, ID, post_type, post_name FROM wp_posts wposts LEFT JOIN wp_postmeta wpostmeta ON wposts.ID = wpostmeta.post_id LEFT JOIN wp_term_relationships ON (wposts.ID = wp_term_relationships.object_id) LEFT JOIN wp_term_taxonomy ON (wp_term_relationships.term_taxonomy_id = wp_term_taxonomy.term_taxonomy_id) WHERE (post_type='page' OR (wp_term_taxonomy.taxonomy = 'category' AND wp_term_taxonomy.term_id IN(11))) AND post_status = 'publish' AND LENGTH(post_title)>=5 ORDER BY LENGTH(post_title) ASC LIMIT 500

Because above query uses SQL Column names and Tables which are not hard coded in PHP code, to find out which plugins is most probably to launch this complex LEFT JOIN query, I used a quick bash one-liner:

 

# cd /var/www/blog/wp-content/plugins

 

# for i in $(grep -rli 'SELECT DISTINCT' *); do grep -rli 'LEFT JOIN' $i; done 
./seo-automatic-links/seo-links.php
./wp-postviews/wp-postviews.php
./yet-another-related-posts-plugin/classes/YARPP_Cache_Tables.php

 

I wanted to put less load on CPU during grep so looked for string only in .PHP extensioned files with:

 

 # for i in $(find . -iname '*.php' -exec grep -rli 'SELECT DISTINCT' '{}' \;); do grep -rli 'LEFT JOIN' $i; done
./seo-automatic-links/seo-links.php
./wp-postviews/wp-postviews.php
./yet-another-related-posts-plugin/classes/YARPP_Cache_Tables.php


As you can see the complex query is being called from PHP file belonging to one of 3 plugins

  • SEO Automatic Links – this is SEO Smart Links WP plugin (Does internal bliog interlinking in order to boast SEA)
  • WP PostViews – WordPress Post Views plugin (Which allows me to show how many times an article was read in WP Widget menu)
  • Yet Another Related Posts – Which is WP plugin I installed / enabled to show Related posts down on each blog post


2. Basic way to optimize MySQL slow queries (EXPLAIN / SHOW CREATE TABLE)

Now as I have a basic clue on plugins locking my Database, I disabled them one by one while keeping enabled mysql slow query log and viewing queries in mytop and I figure out that actually all of the plugins were causing a short time overheat (lock) on server Database because of LEFT JOINs. Though I really like what this plugins are doing, as they boast SEO and attract prefer to disable them for now and have my blog all the time responsible light fast instead of having a little bit better Search Engine Optimization (Ranking) and loosing many of my visitors because they're annoyed to wait until my articles open

Before disabling I tried to optimize the queries using MySQL EXPLAIN command + SHOW CREATE TABLE (2 commands often used to debug slow SQL queries and find out whether a Column needs to have added INDEX-ing to boast MySQL query).

Just in case if you decide to give them a try here is example on how they're used to debug problematic SQL query:
 

  1. mysql> explain SELECT DISTINCT post_title, ID, post_type, post_name
  2.     -> FROM wp_posts wposts LEFT JOIN wp_postmeta wpostmeta
  3.     -> ON wposts.ID = wpostmeta.post_id LEFT JOIN wp_term_relationships
  4.     -> ON (wposts.ID = wp_term_relationships.object_id) LEFT JOIN wp_term_taxonomy
  5.     -> ON (wp_term_relationships.term_taxonomy_id = wp_term_taxonomy.term_taxonomy_id)
  6.     -> WHERE (post_type='page'
  7.     -> OR (wp_term_taxonomy.taxonomy = 'category'
  8.     -> AND wp_term_taxonomy.term_id IN(11,15,17)))
  9.     -> AND post_status = 'publish'
  10.     -> AND LENGTH(post_title)>=5
  11.     -> ORDER BY LENGTH(post_title) ASC
  12.     -> LIMIT 500;
  13. +—-+————-+———————–+——–+——————+———+———+———————————————+——+———————————————-+
  14. | id | select_type | table                 | type   | possible_keys    | key     | key_len | ref                                         | rows | Extra                                        |
  15. +—-+————-+———————–+——–+——————+———+———+———————————————+——+———————————————-+
  16. |  1 | SIMPLE      | wposts                | ALL    | type_status_date | NULL    | NULL    | NULL                                        | 1715 | Using where; Using temporary; Using filesort |
  17. |  1 | SIMPLE      | wpostmeta             | ref    | post_id          | post_id | 8       | blog.wposts.ID                              |   11 | Using index; Distinct                        |
  18. |  1 | SIMPLE      | wp_term_relationships | ref    | PRIMARY          | PRIMARY | 8       | blog.wposts.ID                              |   19 | Using index; Distinct                        |
  19. |  1 | SIMPLE      | wp_term_taxonomy      | eq_ref | PRIMARY          | PRIMARY | 8       | blog.wp_term_relationships.term_taxonomy_id |    1 | Using where; Distinct                        |
  20. +—-+————-+———————–+——–+——————+———+———+———————————————+——+———————————————-+
  21. 4 rows in set (0.02 sec)
  22.  
  23. mysql>
  24.  

     

     

  1. mysql> show create table wp_posts;
  2. +———-+————————–+
  3. | Table    | Create Table                                                                                                                                                                                                                                                                                                                                                                                                                                 |
  4. +———-+————————–+
  5. | wp_posts | CREATE TABLE `wp_posts` (
  6.   `ID` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
  7.   `post_author` bigint(20) unsigned NOT NULL DEFAULT '0',
  8.   `post_date` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
  9.   `post_date_gmt` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
  10.   `post_content` longtext NOT NULL,
  11.   `post_title` text NOT NULL,
  12.   `post_excerpt` text NOT NULL,
  13.   `post_status` varchar(20) NOT NULL DEFAULT 'publish',
  14.   `comment_status` varchar(20) NOT NULL DEFAULT 'open',
  15.   `ping_status` varchar(20) NOT NULL DEFAULT 'open',
  16.   `post_password` varchar(20) NOT NULL DEFAULT '',
  17.   `post_name` varchar(200) NOT NULL DEFAULT '',
  18.   `to_ping` text NOT NULL,
  19.   `pinged` text NOT NULL,
  20.   `post_modified` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
  21.   `post_modified_gmt` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
  22.   `post_content_filtered` longtext NOT NULL,
  23.   `post_parent` bigint(20) unsigned NOT NULL DEFAULT '0',
  24.   `guid` varchar(255) NOT NULL DEFAULT '',
  25.   `menu_order` int(11) NOT NULL DEFAULT '0',
  26.   `post_type` varchar(20) NOT NULL DEFAULT 'post',
  27.   `post_mime_type` varchar(100) NOT NULL DEFAULT '',
  28.   `comment_count` bigint(20) NOT NULL DEFAULT '0',
  29.   PRIMARY KEY (`ID`),
  30.   KEY `post_name` (`post_name`),
  31.   KEY `type_status_date` (`post_type`,`post_status`,`post_date`,`ID`),
  32.   KEY `post_parent` (`post_parent`),
  33.   KEY `post_author` (`post_author`),
  34.   FULLTEXT KEY `post_related` (`post_title`,`post_content`)
  35. ) ENGINE=MyISAM AUTO_INCREMENT=12033 DEFAULT CHARSET=utf8 |
  36. +———-+———————-+
  37. 1 row in set (0.00 sec)
  38.  
  39. mysql>
  40.  


By the way above output is a paste from the the new PasteBin Open Source (Stikked powered) service I started on pc-freak.net – paste.pc-freak.net (p.pc-freak.net) 🙂

Before I took final decision to disable slow WP plugins, I've experimented a bit trying to add INDEX to Table Column (wposts) in hope that this would speed up SQL queries with:

 

mysql> ALTER TABLE TABLE_NAME ADD INDEX (wposts);

 

But this didn't improve query speed even on the contrary it make execution time worse.

3. Tracking WordPress Plugin PHP Code Execution time and Plugins causing Slow SQL Queries (MySQL bottleneck) issues through WP itself

Well fine, I'm running my own hosted Blog and WordPress sites, but for people who have wordpress sites on shared hosting, there is usually no SSH (Terminal) Access to server, those people will be happy to hear there are 2 Free easy installable WordPress plugins which can be used to Debug Slow WordPress Plugins SQL Queries as well as plugin to Track which plugin takes most time to execute, this are:
 

 

a) P3 Plugin Performance Profiler  

runs a scan over your site to determine what resources your plugins are using, and when, during a standard page request. P3 PPP Can even create reports in a beatiful Excel like Pie chart sheet.

p3-plugin-performance-profiler-godaddy-screenshot-debian-gnu-linux-wordpress-website

Another useful thing to see with P3 PPP is Detailed Timeline it shows when the plugins are being loaded during new page request so you can see if there is a certain sequence in time when a plugin slows down the website.

detailed_timeline-wordpress-p3-performance-plugin-on-website-screenshot

The pictures says it all as P3 PPP is Godaddy's work, congrats to GoDaddy, they've done great job.

 

b) WordPress memory Viewer WP plugins

Is useful to check how much memory each of WordPress plugin is taking on user (visitor) request.
Memory Viewer is allows you to view WordPress’ memory utilization at several hooks during WordPress’ execution. It also shows a summary of MySQL Queries that have ran as well as CPU time.
To use it download it to plugins/ folder as usual enable it from:

Installed Plugins -> (Inactive) -> Memory Viewer (Enable)

To see statistics from Memory Viewer open any post from your blog website and scroll down to the bottom you will notice the statistics, showing up there, like on below screenshot.

wordpress-memory-viewer-plugin-debian-gnu-linux-hosted-website-show-which-plugin-component-eats-most-memory-in-wordprses-blog
 

Though WP Memory Viewer is said to work only up to WP version 3.2.1, I've tested it and it works fine on my latest stable WordPress 4.1 based blog.

c) WordPress Query Monitor

wordpress-query-monitor-plugin-to-monitor-track-and-optimize-problems-with-sql-caused-by-wp-plugins.png
 

Query Monitor is a debugging plugin for anyone developing with WordPress but also very helpful for anyone who want to track issues with plugins who use the database unefficient.
It has some advanced features not available in other debugging plugins, including automatic AJAX debugging and the ability to narrow down things by plugin or theme.
You can view plenty of precious statistics on how enabled plugins query the database server, here is a short overview on its Database Queries capabilities:

  • Shows all database queries performed on the current page
  • Shows affected rows and time for all queries
  • Show notifications for slow queries and queries with errors
  • Filter queries by query type (SELECT, UPDATE, DELETE, etc)
  • Filter queries by component (WordPress core, Plugin X, Plugin Y, theme)
  • Filter queries by calling function
  • View aggregate query information grouped by component, calling function, and type
  • Super advanced: Supports multiple instances of wpdb on one page
  • Once enabled from Plugins you will see it appear as a new menu on bottom Admin raw.

An important note to make here is latest Query Monitor extension fails when loaded on current latest Wordpress 4.1, to use it you will have to download and useolder Query Monitor plugin version 2.6.8 you can download it from here

d) Debug Bar

If you want you want a Memory Viewer like plugin for more complex used components memory debugging, reporting if (WP_DEBUG is set in wp-config.php) also check out Debug Bar .
For me Debug Bar was very useful because it show me depreciated functions some plugins used, so I substituted the obsoleted function with new one.

 

debug-bar-debug-wordpress-plugins-memory-use-screenshot-website


4. Server Hardware hungry (slow) WordPress plugins that you better not use

While spending time to Google for some fixes to WP slow query plugins – I've stumbled upon this post giving a good list with WordPress Plugins better off not to use because they will slow down your site
This is a publicly well known list of WP plugins every WordPress based site adminstrator should avoid, but until today I didn't know so my assumption is you don't know either ..

Below plugins are extremely database intensive mentioned in article that we should better (in all cases!) avoid:

  • Dynamic Related Posts
  • SEO Auto Links & Related Posts
  • Yet Another Related Posts Plugin
  • Similar Posts
  • Contextual Related Posts
  • Broken Link Checker — Overwhelms even our robust caching layer with an inordinate amount of HTTP requests.
  • MyReviewPlugin — Slams the database with a fairly significant amount of writes.
  • LinkMan — Much like the MyReviewPlugin above, LinkMan utilizes an unscalable amount of database writes.
  • Fuzzy SEO Booster — Causes MySQL issues as a site becomes more popular.
  • WP PostViews — Inefficiently writes to the database on every page load. To track traffic in a more scalable manner, both the stats module in Automattic’s Jetpack plugin and Google Analytics work wonderfully.
  • Tweet Blender — Does not play nicely with our caching layer and can cause increased server load.


A good Complete list of known WordPress slow plugins that will hammer down your wordpress performance is here

There are few alternatives to this plugins and when I have some free time I will download and test their alternatives but for now I plan the plugins to stay disabled.
 

For the absolute WP Performance Optimization Freaks, its good to check out the native way to Debug a wordpress installation through using few embedded
variables

 

define('WP_DEBUG', true);
define('WP_DEBUG', false);
define('WP_DEBUG_LOG', true);
define('WP_DEBUG_DISPLAY', false);
define('SAVEQUERIES', true);

 

An article describing how you can use native WP debug variables is here


Happy Optimizing  ! 🙂

Optimize WordPress Pictures with EWWW Image Optimizer, Async JS and CSS and Autoptimize for better Search Engine Ranking

Tuesday, December 9th, 2014

 


wordpress-ewww-image-optimizer_settings_screenshot-plugin-seo-for-images-wp_3

While optimizing picture performance with console tools optipng, jpegoptin, jpegtran, pngcrush (could save you a lot of server space and make pictures downloads faster (and hence increase your website responsiveness and SEO – check out), still for Blogs and WebSites based on WordPress its not worthy to loose time with console acrobatics but simply use EWWW Image Optimizer to Optimize all old or new uploaded Images.

To work EWWW Image Optimizer needs jpegtran, optipng, pngout and gifsicle to be installed on the Linux / BSD server. EWWW Image Optimizer can load the command line tools also from a Cloud, if a cloud service is running on the server. Once installed the plugin does scan all the imported WordPress Media files and can be run to optimize picture files on present blog psot / pages.

EWWW Image Opitimizer plugin does a good job in reducing file size on  NextGEN, GRAND FlAGallery galleries.

wordpress-ewww-image-optimizer_settings_screenshot-plugin-seo-for-images-wp

Here is how EWWW Image Optimizer works taken from plugin's website:
How are JPGs optimized?

Lossless optimization is done with the command jpegtran -copy all -optimize -progressive -outfile optimized-file original-file. Optionally, the -copy switch gets the 'none' parameter if you choose to strip metadata from your JPGs on the options page. Lossy optimization is done using the outstanding JPEGmini utility.
It is better if the server has not the jpegtran, pngout, gifsicle utilities installed as the plugin provides an uptodate static compiled Linux binaries.

How are PNGs optimized?

There are three parts (and all are optional). First, using the command pngquant original-file, then using the commands pngout-static -s2 original-file and optipng -o2 original-file. You can adjust the optimization levels for both tools on the settings page. Optipng is an automated derivative of pngcrush, which is another widely used png optimization utility.

How are GIFs optimized?

Using the command gifsicle -b -O3 –careful original file. This is particularly useful for animated GIFs, and can also streamline your color palette. That said, if your GIF is not animated, you should strongly consider converting it to a PNG. PNG files are almost always smaller, they just don't do animations. The following command would do this for you on a Linux system with imagemagickconvert somefile.gif somefile.png

wordpress-ewww-image-optimizer_settings_screenshot-plugin-seo-for-images-wp

Some othe plugins that could strenghten your WordPress Search Engine Optimization ranking worthy to check are:
 

  • Async JS and CSS
     

Most importantly plugin solves "Render-blocking JavaScript and CSS" warning shown during site audit with  Google Developers PageSpeed InsightBy the way Google PageSpeed Insight is a precious tool so I recommend you check if you already haven't, Google's suggestions could often double or triple daily site visitors 

What Async JS and CSS does is:

Converts render-blocking CSS and JS files into NON-render-blocking, improving performance of web page

async_js_and_css_wordpress-plugin_configuration_menu

The plugin makes ALL scripts loaded by other plugins to be loaded in asynchronous. All CSS files will be inserted inline into the document code or moved from the document beginning to the end, just before closing BODY tag (or just where you placed wp_foot() function). There are various methods to do that via plugin configuration page.
 

  • Autoptimize

     

     

     

    Wordpress-Autoptimize-screenshot-a-plugin-to-minify-wordpress-html-js-and-css-scripts

Autoptimize speeds up your website and helps you save bandwidth by aggregating and minimizing JS, CSS and HTML.

What does the plugin do to help speed up site?

It concatenates all scripts and styles, minifies and compresses them, adds expires headers, caches them, and moves styles to the page head, and scripts to the footer. It also minifies the HTML code itself, making your page really lightweight. Autoptimize is very much like WP Mnify (CSS / JS) minifaction WP plugin. The only difference and reason why you might want to use WP Mnify is it does HTML minification – something that WP Minify does not. Both plugins play nice together the only thing to be careful is not to configure CSS / JS minification in both Autoptimize and WP Minifyas this might slower instead of fasten the WP site.

A great bunch of other useful WP plugins to make a WordPress Blog friendly to Search Engines is here.

Manually deleting spam comments from WordPress blogs and websites to free disk space and optimize MySQL

Monday, November 24th, 2014

WordPress-delete_spam_comments_manually_with_sql_query_to-optimize_mysql-and-free-disk-space
If you're a web-hosting company or a web-development using WordPress to build multitudes of customer blogs or just an independent blogger or sys-admin with a task to optimize a server's MySQL allocated storage  / performance on triads of WordPress-es a a good tip that would help is to removing wp_comments marked as spam.

Even though sites might be protected of thousands of spam message daily caught by WP anti-spam plugin Akismet, spam caught messages aer forwarder by Akismet to WP's Spam filter and kept wp_comments table with comments_approved column  record 'spam'.

Therefore you will certainly gain of freeing disk space uselessly allocated by spam messages into current MySQL server storage dir (/var/lib/mysql   /usr/local/mysql/data – the directory where my.cnf tells the server to keep its binary data .MYI, .MYD, .frm files) as well as save a lot of disk space by excluding the useless spam messages from SQL daily backup archives.

Here is how to remove manually spam comments from a WordPress blog under database (wp_blog1);

mysql> use wp_blog1;
mysql> describe wp_comments;
+———————-+———————+——+—–+———————+—————-+
| Field | Type | Null | Key | Default | Extra |
+———————-+———————+——+—–+———————+—————-+
| comment_ID | bigint(20) unsigned | NO | PRI | NULL | auto_increment |
| comment_post_ID | bigint(20) unsigned | NO | MUL | 0 | |
| comment_author | tinytext | NO | | NULL | |
| comment_author_email | varchar(100) | NO | | | |
| comment_author_url | varchar(200) | NO | | | |
| comment_author_IP | varchar(100) | NO | | | |
| comment_date | datetime | NO | | 0000-00-00 00:00:00 | |
| comment_date_gmt | datetime | NO | MUL | 0000-00-00 00:00:00 | |
| comment_content | text | NO | | NULL | |
| comment_karma | int(11) | NO | | 0 | |
| comment_approved | varchar(20) | NO | MUL | 1 | |
| comment_agent | varchar(255) | NO | | | |
| comment_type | varchar(20) | NO | | | |
| comment_parent | bigint(20) unsigned | NO | MUL | 0 | |
| user_id | bigint(20) unsigned | NO | | 0 | |
+———————-+———————+——+—–+———————+—————-+


The most common and quick way useful for scripting (whether you have to do it for multiple blogs with separate dbs) is to delete all comments being filled as 'Spam'.

To delete all messages which were filled by Akismet's spam filter with high probabily being a spam issue from mysql cli interface:

DELETE FROM wp_comments WHERE comment_approved = 'spam';


For Unread (Unapproved) messages the value of comment_approved field are 0 or 1, 0 if the comment is Red and Approved and 1 if still it is to be marked as read (and not spam).
If a wordpress gets heavily hammered with mainly spam and the probability that unapproved message is different from spam is low and you want to delete any message waiting for approvel as not being spam from wordpress use following SQL query:

DELETE FROM wp_comments WHERE comment_approved = 0;

Another not very common you might want to do is delete only all apprved comments:

DELETE FROM wp_comments WHERE comment_approved = 1;

For old installed long time unmaintained blogs (with garbish content), it is very likely that 99% of the messages might be spam and in case if there are already >= 100 000 spam messages and you don't have the time to inspect 100 000 spam comments to get only some 1000 legitimate and you want to delete completely all wordpress comments for a blog in one SQL query use:

TRUNCATE wp_comments;

Another scenario if you know a blog has been maintained until certain date and comments were inspected and then it was left unmaintained for few years without any spam detect and clear plugin like Akismet, its worthy to delete all comments starting from the date wordpress site stopped to be maintained:

DELETE FROM wp_comments WHERE comment_date > '2008-11-20 05:00:10' AND comment_date <= '2014-11-24 00:30:00'

Script to Automatically change current MySQL server in wp-config.php to another MySQL host to minimize WordPress and Joomla downtimes

Friday, July 20th, 2012

I'm running a two servers for a couple of home hosted websites. One of the servers is serving as Apache host1 and has configured MySQL running on it and the second is used just for database host2 – (has another MySQL configured on it).
The MySQL servers are not configured to run as a MySQL MASTER and MySQL SLAVE (no mysql replication), however periodically (daily), I have a tiny shell script that is actualizing the data from the active SQL host2 server to host1.

Sometimes due to electricity problems or CPU overheats the active MySQL host at host2 gets stoned and stops working causing the 2 WordPress based websites and One joomla site inaccessible.
Until I manually get to the machine and restart host2 the 3 sites are down from the net and as you can imagine this has a very negative impact on the existing website indexing (PageRank) in Google.

When I'm at home, this is not a problem as I have physical access to the servers and if somethings gets messy I fix it quickly. The problem comes, whether I'm travelling or in another city far from home and there is no-one at home to give the hanged host hard reboot ….

Lately the problems with hang-ups of host2 happaned 3 times or so for 2 weeks, as a result the websites were inaccessible for hours and since there is nobody to reboot the server for hours; the websites keep hanging until the DB host is restarted ;;;;

To work-around this I came with the idea to write a tiny shell script to check if host2 is ping-able in order to assure the Database host is not down and then if script determines host2 (mysql) host is down it changes wp-config.php (set to use host2) to a wp-config.php (which I have beforehand configured to use) host1.

Using the script is a temporary solution, since I have to actually find the real hang-up causing troubles, but at least it saves me long downtimes. Here is a download link to the script I called change_blog_db.sh .
I've configured the script to be run on the Apache node (host1) via a crontab calling the script every 10 minutes, here is the crontab:
 

*/10 * * * * /usr/sbin/change_blog_db.sh > /dev/null 2>&1

The script is written in a way so if it determins host2 is reachable a copy of wp-config.php and Joomla's configuration.php tuned to use host2 is copied over the file config originals. In order to use the script one has to configured the head variables script section, e.g.:

host_to_ping='192.168.0.2';
blog_dir='/var/www/blog';
blog_dir2='/var/www/blog1';blog_dir3='/var/www/joomla';
notify_mail='hipo@pc-freak.net';
wp_config_orig='wp-config.php';
wp_config_localhost='wp-config-localhost.php';
wp_config_other_host='wp-config-192.168.0.2.php';
joomla_config_orig='configuration.php';
joomla_config_other_host='configuration-192.168.0.2.php';

You will have to manually prepare;;;

wp-config-localhost.php, wp-config-192.168.0.2.php ,configuration-192.168.0.2.php, wp-config-localhost.php to be existing files configured to with proper host1 and host2 IP addresses.
Hope the script will be useful to others, experiencing database downtimes with WordPress or Joomla installs.
 

How to count lines of PHP source code in a directory (recursively)

Saturday, July 14th, 2012

Count PHP and other programming languages lines of source code (source code files count) recursively

Being able to count the number of PHP source code lines for a website is a major statistical information for timely auditting of projects and evaluating real Project Managment costs. It is inevitable process for any software project evaluation to count the number of source lines programmers has written.
In many small and middle sized software and website development companies, it is the system administrator task to provide information or script quickly something to give info on the exact total number of source lines for projects.

Even for personal use out of curiousity it is useful to know how many lines of PHP source code a wordpress or Joomla website (with the plugins) contains.
Anyone willing to count the number of PHP source code lines under one directory level, could do it with:::

serbver:~# cd /var/www/wordpress-website
server:/var/www/wordpress-website:# wc -l *.php
17 index.php
101 wp-activate.php
1612 wp-app.php
12 wp-atom.php
19 wp-blog-header.php
105 wp-comments-post.php
12 wp-commentsrss2.php
90 wp-config-sample.php
85 wp-config.php
104 wp-cron.php
12 wp-feed.php
58 wp-links-opml.php
59 wp-load.php
694 wp-login.php
236 wp-mail.php
17 wp-pass.php
12 wp-rdf.php
15 wp-register.php
12 wp-rss.php
12 wp-rss2.php
326 wp-settings.php
451 wp-signup.php
110 wp-trackback.php
109 xmlrpc.php
4280 total

This will count and show statistics, for each and every PHP source file within wordpress-website (non-recursively), to get only information about the total number of PHP source code lines within the directory, one could grep it, e.g.:::

server:/var/www/wordpress-website:# wc -l *.php |grep -i '\stotal$'
4280 total

The command grep -i '\stotal$' has \s in beginning and $ at the end of total keyword in order to omit erroneously matching PHP source code file names which contain total in file name; for example total.php …. total_blabla.php …. blabla_total_bla.php etc. etc.

The \s grep regular expression meaning is "put empty space", "$" is placed at the end of tital to indicate to regexp grep only for words ending in string total.

So far, so good … Now it is most common that instead of counting the PHP source code lines for a first directory level to count complete number of PHP, C, Python whatever source code lines recursively – i. e. (a source code of website or projects kept in multiple sub-directories). To count recursively lines of programming code for any existing filesystem directory use find in conjunction with xargs:::

server:/var/www/wp-website1# find . -name '*.php' | xargs wc -l
1079 ./wp-admin/includes/file.php
2105 ./wp-admin/includes/media.php
103 ./wp-admin/includes/list-table.php
1054 ./wp-admin/includes/class-wp-posts-list-table.php
105 ./wp-admin/index.php
109 ./wp-admin/network/user-new.php
100 ./wp-admin/link-manager.php
410 ./wp-admin/widgets.php
108 ./wp-content/plugins/akismet/widget.php
104 ./wp-content/plugins/google-analytics-for-wordpress/wp-gdata/wp-gdata.php
104 ./wp-content/plugins/cyr2lat-slugs/cyr2lat-slugs.php
,,,,
652239 total

As you see the cmd counts and displays the number of source code lines encountered in each and every file, for big directory structures the screen gets floated and passing | less is nice, e.g.:

find . -name '*.php' | xargs wc -l | less

Displaying lines of code for each file within the directories is sometimes unnecessery, whether just a total number of programming source code line is required, hence for scripting purposes it is useful to only get the source lines total num:::

server:/var/www/wp-website1# find . -name '*.php' | xargs wc -l | grep -i '\stotal$'

Another shorter and less CPU intensive one-liner to calculate the lines of codes is:::

server:/var/www/wp-website1# ( find ./ -name '*.php' -print0 | xargs -0 cat ) | wc -l

Here is one other shell script which displays all file names within a directory with the respective calculated lines of code

For more professional and bigger projects using pure Linux bash and command line scripting might not be the best approach. For counting huge number of programming source code and displaying various statistics concerning it, there are two other tools – SLOCCount
as well as clock (count lines of code)

Both tools, are written in Perl, so for IT managers concerned for speed of calculating projects source (if too frequent source audit is necessery) this tools might be a bit sluggish. However for most projects they should be of a great add on value, actually SLOCCount was already used for calculating the development costs of GNU / Linux and other projects of high importance for Free Software community and therefore it is proven it works well with ENORMOUS software source line code calculations written in programming languages of heterogenous origin.

sloccount and cloc packages are available in default Debian and Ubuntu Linux repositories, so if you're a Debilian user like me you're in luck:::

server:~# apt-cache search cloc$
cloc - statistics utility to count lines of code
server:~# apt-cache search sloccount$
sloccount - programs for counting physical source lines of code (SLOC)

Well that's all folks, Cheers en happy counting 😉

How to fix bug with WordPress domain extra trailing slash (Double wordpress trailing slash)

Monday, July 9th, 2012

How to fix bug with wordpress extra slash, domain double slash issue pic

2 of the wordpress installations, I take care for had been reported an annoying bug today by some colleagues.
The bug consisted in double trailing slash at the end of the domain url e.g.;

http://our-company-domainname.com//

As a result in the urls everywhere there was the double trailing slash appearing i.e.::

http://our-company-domainname.com//countact-us/
http://our-company-domainname.com//languages/

etc.

The bug was reported to happen in the multiolingual version of the wordpress based sites, as the Qtranslate plugin is used on this installations to achieve multiple languages it seemed at first logical that the double slash domain and url wordpress issues are caused for some reason by qTranslate.

Therefore, I initially looked for the cause of the problem, within the wordpress admin settings for qTranslate plugin. After not finding any clue pointing the bug to be related to qTranslate, I've then checked the settings for each individual wordpress Page and Post (There in posts usually one can manually set the exact url pointing to each post and page).
The double slash appeared also in each Post and Page and it wasn't possible to edit the complete URL address to remove the double trailin slashes. My next assumption was the cause for the double slash appearing on each site link is because of something wrong with the sites .htaccess, therefore I checked in the wp main sites directory .htaccess
Strangely .htacces seemed OKAY and there was any rule that somehow might lead to double slashes in URL. WP-sites .htaccess looked like so:
 

server:/home/wp-site1/www# cat .htaccess
RewriteEngine On
RewriteBase /

# Rewrite rules for new content and scripts folder
RewriteRule ^jscripts/(.*)$ wp-includes/js/$1
RewriteRule ^gallery/(.*)$ wp-content/uploads/$1
RewriteRule ^modules/(.*)$ wp-content/plugins/$1
RewriteRule ^gui/(.*)/(.*)$ wp-content/themes/$1/$2 [L]

# Disable direct acceees to wp files if referer is not valid
#RewriteCond %{THE_REQUEST} .wp-*
#RewriteCond %{REQUEST_URI} .wp-*
#RewriteCond %{REQUEST_URI} !.*media-upload.php.*
#RewriteCond %{HTTP_REFERER} !.*cadia.*
#RewriteRule . /error404 [L]

# Standard WordPress rewrite
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]

Onwards, I thought a possible way to fix bug by adding mod_rewrite rules in .htaccess which would do a redirect all requests to http://www.our-company-domainname.com//contact-us/ to http://www.our-company-domainname.com//contact-us/ etc. like so:

RewriteRule ^/(.*)$ /$1

This for unknown reasons to me didn't worked either, finally thanks God I remembered to check the variables in wp-config.php (some month ago or so I added there some variables in order to improve the wordpress websites opening times).

I've figured out I did a mistake in one of the variables by adding an ending slash to the URL. The variable added was:

define('WP_HOME','http://our-company-domainname.com/');

whether instead it should be without the ending trailing slash like so:

define('WP_HOME','http://our-company-domainname.com');

By removing the ending trailing slash:

define('WP_HOME','http://our-company-domainname.com/');

to:

define('WP_HOME','http://our-company-domainname.com');
fixed the issue.
Cheers 😉

Fix Null error in WordPress comment reply with wordpress-threaded-comments plugin enabled

Friday, April 6th, 2012

I'm running WordPress for already 3 years or so now. Since some very long time. The first wordpress install, I can hardly remember but it something like wordpress 2.5 or wordpress 2.4

Since quite a long time my wordpress blog is powered by a number of plugins, which I regularly update, whenever new plugins pops up …
I haven't noticed most of the time problems during major WordPress platform updates or the update of the installed extensions. However, today while I tried to reply back to one of my blog comments, I've been shocked that, I couldn't.
Pointing at the the Comment Reply box and typing inside was impossible and a null message was stayed filled in the form:

To catch what was causing this weird misbehaving with the reply comments functionality, I grepped through my /var/www/blog/wp-content/plugins/* for the movecfm(null,0,1,null):

# cd /var/www/blog/wp-content/plugins
# grep -rli 'movecfm(null,0,1,null)' */*.php
wordpress-thread-comment/wp-thread-comment.php

I've taken the string movecfm(null,0,1,null) from the browser page source in in my Firefox by pressing – Ctrl+U).

Once I knew of the problem, I first tried commenting the occurances of the null fields in wp-thread-comment.php, but as there, were other troubles in commenting this and I was lazy to read the whole code, checked online if some other fellows experienced the same shitty null void javascript error and already someone pointed at a solution. In the few minutes search I was unable to find anyone who reported for this bug, but what I found is some user threads on wordpress.org mentioning since WordPress 2.7+ the wordpress-threaded-comments is obsolete and the functionality provided by the plugin is already provided by default in newer WPinstalls.

Hence in order to enable the threaded comments WordPress (embedded) reply functionality from within the wp-admin panel used:

Settings -> Discussions -> Enable Threaded (nested) comments (Tick)

Enable Nested Comments WordPress default wp comments enable reply functionality screenshot

You see there is also an option to define how many nested comments subcomments, can be placed per comment, the default was 5, but I thought 5 is a bit low so increased it to 10 comments reply possible per comment.

Finally, to prevent the default threaded comments to interfere with the WordPress Threaded Comments plugin, disabled the plugin through menus:

Plugins -> Active -> WordPress Thread Comments (Deactivate)

This solved the weird javascript null "bug" caused by wordpress-threaded-comments once and for all.
Hopefully onwards, my blog readers will not have issues with threaded Reply Comments.