GNU Archives - ☩ Walking in Light with Christ - Faith, Computing, Diary ☩ Walking in Light with Christ

Posts Tagged ‘GNU’

find text strings recursively in Linux and UNIX – find grep in sub-directories command examples

Tuesday, May 13th, 2014

GNU Grep is equipped with a special option "-r" to grep recursively. Looking for string in a file in a sub-directories tree with the -r option is a piece of cake. You just do:

grep -r 'string' /directory/

or if you want to search recursively non-case sensitive for text

grep -ri 'string' .

Another classic GNU grep use (I use almost daily) is whether you want to match all files containing (case insensitive) string among all files:

grep -rli 'string' directory-name

Now if you want to grep whether a string is contained in a file or group of files in directory recursively on some other UNIX like HP-UX or Sun OS / Solaris where there is no GNU grep installed by default here is how to it:

find /directory -exec grep 'searched string' {} dev/null ;

Note that this approach to look for files containing string on UNIX is very slowThus on not too archaic UNIX systems for some better search performance it is better to use xargs;

find . | xargs grep searched-string

A small note to open here is by using xargs there might be weird results when run on filesystems with filenames starting with "-".

Thus comes the classical (ultimate) way to grep for files containing string with find + grep, e.g.

find / -exec grep grepped-string {} dev/null ;

Another way to search a string recursively in files is by using UNIX OS '*' (star) expression:

grep pattern * */* */*/* 2>/dev/null

Talking about recursive directory text search in UNIX, should mention another good GNU GREP alternative ACK – check it on betterthangrep.com 🙂 . Ack is perfect for programmers who have to dig through large directory trees of code for certain variables, functions, objects etc.

Tags: cake, code, directory, file, filesystems, GNU, grep, How to, Linux, look, match, note, piece, piece of cake, recursively, string, text, unix, variables, xargs
Posted in Programming, System Administration, Various | 1 Comment »

Save data from failing hard disk on Linux – Rescuing data from failing disk with bad blocks

Wednesday, April 16th, 2014

save-data-from-failing-hard-drive-data-recovery-badblocks-linux_1.jpg
Sooner or later your Linux Desktop or Linux server hard drive will start breaking up, whether you have a hardware or software RAID 1, 6 or 10 you can and good hard disk health monitoring software you can react on time but sometimes as admins we have to take care of old servers which either have RAID 0 or missing RAID configuration and or disk firmware is unable to recognize failing blocks on time and remap them. Thus it is quite useful to have techniques to save data from failing hard disk drives with physical badblocks.

With ddrescue tool there is still hope for your Linux data though disk is full of unrecoverable I/O errors.

apt-cache show ddrescue

apt-cache show ddrescue|grep -i description -A 12

Description: copy data from one file or block device to another
dd_rescue is a tool to help you to save data from crashed
partition. Like dd, dd_rescue does copy data from one file or
block device to another. But dd_rescue does not abort on errors
on the input file (unless you specify a maximum error number).
It uses two block sizes, a large (soft) block size and a small
(hard) block size. In case of errors, the size falls back to the
small one and is promoted again after a while without errors.
If the copying process is interrupted by the user it is possible
to continue at any position later. It also does not truncate
the output file (unless asked to). It allows you to start from
the end of a file and move backwards as well. dd_rescue does
not provide character conversions.

To use ddrescue for saving data first thing is to shutdown the Linux host boot the system with a Rescue LiveCD like SystemRescueCD – (Linux system rescue disk), Knoppix (Most famous bootable LiveCD / LiveDVD), Ubuntu Rescue Remix or BackTrack LiveCD – (A security centered "hackers" distro which can be used also for forensics and data recovery), then mount the failing disk (I assume disk is still mountable :). Note that it is very important to mount the disk as read only, because any write operation on hard drive increases chance that it completely becomes unusable before saving your data!

To make backup of your whole hard disk data to secondary mounted disk into /mnt/second_disk

# mkdir /mnt/second_disk/rescue
# mount /dev/sda2 /mnt/second_disk/rescue
# dd_rescue -d -r 10 /dev/sda1 /mnt/second_disk/rescue/backup.img

# mount -o loop /mnt/second_disk/rescue/backup.img

In above example change /dev/sda2 to whatever your hard drive device is named.

Whether you have already an identical secondary drive attached to the Linux host and you would like to copy whole failing Linux partition (/dev/sda) to the identical drive (/dev/sdb) issue:

ddrescue -d -f -r3 /dev/sda /dev/sdb /media/PNY_usb/rescue.logfile

If you got just a few unreadable files and you would like to recover only them then run ddrescue just on the damaged files:

ddrescue -d –R -r 100 /damaged/disk/some_dir/damaged_file /mnt/secondary_disk/some_dir/recoveredfile

-d instructs to use direct I/O
-R retrims the error area on each retry
-r 100 sets the retry limit to 100 (tries to read data 100 times before resign)

Of course this is not always working as on some HDDs recovery is impossible due to hard physical damages, if above command can't recover a file in 10 attempts it is very likely that it never succeeds …

A small note to make here is that there is another tool dd_rescue (make sure you don't confuse them) – which is also for recovery but GNU ddrescue performs better with recovery.
How ddrescue works is it keeps track of the bad sectors, and go back and try to do a slow read of that data in order to read them.
By the way BSD users would happy to know there is ddrescue port already, so data recovery on BSDs *NIX filesystems if you're a Windows user you can use ddrescue to recover data too via Cygwin.
Of course final data recovery is also very much into God's hands so before launching ddrescue, don't forget to say a prayer 🙂

Tags: bad blocks, data, firmware, GNU, good, hard disk, hard drive, health, input file, Linux, Linux data recovery, linux desktop, mnt, mount dev, Raid, read, recovery, run, software, system, system rescuecd ddrescue, time, Ubuntu Rescue Remix, use
Posted in Linux, Linux Backup tools, System Administration | No Comments »

How to convert OGG Vorbis .ogg to MP3 on GNU / Linux and FreeBSD

Friday, July 27th, 2012

I’ve used K3B just recently to RIP an Audio CD with music to MP3. K3b has done a great job ripping the tracks, the only problem was By default k3b RIPs songs in OGG Vorbis (.ogg) and not mp3. I personally prefer OGG Vorbis as it is a free freedom respecting audio format, however the problem was the .ogg-s cannot be read on many of the audio players and it could be a problem reading the RIPped oggs on Windows. I’ve done the RIP not for myself but for a Belarusian gfriend of mine and she is completely computer illiterate and if I pass her the songs in .OGG, there is no chance she succed in listening the oggs. I’ve seen later k3b has an option to choose to convert directly to MP3 Using linux mp3 lame library this however is time consuming and I have to wait another 10 minutes or so for the songs to be ripped to shorten the time I decided to directly convert the existing .ogg files to .mp3 on my (Debian Linux). There are probably many ways to convert .ogg to mp3 on linux and likely many GUI frontends (like SoundConverter) to use in graphic env.

SoundConverter Debian GNU Linux graphic GUI environment program for convertion of ogg to mp3 and mp3 to ogg, convert multiple sound formats on GNU / Linux.

I however am a console freak so I preferred doing it from terminal. I’ve done quick research on the net and figured out the good old ffmpeg is capable of converting .oggs to .mp3s. To convert all mp3s just ripped in the separate directory I had to run ffmpeg in a tiny bash loop.

A short bash shell script 1 liner combined with ffmpeg does it, e.g.;

for f in *.ogg; do ffmpeg -i "$f" "`basename "$f" .ogg`.mp3"; done.....

The loop example is in bash so in order to make the code work on FreeBSD it is necessery it is run in a bash shell and not in BSDs so common csh or tcsh.

Well, that’s all oggs are in mp3; Hip-hip Hooray 😉

Tags: audio cd, Auto, basename, bash shell script, BSDs, chance, code, consuming, csh, debian linux, Draft, ffmpeg, format, freak, freebsd, freedom, frontends, GNU, gnu linux, job, k3b, Linux, necessery, Ogg, ogg files, oggs, option, RIPs, script, Shell, soundconverter, terminal, time, time consuming, vorbis ogg, work
Posted in Everyday Life, FreeBSD, Linux and FreeBSD Desktop, Linux Audio & Video | No Comments »

Disabling PHP system(); and exec(); functions to raise up Apache security on Debian GNU / Linux

Wednesday, July 18th, 2012

At security critical hosts running Apache + PHP based sites it is recommended functions like:

system(); exec();shell_exec();.....

to be disabled. The reason is to mainly harden against script kiddies who might exploit your site/s and upload some shitty SK tool like PHP WebShell, PHP Shell and the probably thousands of “hacker” variations that exist nowdays.

In latest Debian stable Squeeze, suhosin – advanced protection module for php5 is being installed and enabled in Apache (by default).
Simply disabling a number of functions using suhosin, could prevent multiple of future headaches and hours of pondering on who 0wn3d your server ….

Disabling the basic PHP system(); and other similar functions which allows shell spawn is not always possible, since some websites or CMS platforms depends on them for proper runnig, anyways whether it is possible disabling ’em is a must.
There are two ways to disable system(); functions; One is through using /etc/php5/apache2/conf.d/suhosin.ini and 2nd by adding a list of functions that has to be disabled directly in Website Virtualhost file or in apache2.conf (/etc/apache2/apache2.conf;
For people hosting multiple virtualhost websites on the same server using the custom domain Virtualhost method is probably better, since on a global scale the functions could be enabled if some of the websites hosted on the server requires exec(); to work OK. In any case using /etc/php5/apache2/conf.d/suhosin.ini to disable system(); functions in PHP is less messy …

1. Disabling PHP system(); fuctions through /etc/apache2/apache2.conf and custom site Vhosts

Place somewhere (I prefer near the end of config);;;

php_admin_flag safe_mode on php_admin_value disable_functions "system, exec, shell_exec, passthru , ini_alter, dl, pfsockopen, openlog, syslog, readlink, symlink, link, leak, fsockopen, popen, escapeshellcmd, apache_child_terminate apache_get_modules, apache_get_version, apache_getenv, apache_note,apache_setenv,virtual"

Disabling it for custom virtualhost is done by simply adding above Apache directvies (before the closing tag in /etc/apache2/sites-enabled/custom-vhost.com

2. Disabling PHP system();, exec(); shell spawn with suhosin.ini

In /etc/php5/apache2/conf.d/suhosin.ini add;;

suhosin.executor.func.blacklist =system, exec, shell_exec, passthru, ini_alter, dl, pfsockopen, openlog, syslog, readlink, symlink, link, leak, fsockopen, popen, escapeshellcmd, apache_child_terminate apache_get_modules, apache_get_version, apache_getenv, apache_note,apache_setenv,virtual

To do it directly via shell issue;;;

server: conf.d/# cd /etc/php5/apache2/conf.d/ server: conf.d# echo 'suhosin.executor.func.blacklist =system, exec, shell_exec, passthru, ini_alter, dl,' >> suhosin.ini server: conf.d# echo 'pfsockopen, openlog, syslog, readlink, symlink, link, leak, fsockopen, popen,' >> suhosin.ini server: conf.d# echo escapeshellcmd, apache_child_terminate apache_get_modules, apache_get_version,' >> suhosin.ini server: conf.d# echo 'apache_getenv, apache_note,apache_setenv,virtual' >> suhosin.ini

Then to re-load the memory loaded Apache libphp library an Apache restart is necessary;

server: conf.d# /etc/init.d/apache2 restart Restarting web server: apache2 ... waiting . server: conf.d#

Tadam, this should be quite a good security against annoying automated script attacks. Cheers 😉

Tags: apache security, apache2, Auto, custom, custom domain, debian gnu, Draft, escapeshellcmd, exec, func, global scale, GNU, gnu linux, harden, headaches, ini, Linux, number, passthru, php admin, php5, popen, protection, reason, runnig, safe mode, script, script kiddies, Shell, squeeze, suhosin, symlink, syslog, system, system functions, tool, two ways, VhostsPlace, Virtualhost, WebShell
Posted in Computer Security, System Administration, Web and CMS | 3 Comments »

‘host-name’ is blocked because of many connection errors; unblock with ‘mysqladmin flush-hosts’

Sunday, May 20th, 2012

My home run machine MySQL server was suddenly down as I tried to check my blog and other sites today, the error I saw while trying to open, this blog as well as other hosted sites using the MySQL was:

Error establishing a database connection

The topology, where this error occured is simple, I have two hosts:

1. Apache version 2.0.64 compiled support externally PHP scripts interpretation via libphp – the host runs on (FreeBSD)

2. A Debian GNU / Linux squeeze running MySQL server version 5.1.61

The Apache host is assigned a local IP address 192.168.0.1 and the SQL server is running on a host with IP 192.168.0.2

To diagnose the error I've logged in to 192.168.0.2 and weirdly the mysql-server was appearing to run just fine:

debian:~# ps ax |grep -i mysql
31781 pts/0 S 0:00 /bin/sh /usr/bin/mysqld_safe
31940 pts/0 Sl 12:08 /usr/sbin/mysqld –basedir=/usr –datadir=/var/lib/mysql –user=mysql –pid-file=/var/run/mysqld/mysqld.pid –socket=/var/run/mysqld/mysqld.sock –port=3306
31941 pts/0 S 0:00 logger -t mysqld -p daemon.error
32292 pts/0 S+ 0:00 grep -i mysql

Moreover I could connect to the localhost SQL server with mysql -u root -p and it seemed to run fine. The error Error establishing a database connection meant that either something is messed up with the database or 192.168.0.2 Mysql port 3306 is not properly accessible.

My first guess was something is wrong due to some firewall rules, so I tried to connect from 192.168.0.1 to 192.168.0.2 with telnet:

freebsd# telnet 192.168.0.2 3306
Trying 192.168.0.2…
Connected to jericho.
Escape character is '^]'.
Host 'webserver' is blocked because of many connection errors; unblock with 'mysqladmin flush-hosts'
Connection closed by foreign host.

Right after the telnet was initiated as I show in the above output the connection was immediately closed with the error:

Host 'webserver' is blocked because of many connection errors; unblock with 'mysqladmin flush-hosts'Connection closed by foreign host.

In the error 'webserver' is my Apache machine set hostname. The error clearly states the problems with the 'webserver' apache host unable to connect to the SQL database are due to 'many connection errors' and a fix i suggested with mysqladmin flush-hosts

To temporary solve the error and restore my normal connectivity between the Apache and the SQL servers I logged I had to issue on the SQL host:

mysqladmin -u root -p flush-hostsEnter password:

Thogh this temporar fix restored accessibility to the databases and hence the websites errors were resolved, this doesn't guarantee that in the future I wouldn't end up in the same situation and therefore I looked for a permanent fix to the issues once and for all.

The permanent fix consists in changing the default value set for max_connect_error in /etc/mysql/my.cnf, which by default is not too high. Therefore to raise up the variable value, added in my.cnf in conf section [mysqld]:

debian:~# vim /etc/mysql/my.cnf ... max_connect_errors=4294967295

and afterwards restarted MYSQL:

debian:~# /etc/init.d/mysql restart Stopping MySQL database server: mysqld. Starting MySQL database server: mysqld. Checking for corrupt, not cleanly closed and upgrade needing tables..

To make sure the assigned max_connect_errors=4294967295 is never reached due to Apache to SQL connection errors, I've also added as a cronjob.

debian:~# crontab -u root -e 00 03 * * * mysqladmin flush-hosts

In the cron I have omitted the mysqladmin -u root -p (user/pass) input options because for convenience I have already stored the mysql root password in /root/.my.cnf

Here is how /root/.my.cnf looks like:

debian:~# cat /root/.my.cnf [client] user=root password=a_secret_sql_password

Now hopefully, this would permanently solve SQL's 'failure to accept connections' due to too many connection errors for future.

Tags: apache version, Auto, basedir, bin, cnf, connection, connectionThe, daemon, database connection, debian gnu, default, Draft, due, error error, firewall rules, fix, freebsd, GNU, guess, host, host name, hostname, lib, Linux, local ip address, localhost, machine, mysql server, mysqladmin, mysqld, mysqlMoreover, nbsp, occured, password, port 3306, root, root password, running, server version, sl 12, Socket, something, SQL, sql server, squeeze, support, topology, value, webserver
Posted in MySQL, System Administration, Web and CMS | No Comments »

How to permanently enable Cookies in Lynx text browser – Disable accept cookies prompt in lynx console browser

Wednesday, April 18th, 2012

The default behaviour of lynx – console text browser on Linuces, BSD and other free OSes is to always ask, for the accept cookies prompt once an internet web page is opened that requires browser cookies to be enabled.

I should admin, having this "secure by default" (always ask for new cookies) behaviour in lynx was a good practice from a security point of view.

Another reason, why this cookies prompt is enabled by default is back in the days, when lynx was actively developed by programmers the websites with cookies support was not that many and even cookies was mostly required for user/pass authentication (all those who still remember this days the websites that requires authentication was a way less than today) …
With this said the current continuing security cautious behaviour in the browser, left from its old days is understandable.

Screenshot Google Accept cookies Lynx dialog FreeBSD

However I personally sometimes, need to use lynx more frequently and this behaviour of always opening a new website in text mode in console to prompts me for a cookie suddenly becomes a big waste of time if you use lynx to browser more than few sites. Hence I decided to change the default way lynx handles cookies and make them enabled by default instead.
Actually even in the past, when I was mainly using internet in console on every new server or home Linux install, I was again making the cookies to be permanently accepted.
Everyone who used lynx a few times already knows its "annoying" to all time accept cookie prompts … This provoked me to write this short article to explain how enabling of constant cookie accepting in lynx is done

To enable the persistent cookies in lynx, one needs to edit lynx.cfg on different GNU / Linux and BSD* distributions lynx.cfg is located in different directory.

Most of the lynx.cfg usual locations are /etc/lynx/lynx.cfg or /etc/lynx.cfg as of time of writting this post in Debian Squeeze GNU / Linux the lynx.cfg is located in /etc/lynx-cur/lynx.cfg, whether for FreeBSD / NetBSD / OpenBSD users the file is located in /usr/local/etc/lynx.cfg

What I did to allow all cookies is open lynx.cfg in vim edit and change the following lines:

#FORCE_SSL_COOKIES_SECURE:FALSE

with

FORCE_SSL_COOKIES_SECURE:TRUE

#SET_COOKIES:TRUE

uncomment it to:

SET_COOKIES:TRUE

c) next, change

ACCEPT_ALL_COOKIES:FALSE

ACCEPT_ALL_COOKIES:TRUE

Onwards opening any website with lynx auto-accepts the cookies.

lynx Always allowing from domain cookies Linux screenshot

Google in Bulgarian Lynx browser screenshot

For people who care about there security (who still browse in console (surely not many anymore)), permanently allowing the cookies is not a good idea. But for those who are ready to drop off little security for convenience its ok.

Tags: ALL, authentication, Auto, browser cookies, BSD, bsd distributions, cfg, change, convenience, Cookie, default behaviour, Draft, everyone, file, free oses, GNU, gnu linux, good, How to, internet web, Linux, Lynx, lynx one, NetBSD, Onwards, OpenBSD, page, persistent cookies, point of view, programmers, quot, reason, security point, squeeze, support, text, text browser, text mode, time, TRUEb, TRUEc, TRUEuncomment, uncomment, use, using internet, vim, waste, waste of time, web page, writ
Posted in Curious Facts, Everyday Life, FreeBSD, Linux, System Administration | 1 Comment »

How to quickly check unread Gmail emails on GNU / Linux – one liner script

Monday, April 2nd, 2012

I've hit an interesting article explaining how to check unread gmail email messages in Linux terminal. The original article is here

Being able to read your latest gmail emails in terminal/console is great thing, especially for console geeks like me.
Here is the one liner script:

curl -u GMAIL-USERNAME@gmail.com:SECRET-PASSWORD \ --silent "https://mail.google.com/mail/feed/atom" | tr -d '\n' \ | awk -F '' '{for (i=2; i<=NF; i++) {print $i}}' \ | sed -n "s/

Linux Users Group M. – [7] discussions, [10] comments and [2] jobs on LinkedIn Twitter – Lynn Serafinn (@LynnSerafinn) has sent you a direct message on Twitter! Facebook – Sys, you have notifications pending Twitter – Email Marketing (@optinlists) is now following you on Twitter! Twitter – Lynn Serafinn (@LynnSerafinn) is now following you on Twitter! NutshellMail – 32 New Messages for Sat 3/31 12:00 PM Linux Users Group M. – [10] discussions, [5] comments and [8] jobs on LinkedIn eBay – Deals up to 60% OFF + A Sweepstakes! LinkedIn Today – Top news today: The Magic of Doing One Thing at a Time NutshellMail – 29 New Messages for Fri 3/30 12:00 PM Linux Users Group M. – [16] discussions, [8] comments and [8] jobs on LinkedIn Ervan Faizal Rizki . – Join my network on LinkedIn Twitter – LEXO (@LEXOmx) retweeted one of your Tweets! NutshellMail – 24 New Messages for Thu 3/29 12:00 PM Facebook – Your Weekly Facebook Page Update Linux Users Group M. – [11] discussions, [9] comments and [16] jobs on LinkedIn

As you see this one liner uses curl to fetch the information from mail.google.com's atom feed and then uses awk and sed to parse the returned content and make it suitable for display.

If you want to use the script every now and then on a Linux server or your Linux desktop you can download the above code in a script file -quick_gmail_new_mail_check.sh here

Here is a screenshot of script's returned output:

A good use of a modified version of the script is in conjunction with a 15 minutes cron job to launch for new gmail mails and launch your favourite desktop mail client.
This method is useful if you don't want a constant hanging Thunderbird or Evolution, pop3 / imap client on your system to just take up memory or dangle down the window list.
I've done a little modification to the script to simply, launch a predefined email reader program, if gmail atom feed returns new unread mails are available, check or download my check_gmail_unread_mail.sh here
Bear in mind, on occasions of errors with incorrect username or password, the script will not return any errors. The script is missing a properer error handling.Therefore, before you use the script make sure:

gmail_username='YOUR-USERNAME'; gmail_password='YOUR-PASSWORD';

are 100% correct.

To launch the script on 15 minutes cronjob, put it somewhere and place a cron in (non-root) user:

# crontab -u root -e ... */15 * * * * /path/to/check_gmail_unread_mail.sh

Once you read your new emails in lets say Thunderbird, close it and on the next delivered unread gmail mails, your mail client will pop up by itself again. Once the mail client is closed the script execution will be terminated.
Consised that if you get too frequently gmail emails, using the script might be annoying as every 15 minutes your mail client will be re-opened.

If you use any of the shell scripts, make sure there are well secured (make it owned only by you). The gmail username and pass are in plain text, so someone can steal your password, very easily. For a one user Linux desktops systems as my case, security is not such a big concern, putting my user only readable script permissions (e.g. chmod 0700)is enough.

Tags: article, atom, Auto, awk, client, com, cron, Desktop, download, Draft, ebay, email, email marketing, email messages, facebook, Fri, geeks, Gmail, GMAIL-USERNAME, GNU, gnu linux, google, Group, https mail, liner, LinkedIn, Linux, linux server, linux users group, mail, marketing, news today, OFF, original article, password, quot, rizki, sat 3, script, SECRET-PASSWORD, terminal, thing, top news, twitter, username, Users, Weekly
Posted in Linux, System Administration, Various | 2 Comments »

Geeqie and GThumb advanced picture viewing Picasa alternative for GNU / Linux – GQView on Debian now a meta package for Geeqie

Saturday, March 31st, 2012

I'm currently learning some basic graphic design – reading GIMP's documentation etc. While reading http://docs.gimp.org, I saw a reference to Gthumb and GQView picture viewing apps, so I thought of installing them on my Debian LinuxGQView has a lot of santimental value to me as it reminds me of the the old times when I used gqview as a default picture viewing program on a old machine running Debian Woody Linux with Window Maker as desktop environment.

I wanted to see how GQView looks nowdays so installed it:

noah:~# apt-get --yes install gqview

And onwards launch it:

hipo@noah:~/Desktop$ gqview This is a compatibility alias for Geeqie! Please use /usr/bin/geeqie instead! Creating Geeqie dir:/home/hipo/.config/geeqie Creating Geeqie dir:/home/hipo/.local/share/geeqie/collections Creating Geeqie dir:/home/hipo/.cache/geeqie/thumbnails Creating Geeqie dir:/home/hipo/.local/share/geeqie/metadata

GQview Debian linux screenshot ancient icon of saint Paul from Ephesus

What poped up instead of gqview is Geeqie – a picture viewer nowdays available on a default Slackware Linux install.

Fedora, CentOS users will have to build geekie from its source, as of time of writting there is no available rpm package.

It was new to me Gqview is no longer developed, its dev is forked (because its head developer is not reachable any more). Hence QGView continues to live on nowdays under the hood of Geekie
As you can see from the prior screenshot Geeqie has very similar interface to GQView. Going through the interface, I've found it has much more features than GQView. You see one of the many new nice features is the support for drawing Image Histograms.
There are few type of Histograms to display in Geekiq, available by navigating to:

View -> Image Overlay

Even if you're not familiar with Image histograms, probably you have seen them appear on a digital camera while browsing in menus. Its likely you didn't know what you saw a digital camera display is a histogram.
Anyways being not familiar with histograms is perfectly fine as for most of us (regular) users image historograms doesn't make much sense.

BTW Histograms are very useful for pro Photographers. Photographers can use them as an aid to show the distribution of picture "tones" captured, and whether image detail has been lost to blown-out highlights or blacked-out shadows.
Interesting fact concerning "reading" and understanding Histograms is on a Histogram for a very dark image the majority of data points are on the left side and center of the graph, whether histogram for a very bright image with few dark areas and/or shadows will have most of its data points located on the right side and center of the graph.
With this said in mind, you can see, the above Geeqie picture visibile histogram obviously has most of its data concentrated on the right and the center so this means the histogram belongs to a bright pic. Well, Actually you can see this without a histogram too 😉
Though Image Histograms might seem pretty useles they're very much needed in Professional Graphic Manipulation. For instance for correction of image color gamma or manually adjusting the brightness for each picture pixel brightness.
So Enough with histograms, I will switch now to a short review of Gthumb

Since I haven't used/seen Gthumb "for ages", I was also curious how the program looks nowdays. I did not needed to install gthumb, as I had it installed already on my notebook. I assume gthumb package was installed as some package dependency or I did it install some very long time ago and I forgot.

For people who didn't have it install do:

noah:~# apt-get --yes install gthumb ...

This installs a Gthumb version 2.12.2. The current latest stable Gthumb release is way ahead from the existing deb stable package, the latest available version on sourceforge is 2.7.4

Ubuntu gthumb latest packages would probably be newer than my debian installed one, so Ubuntu users can have the joy to use a newer version of gthumb…

gthumb is also installable for Fedora and CentOS users by default from default assigned package repos:

To install on RPM based Linux distro:

[root@centos:~ ]# yum -y install gthumb ...

Once installed, I ran it via a gnome run application shortcut ALT+F2 and typed:

gthumb

If you're more of a gui user than me you can run it also through GNOME menus:

Applications -> Graphics -> Gthumb Image Viewer

Gthumb Debian Linux Squeeze Screenshot

You see gthumb appears, quite similar in "look&feel" to Geeqie.
I went through the program options just for the sake to compare with Geeqie. Based on the fact Geeqie seems to have more functionality it is probably superior and better choice for people looking for professional image vieweing / editting.However there are some other aspects I've noticed, where it lacks behind Gthumb.
Gthumb has few great functionality which unfortunately as of time of writting are missing or hardly achievable in Geeqie

1. Bookmarking pictures.

This allows for your favourite pictures to be bookmarked in the same way as bookmarks work in a browser, really neat feature.

2. Share functionality

Share button is located on the rightest upper corner and makes sharing with major web storage places easy.
Through the Share button, you can export (share) your pictures directly in Picasa, Facebook,Flickr and Photobucket.
Gthumb Share button screenshot on my Debian Squeeze Linux

Through this share button it is also possible to "share your photos on a CD or DVD", – burn them on a CD/DVD directly from gthumb. This is doable via:

Gthumb Share Button list options

One other Gthumb advantagle is its extensibility. It has a powerful plugin system. Many of the features it has are already there thanks to its enabled extensions.

GThumb extensions menu Debian GNU / Linux

3. Photo Tagging

Nowdays tagging, photos has become so popular with the boom of facebook tagging. Gthumb has a relatively easy to use image tagging integrated.
tagging a picture is done by simply going through the picture and selecting it with the (rightest located mouse button).

GThumb picture properties tag menu shot

There is a list of few already predefined tags, which can be used to tag images.Assigning more tags is done via menu:

Tags -> Other

Gthumb assign new tag category menu shot

4. Organizing pictures

Through the Organize button located right below the Share button, one can easily organize his picture collections, based on few criterias.

GTHUMB organize files Debian cropped screenshot

The organize easy feature is not so important for not people, who are not in depth in photographing but for professional photographers this is an absolute necessity. For people who are image freaks and store tens of thousands of pictures organizing pictures for easy later retrieval is really nice.
To sum it, up Geeqie and Gthumb are richer in functionality if compared to GNOME's default pic viewer (eye of the gnome) – eog.

Geeqie, as the name suggests is more for geek oriented and has a lot of great options which can serve well for advanced computer users. For an ordinary desktop user however it will probably be not easy to "digest". Gthumb on the other hand is ideal for anyone who prefers gui convenience. Gthumb is a good option for anyone looking for some similar to Picasa free software program that is feature rich and easy to use.For people who has to review a dozens of pictures daily, associating one of the two as a default picture viewer will probably be useful.

Tags: alias, alternative, Auto, center, CentOS, config, debian woody, default, design reading, Desktop, desktop environment, digital camera, Draft, fedora, geekie, GIMP, GNU, gnu linux, gqview, graphic design, head developer, hipo, histogram, histograms, image overlay, LinuxGQView, machine, meta package, metadata, noah, package, Photographers, picture, poped, quot, reading, rpm, screenshot, time, Ubuntu, value, window, woody linux, writting
Posted in Gnome, Linux and FreeBSD Desktop, News | No Comments »

How to reduce spam in PHPBB based internet forum on Debian GNU / Linux

Monday, March 26th, 2012

phpbb reduce spam bot registrations on Debian Linux tiny script

I had to install two PHPBB based internet forums, some long time ago. Since long time passed and I haven't checked what's happening with them I just noticed. They start filling up spam threads. The phpbb installations are done using the standard shipped deb packages in Debian Linux Lenny.

After checking online, I found one smart solution to . The idea is very simple most spam bots are written in a way that they don't have a properly set timezone. Therefore the quickest way to get rid of spam bots which try to auto register and put spam content inside the a forum category or post is to add a simple if condition in php to check the browser set timezone:

The file to add the php if condition is ucp_register.phpThe phpbb package install places default phpbb path on Debian is /usr/share/phpbb3/ and hence the file I had to modify is located in:

/usr/share/phpbb3/www/includes/ucp/ucp_register.php

To make the TZ check one needs to modify ../www/includes/ucp/ucp_register.php and look for php array definition:

$data = array( 'username' => utf8_normalize_nfc(request_var('username', '', true)), 'new_password' => request_var('new_password', '', true), 'password_confirm' => request_var('password_confirm', '', true), 'email' => strtolower(request_var('email', '')), 'email_confirm' => strtolower(request_var('email_confirm', '')), 'confirm_code' => request_var('confirm_code', ''), 'lang' => basename(request_var('lang', $user->lang_name)), 'tz' => request_var('tz', (float) $timezone), );

Right after this chunk of code add the if condition code which is like so:

if ($data['tz'] == '-12.00') { die('Die, bot! Die.'); }

From now onwards, any attempt for new user registration with an incorrect timezone of -12.00 will be immediately stopped while the forum spammer bot will be offered an empty page 🙂

Another good practice is to disable Birthday Listing from phpbb Admin Control panel (ACP). Go to menus:

ACP -> General -> Board Settings -> Enable Birthday listing: (No)

Enable birthday listing phpbb forum screenshot

I like disabling birthday listing, as when it is enabled and you have some spammer registrations, which even though didn't succeeded to contaminate your forum content has specified a birthday and therefore there profiles gets popping up each different day on the main page of the forum.
This will not eradicate all spammer bots, but at least will significantly decrease spammer bot registrations.

Tags: array definition, Auto, checking, chunk, code 39, deb, deb packages, Debian, debian gnu, debian linux, Draft, email, forum, forum category, GNU, Internet, internet forum, internet forums, Linux, long time, online, PHPBB, smart solution, solution, spam, spam bots, threads, time
Posted in System Administration, Web and CMS | No Comments »

FreeBSD Jumbo Frames network configuration short how to

Wednesday, March 14th, 2012

Recently I wrote a post on how to enable Jumbo Frames on GNU / Linux , therefore I thought it will be useful to write how Jumbo Frames network boost can be achieved on FreeBSD too.

I will skip the details of what is Jumbo Frames, as in the previous article I have thoroughfully explained. Just in short to remind you what is Jumbo Frames and why you might need it? – it is a way to increase network MTU transfer frames from the MTU 1500 to MTU of 9000 bytes

It is interesting to mention that according to specifications, the maximum Jumbo Frames MTU possible for assignment are of MTU=16128
Just like on Linux to be able to take advantage of the bigger Jumbo Frames increase in network thoroughput, you need to have a gigabyt NIC card/s on the router / server.

1. Increasing MTU to 9000 to enable Jumbo Frames "manually"

Just like on Linux, the network tool to use is ifconfig. For those who don't know ifconfig on Linux is part of the net-tools package and rewritten from scratch especially for GNU / Linux OS, whether BSD's ifconfig is based on source code taken from 4.2BSD UNIX

As you know, network interface naming on FreeBSD is different, as there is no strict naming like on Linux (eth0, eth1, eth2), rather the interfaces are named after the name of the NIC card vendor for instance (Intel(R) PRO/1000 NIC is em0), RealTek is rl0 etc.

To set Jumbro Frames Maximum Transmission Units of 9000 on FreeBSD host with a Realtek and Intel gigabyt ethernet cards use: freebsd# /sbin/ifconfig em0 192.168.1.2 mtu 9000 freebsd# /sbin/ifconfig rl0 192.168.2.2 mtu 9000

!! Be very cautious here, as if you're connected to the system remotely over ssh you might loose connection to it because of broken routing.

To prevent routing loss problems, if you're executing the above two commands remotely, you better run them in GNU screen session:

freebsd# screen freebsd# /sbin/ifconfig em0 192.168.1.2 mtu 9000; /sbin/ifconfig rl0 192.168.1.2 mtu 9000; \ /etc/rc.d/netif restart; /etc/rc.d/routed restart

2. Check MTU settings are set to 9000

If everything is fine the commands will return empty output, to check further the MTU is properly set to 9000 issue:

freebsd# /sbin/ifconfig -a|grep -i em0em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000freebsd# /sbin/ifconfig -a|grep -i rl0 rl0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000

3. Reset routing for default gateway

If you have some kind of routing assigned for em0 and rl0, network interfaces they will be affected by the MTU change and the routing will be gone. To reset the routing to the previously properly assigned routing, you have to restart the BSD init script taking care for assigning routing on system boot time:

freebsd# /etc/rc.d/routing restart default 192.168.1.1 done add net default: gateway 192.168.1.1 Additional routing options: IP gateway=YES.

4. Change MTU settings for NIC card with route command

There is also a way to assign higher MTU without "breaking" the working routing, e.g. avoiding network downtime with bsd route command:

freebsd# grep -i defaultrouter /etc/rc.conf defaultrouter="192.168.1.1" freebsd# /sbin/route change 192.168.1.1 -mtu 9000 change host 192.168.1.1

5. Finding the new MTU NIC settings on the FreeBSD host

freebsd# /sbin/route -n get 192.168.1.1 route to: 192.168.1.1 destination: 192.168.1.1 interface: em0 flags: <UP,HOST,DONE,LLINFO,WASCLONED> recvpipe sendpipe ssthresh rtt,msec rttvar hopcount mtu expire 0 0 0 0 0 0 9000 1009

6. Set Jumbo Frames to load automatically on system load

To make the increased MTU to 9000 for Jumbo Frames support permanent on a FreeBSD system the /etc/rc.conf file is used:

The variable for em0 and rl0 NICs are ifconfig_em0 and ifconfig_rl0.
The lines to place in /etc/rc.conf should be similar to:

ifconfig_em0="inet 192.168.1.1 netmask 255.255.255.0 media 1000baseTX mediaopt half-duplex mtu 9000" ifconfig_em0="inet 192.168.1.1 netmask 255.255.255.0 media 1000baseTX mediaopt half-duplex mtu 9000"

Change in the above lines the gateway address 192.168.1.1 and the netmask 255.255.255.0 to yours corresponding gw and netmask.
Also in the above example you see the half-duplex ifconfig option is set insetad of full-duplex in order to prevent some duplex mismatches. A full-duplex could be used instead, if you're completely sure on the other side of the host is configured to support full-duplex connections. Otherwise if you try to set full-duplex with other side set to half-duplex or auto-duplex a duplex mismatch will occur. If this happens insetad of taking the advantage of the Increase Jumbo Frames MTU the network connection could become slower than originally with standard ethernet MTU of 1500. One other bad side if you end up with duplex-mismatch could be a high number of loss packets and degraded thoroughout …

7. Setting Jumbo Frames for interfaces assigning dynamic IP via DHCP

If you need to assign an MTU of 9000 for a gigabyt network interfaces, which are receiving its TCP/IP network configuration over DHCP server.
First, tell em0 and rl0 network interfaces to dynamically assign IP addresses via DHCP proto by adding in /etc/rc.conf:

ifconfig_em0="DHCP" ifconfig_rl0="DHCP"

Secondly make two files /etc/start_if.em0 and /etc/start_if.rl0 and include in each file:

ifconfig em0 media 1000baseTX mediaopt full-duplex mtu 9000 ifconfig rl0 media 1000baseTX mediaopt full-duplex mtu 9000

Copy / paste in root console:

echo 'ifconfig em0 media 1000baseTX mediaopt full-duplex mtu 9000' >> /etc/start_if.em0 echo 'ifconfig rl0 media 1000baseTX mediaopt full-duplex mtu 9000' >> /etc/start_if.rl0

Finally, to load the new MTU for both interfaces, reload the IPs with the increased MTUs:

freebsd# /etc/rc.d/routing restart default 192.168.1.1 done add net default: gateway 192.168.1.1

8. Testing if Jumbo Frames is working correctly

To test if an MTU packs are transferred correctly through the network you can use ping or tcpdumpa.) Testing Jumbo Frames enabled packet transfers with tcpdump

freebsd# tcpdump -vvn | grep -i 'length 9000'

You should get output like:

16:40:07.432370 IP (tos 0x0, ttl 50, id 63903, offset 0, flags [DF], proto TCP (6), length 9000) 192.168.1.2.80 > 192.168.1.1.60213: . 85825:87285(1460) ack 668 win 14343 16:40:07.432588 IP (tos 0x0, ttl 50, id 63904, offset 0, flags [DF], proto TCP (6), length 9000) 192.168.1.2.80 > 192.168.1.1.60213: . 87285:88745(1460) ack 668 win 14343 16:40:07.433091 IP (tos 0x0, ttl 50, id 63905, offset 0, flags [DF], proto TCP (6), length 9000) 192.168.1.2.80 > 192.168.1.1.60213: . 23153:24613(1460) ack 668 win 14343 16:40:07.568388 IP (tos 0x0, ttl 50, id 63907, offset 0, flags [DF], proto TCP (6), length 9000) 192.168.1.2.80 > 192.168.1.1.60213: . 88745:90205(1460) ack 668 win 14343 16:40:07.568636 IP (tos 0x0, ttl 50, id 63908, offset 0, flags [DF], proto TCP (6), length 9000) 192.168.1.2.80 > 192.168.1.1.60213: . 90205:91665(1460) ack 668 win 14343 16:40:07.569012 IP (tos 0x0, ttl 50, id 63909, offset 0, flags [DF], proto TCP (6), length 9000) 192.168.1.2.80 > 192.168.1.1.60213: . 91665:93125(1460) ack 668 win 14343 16:40:07.569888 IP (tos 0x0, ttl 50, id 63910, offset 0, flags [DF], proto TCP (6), length 9000) 192.168.1.2.80 > 192.168.1.1.60213: . 93125:94585(1460) ack 668 win 14343

b.) Testing if Jumbo Frames are enabled with ping

– Testing Jumbo Frames with ping command on Linux

linux:~# ping 192.168.1.1 -M do -s 8972 PING 192.168.1.1 (192.168.1.1) 8972(9000) bytes of data. 9000 bytes from 192.168.1.1: icmp_req=1 ttl=52 time=43.7 ms 9000 bytes from 192.168.1.1: icmp_req=2 ttl=52 time=43.3 ms 9000 bytes from 192.168.1.1: icmp_req=3 ttl=52 time=43.5 ms 9000 bytes from 192.168.1.1: icmp_req=4 ttl=52 time=44.6 ms --- 192.168.0.1 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3003ms rtt min/avg/max/mdev = 2.397/2.841/4.066/0.708 ms

If you get insetad an an output like:

From 192.168.1.2 icmp_seq=1 Frag needed and DF set (mtu = 1500) From 192.168.1.2 icmp_seq=1 Frag needed and DF set (mtu = 1500) From 192.168.1.2 icmp_seq=1 Frag needed and DF set (mtu = 1500) From 192.168.1.2 icmp_seq=1 Frag needed and DF set (mtu = 1500)

--- 192.168.1.1 ping statistics --- 0 packets transmitted, 0 received, +4 errors

This means a packets with maximum MTU of 1500 could be transmitted and hence something is not okay with the Jumbo Frames config.
Another helpful command in debugging MTU and showing which host in a hop queue support jumbo frames is Linux's traceroute

To debug a path between host and target, you can use:

linux:~# traceroute --mtu www.google.com ...

If you want to test the Jumbo Frames configuration from a Windows host use ms-windows ping command like so:

C:\>ping 192.168.1.2 -f -l 8972 Pinging 192.168.1.2 with 8972 bytes of data: Reply from 192.168.1.2: bytes=8972 time=2ms TTL=255 Reply from 192.168.1.2: bytes=8972 time=2ms TTL=255 Reply from 192.168.1.2: bytes=8972 time=2ms TTL=255 Reply from 192.168.1.2: bytes=8972 time=2ms TTL=255 Ping statistics for 192.168.1.2: Packets: Sent = 4, Received = 4, Lost = 0 (0% loss), Approximate round trip times in milli-seconds: Minimum = 2ms, Maximum = 2ms, Average = 2ms

Here -l 8972 value is actually equal to 9000. 8972 = 9000 – 20 (20 byte IP header) – 8 (ICMP header)

Tags: article, Auto, baseTX, boost, BSD, card, card vendor, change, code, Configuration, connection, Draft, eth, ethernet cards, Frames, freebsd, gateway, gigabyt, GNU, gnu screen, host, Jumbo, jumbo frames, Linux, linux os, maximum transmission, mediaopt, MTU, Netmask, network, network configuration, network interface, network tool, nic card, package, post, quot, Realtek, router server, scratch, screen, screen session, source code, ssh, tool, transmission units, way
Posted in FreeBSD, System Administration | 1 Comment »

☩ Walking in Light with Christ – Faith, Computing, Diary

Posts Tagged ‘GNU’

Geeqie and GThumb advanced picture viewing Picasa alternative for GNU / Linux – GQView on Debian now a meta package for Geeqie

Daily Bible quote

GET ARTICLE UPDATES

Useful blog? Help it:

Links to Other Places

Recent Posts

Ads

Categories

About Myself

Recent Comments

Top Post Views

blogtopsites