Posts Tagged ‘Free’

Fix Out of inodes on Postfix Linux Mail Cluster. How to clean up filesystem running out of Inodes, Filesystem inodes on partition is 100% full

Wednesday, August 25th, 2021

Inode_Entry_inode-table-content

Recently we have faced a strange issue with with one of our Clustered Postfix Mail servers (the cluster is with 2 nodes that each has configured Postfix daemon mail servers (running on an OpenVZ virtualized environment).
A heartbeat that checks liveability of clusters and switches nodes in case of one of the two gets broken due to some reason), pretty much a standard SMTP cluster.

So far so good but since the cluster is a kind of abondoned and is pretty much legacy nowadays and used just for some Monitoring emails from different scripts and systems on servers, it was not really checked thoroughfully for years and logically out of sudden the alarming email content sent via the cluster stopped working.

The normal sysadmin job here  was to analyze what is going on with the cluster and fix it ASAP. After some very basic analyzing we catched the problem is caused by a  "inodes full" (100% of available inodes were occupied) problem, e.g. file system run out of inodes on both machines perhaps due to a pengine heartbeat process  bug  leading to producing a high number of .bz2 pengine recovery archive files stored in /var/lib/pengine>

Below are the few steps taken to analyze and fix the problem.
 

1. Finding out about the the system run out of inodes problem


After logging on to system and not finding something immediately is wrong with inodes, all I can see from crm_mon is cluster was broken.
A plenty of emails were left inside the postfix mail queue visible with a standard command

[root@smtp1: ~ ]# postqueue -p

It took me a while to find ot the problem is with inodes because a simple df -h  was showing systems have enough space but still cluster quorum was not complete.
A bit of further investigation led me to a  simple df -i reporting the number of inodes on the local filesystems on both our SMTP1 and SMTP2 got all occupied.

[root@smtp1: ~ ]# df -i
Filesystem            Inodes   IUsed   IFree IUse% Mounted on
/dev/simfs            500000   500000  0   100% /
none                   65536      61   65475    1% /dev

As you can see the number of inodes on the Virual Machine are unfortunately depleted

Next step was to check directories occupying most inodes, as this is the place from where files could be temporary moved to a remote server filesystem or moved to another partition with space on a server locally attached drives.
Below command gives an ordered list with directories locally under the mail root filesystem / and its respective occupied number files / inodes,
the more files under a directory the more inodes are being occupied by the files on the filesystem.

 

run-out-if-inodes-what-is-inode-find-out-which-filesystem-or-directory-eating-up-all-your-system-inodes-linux_inode_diagram.gif
1.1 Getting which directory consumes most of the inodes on the systems

 

[root@smtp1: ~ ]# { find / -xdev -printf '%h\n' | sort | uniq -c | sort -k 1 -n; } 2>/dev/null
….
…..

…….
    586 /usr/lib64/python2.4
    664 /usr/lib64
    671 /usr/share/man/man8
    860 /usr/bin
   1006 /usr/share/man/man1
   1124 /usr/share/man/man3p
   1246 /var/lib/Pegasus/prev_repository_2009-03-10-1236698426.308128000.rpmsave/root#cimv2/classes
   1246 /var/lib/Pegasus/prev_repository_2009-05-18-1242636104.524113000.rpmsave/root#cimv2/classes
   1246 /var/lib/Pegasus/prev_repository_2009-11-06-1257494054.380244000.rpmsave/root#cimv2/classes
   1246 /var/lib/Pegasus/prev_repository_2010-08-04-1280907760.750543000.rpmsave/root#cimv2/classes
   1381 /var/lib/Pegasus/prev_repository_2010-11-15-1289811714.398469000.rpmsave/root#cimv2/classes
   1381 /var/lib/Pegasus/prev_repository_2012-03-19-1332151633.572875000.rpmsave/root#cimv2/classes
   1398 /var/lib/Pegasus/repository/root#cimv2/classes
   1696 /usr/share/man/man3
   400816 /var/lib/pengine

Note, the above command orders the files from bottom to top order and obviosuly the bottleneck directory that is over-eating Filesystem inodes with an exceeding amount of files is
/var/lib/pengine
 

2. Backup old multitude of files just in case of something goes wrong with the cluster after some files are wiped out


The next logical step of course is to check what is going on inside /var/lib/pengine just to find a very ,very large amount of pe-input-*NUMBER*.bz2 files were suddenly produced.

 

[root@smtp1: ~ ]# ls -1 pe-input*.bz2 | wc -l
 400816


The files are produced by the pengine process which is one of the processes that is controlling the heartbeat cluster state, presumably it is done by running process:

[root@smtp1: ~ ]# ps -ef|grep -i pengine
24        5649  5521  0 Aug10 ?        00:00:26 /usr/lib64/heartbeat/pengine


Hence in order to fix the issue, to prevent some inconsistencies in the cluster due to the file deletion,  copied the whole directory to another mounted parition (you can mount it remotely with sshfs for example) or use a local one if you have one:

[root@smtp1: ~ ]# cp -rpf /var/lib/pengine /mnt/attached_storage


and proceeded to clean up some old multitde of files that are older than 2 years of times (720 days):


3. Clean  up /var/lib/pengine files that are older than two years with short loop and find command

 


First I made a list with all the files to be removed in external text file and quickly reviewed it by lessing it like so

[root@smtp1: ~ ]#  cd /var/lib/pengine
[root@smtp1: ~ ]# find . -type f -mtime +720|grep -v pe-error.last | grep -v pe-input.last |grep -v pe-warn.last -fprint /home/myuser/pengine_older_than_720days.txt
[root@smtp1: ~ ]# less /home/myuser/pengine_older_than_720days.txt


Once reviewing commands I've used below command to delete the files you can run below command do delete all older than 2 years that are different from pe-error.last / pe-input.last / pre-warn.last which might be needed for proper cluster operation.

[root@smtp1: ~ ]#  for i in $(find . -type f -mtime +720 -exec echo '{}' \;|grep -v pe-error.last | grep -v pe-input.last |grep -v pe-warn.last); do echo $i; done


Another approach to the situation is to simply review all the files inside /var/lib/pengine and delete files based on year of creation, for example to delete all files in /var/lib/pengine from 2010, you can run something like:
 

[root@smtp1: ~ ]# for i in $(ls -al|grep -i ' 2010 ' | awk '{ print $9 }' |grep -v 'pe-warn.last'); do rm -f $i; done


4. Monitor real time inodes freeing

While doing the clerance of old unnecessery pengine heartbeat archives you can open another ssh console to the server and view how the inodes gets freed up with a command like:

 

# check if inodes is not being rapidly decreased

[root@csmtp1: ~ ]# watch 'df -i'


5. Restart basic Linux services producing pid files and logs etc. to make then workable (some services might not be notified the inodes on the Hard drive are freed up)

Because the hard drive on the system was full some services started to misbehaving and /var/log logging was impacted so I had to also restart them in our case this is the heartbeat itself
that  checks clusters nodes availability as well as the logging daemon service rsyslog

 

# restart rsyslog and heartbeat services
[root@csmtp1: ~ ]# /etc/init.d/heartbeat restart
[root@csmtp1: ~ ]# /etc/init.d/rsyslog restart

The systems had been a data integrity legacy service samhain so I had to restart this service as well to reforce the /var/log/samhain log file to again continusly start writting data to HDD.

# Restart samhain service init script 
[root@csmtp1: ~ ]# /etc/init.d/samhain restart


6. Check up enough inodes are freed up with df

[root@smtp1 log]# df -i
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/simfs 500000 410531 19469 91% /
none 65536 61 65475 1% /dev


I had to repeat the same process on the second Postfix cluster node smtp2, and after all the steps like below check the status of smtp2 node and the postfix queue, following same procedure made the second smtp2 cluster member as expected 🙂

 

7. Check the cluster node quorum is complete, e.g. postfix cluster is operating normally

 

# Test if email cluster is ok with pacemaker resource cluster manager – lt-crm_mon
 

[root@csmtp1: ~ ]# crm_mon -1
============
Last updated: Tue Aug 10 18:10:48 2021
Stack: Heartbeat
Current DC: smtp2.fqdn.com (bfb3d029-89a8-41f6-a9f0-52d377cacd83) – partition with quorum
Version: 1.0.12-unknown
2 Nodes configured, unknown expected votes
4 Resources configured.
============

Online: [ smtp2.fqdn.com smtp1.fqdn.com ]

failover-ip (ocf::heartbeat:IPaddr2): Started csmtp1.ikossvan.de
Clone Set: postfix_clone
Started: [ smtp2.fqdn.com smtp1fqdn.com ]
Clone Set: pingd_clone
Started: [ smtp2.fqdn.com smtp1.fqdn.com ]
Clone Set: mailto_clone
Started: [ smtp2.fqdn.com smtp1.fqdn.com ]

 

8.  Force resend a few hundred thousands of emails left in the email queue


After some inodes gets freed up due to the file deletion, i've reforced a couple of times the queued mail servers to be immediately resent to remote mail destinations with cmd:

 

# force emails in queue to be resend with postfix

[root@smtp1: ~ ]# sendmail -q


– It was useful to watch in real time how the queued emails are quickly decreased (queued mails are successfully sent to destination addresses) with:

 

# Monitor  the decereasing size of the email queue
[root@smtp1: ~ ]# watch 'postqueue -p|grep -i '@'|wc -l'

Richard Stallman explaining Why IPads and Cell Phones are bad for freedom

Wednesday, July 11th, 2012

It is a public secret that Mobile Phones which does us very good and generally makes our daily lifes way easier are also a big enemy to our natural ihnibited freedom. Life has become such that it is almost inevitable to do any business or do a daily simple jobs without using Mobile Phone. There is almost none practically today that has wilfully rejected to use the mobile phone on any basis, almost anyone except some strangers like Richard Stallman and probably few others security freaks.

I've been shocked to find out the Father of Free Software (Richard Mathew Stallman), well known in the hacker dome as RMS does not own and didn't use any mobiles. The concerns he pointed are very much logical and rightful. Owning a mobile is a great security hole in personal privacy (mobile phones can be easily sniffed by Mobile Operators) as well as anyone wearing a mobile can be tracked up to 5 to 2 meters to the exact location where he is based on the mobile phone cells to which the mobile is connected.

Many people are not aware actually of the severeness of the issue of constant tracking of people everywhere through this call "goodies". Many mobile operators are already running a software which is building place behaviour patterns of every user of their mobile network. In other words, as we're used to bring and use the mobile everywhere in automated program is creating a map for each number assigned in some of the mobile operators. The gathered data about our location going habits can then be easily used as a indicator for predicting our future behaviour, bying habits (how many times we go to super-market), how many times we go to cinema, what kind of interests we hold etc. etc.
This combined with Google, account monitoring could possibly create a system similar to the old movies Big Brother, where all people goods and even attitudes or desires is monitored, influenced and controlled ….

The severeness of the future implications of this constant "personal surveillance and tracking device" as Stallman use to call it is very dangerous for our freedoms.

I tried to live without a mobile phone, just like Stallman for about months, and to tell you the truth the world around seems completely different when you decide not to use 'em. The time I lived wihtout a mobile, clearly show me we have come to the point we cannot any more live without GSM. We fall the trap of dependanding the little "talk box" communication for absolutely everything, obviously sacrificing privacy and freedom for convenience.
Mobiles are just one side of the coin, as the non-free software which is ruling the software market and the use of computers puts another treat and takes away many foundamential freedoms we used to have in the less technological world.

Apple as a vendor of software and hardware also denies and breaks our freedom very badly, as the company tracks everyone who owns anything created by apple connected to the internet. Besides that non-free software producers, could change the user software with a press of a button giving them the opportunity to decide what is good and bad for us, leaving us at a state of a helpless dependable users.

The topic of technological little-by-little enslavement, we're going through nowdays and the denying freedoms, we experience while being convinced by companies that we became more free by each next mambo-jambo gadget or by owning the latest smart-phone is very huge and complex but unfortunately underseen in society. I don't understand why, is it due to the low technical skills of mass users is it due to a "not-care what will happen in future" attitude, but obviously people openly discussing or protesting the technologization taking away our freedom is almost zero ….

Here is the video I found in youtube in which Stallman is asked few, questions on Ipads (IBADS) and Mobile Phone use. I believe his short explanation synthesizes the problem quite well ;;;;

I just wonder after you check the video, Would you still accept an Ipad as a birthday gift ? 🙂
Do you still think cell-phones are "good" freedom safe and reliable ?

The lack of sharing in modern world – One more reason why sharing Movies and any data on the Internet should be always Legal

Saturday, July 7th, 2012

Importance of sharing in modern digital society, sharing should be legal, Sharing caring
 I've been thinking for a lot of time analyzing my already years ongoing passion for Free Software, trying to answer the question "What really made me be a keen user and follower of the ideology of the free software movement"?
I came to the conclusion it is the sharing part of free software that really made me a free software enthusiast. Let me explain ….

In our modern world sharing of personal goods (physical goods, love for fellows, money, resources etc.) has become critically low.The reason is probably the severely individualistic Western World modern culture model which seems to give good economic results.
Though western society might be successful in economic sense in man plan it is a big failure.
The high standard in social culture, the heavy social programming, high level of individualism and the collapsing spirituality in majority of people is probably the major key factors which influenced the modern society to turn into such a non-sharing culture that is almost ruling the whole world nations today.

If we go back a bit in time, one can easily see the idea and general philosophy of sharing is very ancient in nature. It was sharing that for years helped whole societies and culture grow and mature. Sharing is a fundamental part of Christian faith and many other religions as well and has been a people gathering point  for centuries.
However as modern man is more and more turning to the false fables of the materialistic origin of  man (Darwininsm), sharing is started seeing as unnecessary . Perhaps the decreased desire in people to share is also the reason why in large number people started being  self-interest oriented as most of us are nowadays.

As we share less and less of our physical and spiritual goods, our souls start being more and more empty day after day. Many people, especially in the western best developed societies; the masses attitude towards sharing is most evidently hostile.
Another factor which probably decreased our natural human desire to share is technocracy and changing of communication from physical as it used to be until few dacades to digital today.

The huge shift of communication from physical to digital, changes the whole essence of basic life, hence I believe at least the distorted sharing should be encouraged on the Internet (file movies and programs sharing) should be considered normal and not illegal..
I believe Using Free Software instead of non-free (proprietary) one is another thing through which we can stimulate sharing. If we as society appreciate our freedom at all  and  care for our children future, it is my firm conviction, we should do best to keep sharing as much as we can in both physical and digital sense.

BB – A must see ASCII Art Audio / Video portable demo for Linux, FreeBSD, UNIX and DOS

Thursday, May 24th, 2012

bb Audio Visual ASCII art Linux FreeBSD demonstration old school demo logo

I know and I have enjoyed BB – Portable Demo for already a decade.
I'm sure many newbies to the Free And Open Source (FOSS) realm don't know or heard of bb's existence as nowdays ASCII art is not so well known among youngsters. Hence this short post aims to raise some awareness of the existence of this already OLD but GOLD – awesome! text console / terminal demonstation BB 🙂

bb is pretty much in the spirit of Oldschool Assembly DOS demo scene dominating the geeks dome in the late 80's and yearly 90's.

Historically bb used to be one of the main stunning things one could show to a fellow GNU / Linux new comer.

For the year 2000, seeing all this awesome ASCII video demo running on free Operating System like GNU / Linux was a big think.
The fact that such an advanced ASCII art was distributed freely for an OS which used to exist since only (6 / 7 years) was really outstanding of its time.

BB text ascii art Linux demo entry screen characters matrix

I still remember how much I was amazed seeing a plain ascii video stream was possible only Linux. Moreover the minimal requirements of bb were quite low for its time – it worked on mostly all PCs one can find at the time.

BB's minimum requirements to work with no chops is just an old 486/66 DX2 CPU Mhz with few megas of memory (32MB of memory was more than enough to run it)

BB text sacii art Linux demo entry screen char matrix

A very unique feature of bb was it was the first Linux demo that succesfully run simultaneously playing on two monitor screens as one can read on the project website.
Unfortunately I didn't owned two monitors back in the day so never ever had the opportunity to see it running on two screens.
Anyhow I've seen it runnign somewhere on some of the Linux install fests visited some years ago…

The demo was developed by 4 man group ppl – the AA group the same digital artists are also the guys behind the AA Project.

AA Lib mascot logo :)

The main aim of AA-lib was to make possible (Doom, Second Reality, X windows) to run rendered in plain ASCII art text.

The project succeeded in a lot of his goals already as there is already existent such an ascii art ports of large games like QUAKE! Be sure to check this awesome project too AAquake ascii quake page is here
, as well as video and pictures could be viewed under a plain console Linux tty or in terminal (via SSH 🙂 )

Thanks to AA-Lib even text mode doom exists.

bb as well as aa-lib has ports for most modern Linux distros in that number one can easily get rpm or deb packages for most of distros.
On Slackware Linux you should compile it from source. Though compilation should be a straightfoward process, not that i tried it myself but I remember a close friend of mine (a great Slackware devotee) who was the one to show me the demo for a first time on his Slackware box.

1. Installing bb on Debian Linux

Debian Linux users like me are privileged as for already many years a Debian package of bb is maintaned thanks to Uwe Herman

Hence for anyone willing to enjoy bb install it by running:

debian:~# apt-get --yes install bb
....
ho@debian:~$ bb

If you're running a X server the aa-lib will immediately run with its X server compiled support:

Running BB Music Screesnhot

2. Installing BB demo on FreeBSD

On FreeBSD, bb demo has a port to install it run:

freebsd# cd /usr/ports/misc/bb freebsd# make install clean ...

Here is good time to say that even though in most of the machines, I've tested the demo I had on some of the hosts problems with sound due to buggy sound drivers.
As of time of writting hopefully on most machines there will be no troubles as most of the Linux sb drivers are better supported by ALSA.

Everyone interested in both Free Software and ASCII art knows well how big in significance is the AA-lib project for the historical development and attraction for new hackers to the Linux dome.
In that sense AAlib head developer Jan HubickaBy the way Jan Hubicka is also the author of another Linux tool called xaos. Xaos is a tool to deal with some kind of advanced higher mathematics stuff called fractals.

XAOS Screenshot Debian Squeeze Linux

Unfortunately I don't know a bit for fractal maths and what the purpose of the tool is but as you can see on the shot it looks nice running 🙂

Here are also, lot of the major BB parts in shots:

Running bb music screen screenshot Linux Debian 6 Squeeze

BB AScii fire Linux shot

bb demo ascii art fractals

BB demo ascii art back head and description of the dev

bb demo ascii zebra Linux screenshot

bb demo cannon gun shot

BB demo ring screenshot

BB demo spots Debian shot

BB developer head shot 2

BB developer profile shot

bb game ascii invaders demo

Linux extremist BB demo

BB demo zoomed text ascii art text

BB Demo thanks for watching screen

For those on MS-Windows OS platform, here is the demo 🙂

BB ASCII Demo standard size running in Linux (With sound)

Enjoy ! 🙂

Why I never liked Mandrake Linux / Mankdrake Linux has took its name from an 1930s comics Mandrake the Magician

Wednesday, May 9th, 2012

I never liked Mandrake Linux, since day 1 I saw it.
Historically Mandrake Linux was one of the best Linux distributions available for free download in the "Linux scene" some 10 to 12 years ago.

Mandrake was simple gui oriented and trendy. It also one the Linux distribution with the most simplified installer program and generally a lot of GUI software for easy configuration and use by the end user.

Though it's outside nice look, still for me it was like an "intuition" that Mandrake is not so good as it appeared.

Now many years later I found by chance that Mandrake has been sued to change their Operating System name with another, due to a law suit requit by the copyright holders of Mandrake The Magician comics. "Mandrake the Magician" used to be a very popular before the Second World war in the 1930's.

Mandrake the Magician Comics Magazine from 1930's Cover, Mandrake the Black Magic Magician

It obviously not a co-incidence that the Mandrake names was after this comics and not the mandrake herb plants available in Europe, Africa and Asia. This is clear in Mandrake Linux distro earlier mascot, you see below:

Mandrake Linux old distribution logo, magician penguin

Later on they changed Mandrake's logo to loose the connection with Mandrake The Magician and used another new crafted logo:

Mandrake GNU Linux newer logo
Its quite stunning nowdays magician obsession, has so heavily infiltrated our lives that even something like a Free Softwre Linux distribution might have some kind of reference to magician and occult stuff (I saw this from the position of being Christian) …

Later due to the name copyright infringement Mandrake Linux was renamed first to Mandragora Linux.
Instead of putting some nice name non related to occultism or magic stuff the French commercian company behind Mandrake rename it to another non-Christian name Mandragora.
Interestingly the newer name Mandragora as one can read in wikipedia means:
 

Mandragora (demon), in occultism

Well apparently, someone from the head developers of this Linux distribution has a severe obsession with magic and occultism.

Later MandrakeSoft (The French Company behind Mandrake Linux) renamed finally the distribution to Mandriva under the influence of the merger of Mandrake with the Brazillian company Connectiva this put also an over to the legal dispute copyright infringement dispute with Hearst Corporation (owning the rights of Mandrake the Magician).

Having in mind all fact on current Mandriva "dark names history", I think it is better we Christians avoid it …

Editting binary files in console and GUI on FreeBSD and Linux

Thursday, April 26th, 2012

I’ve recently wanted to edit one binary file because there was compiled in the binary a text string with a word I didn’t liked and therefore I wanted to delete. I know I can dig in the source of the proggie with grep and directly substitute my “unwatned text” there but I wanted to experiment, and see what kind of hex binary text editors are for Free OSes.
All those who lived the DOS OS computer era should certainly remember the DOS hex editors was very enjoyable. It was not rare case, where in this good old days, one could simply use the hex editor to “hack” the game and add extra player lives or modify some vital game parameter like put himself first in the top scores list. I even remember some DOS programs and games was possible to be cracked with a text editor … Well it was times, now back to current situation as a Free Software user for the last 12 years it was interesting to see what is the DOS hexeditor like alternatives for FreeBSD and Linux and hence in this article I will present my findings:

A quick search in FreeBSD ports tree and Debian installable packages list, I’ve found a number of programs allowing one to edit in console and GUI binary files.

Here is a list of the hex editors I will in short review in this article:

  • hexedit
  • dhex
  • chexedit
  • hte
  • hexer
  • hexcurse
  • ghex
  • shed
  • okteta
  • bless
  • lfhex

1. hexedit on Linux and BSD – basic hex editor

I’ve used hexedit already on Linux so I’ve used it some long time ago.

My previou experience in using hexedit is not too pinky, I found it difficult to use on Redhat and Debian Linux back in the day. hexedit is definitely not a choice of people who are not “initiated” with hex editting.
Anyways if you want to give it a try you can install it on FreeBSD with:

freebsd# cd /usr/ports/editors/hexedit
freebsd# make install clean

On Debian the hexedit, install package is named the same so installation is with apt:

debian:~# apt-get –yes install hexedit

hexedit screenshot Debian Linux Squeeze

2. Hex editting with chexedit

I’ve installed chexedit the usual way from ports:

freebsd# cd /usr/ports/editors/chexedit
freebsd# make install clean

chexedit is using the ncurses text console library, so the interface is very similar to midnight commander (mc) as you see from below’s screenshot:

Chexeditor FreeBSD 7.2 OS Screenshot

Editting the binary compiled in string was an easy task with chexedit as most of the commands are clearly visible, anyways changing a certain text string contained within the binary file with some other is not easy with chexedit as you need to know the corresponding binary binary value representing each text string character.
I’m not a low level programmer, so I don’t know the binary values of each keyboard character and hence my competence came to the point where I can substitute the text string I wanted with some unreadable characters by simply filling all my text string with AA AA AA AA values…

chexedit on Debian is packaged under a deb ncurses-hexedit. Hence to install it on Deb run:

debian:~# apt-get –yes install ncurses-hexedit

Further on the binary to run chexedit on binary contained within ncurses-hexedit is:

debian:~# hexeeditor

3. Hex Editting on BSD and Linux with hte

Just after trying out chexedit, I’ve found about the existence of one even more sophisticated hexeditor console program available across both FreeBSD and Linux.
The program is called hte (sounds to me a bit like the Indian word for Elephant “Hatti” :))

hte is installable on Debian with cmd:

debian:~# apt-get install ht

On FreeBSD the port name is identical, so to install it I execed:

freebsd# cd /usr/ports/editors/hte
freebsd# make install clean

hte is started on Debian Linux (and presumably other Linux distros) with:

$ hte

On FreeBSD you need to run it with ht command:

freebsd# ht

You see how hte looks like in below screenshot:

ht has the look & feel like midnight commander and I found it easier to use than chexedit and hexeditor
4. hexer VI like interface for Linux

As I was looking through the available packages ready to install, I’ve tried hexer

debian:~# apt-get install –yes hexer

hexer does follow the same standard commands like VIM, e.g. i for insert, a for append etc.

Hexer Debian Linux vim like binary editor screenshot

It was interesting to find out hexer was written by a Bulgarian fellow Petar Penchev 🙂
(Proud to be Bulgarian)

http://people.freebsd.org/~roam/ – Petar Penchev has his own page on FreeBSD.org

As a vim user I really liked the idea, the only thing I didn’t liked is there is no easy way to just substitute a string within the binary with another string.

5. hexcurse another ncurses library based hex editor

On Deb install and run via:

debian:~# apt-get –yes install hexcurse
debian:~# hexcurse /usr/bin/mc

Hexcurse Debian Linux text binary editor screenshot

hexcurse is also available on FreeBSD to install it use cmd:

freebsd# cd /usr/ports/editors/hexcurse
freebsd# make install clean
….

To access the editor functions press CTRL+the first letter of the word in the bottom menu, CTRL+H, CTRL+S etc.
Something I disliked about it is the program search is always in hex, so I cannot look for a text string within the binaries with it.

6. ghex – Editting binary files in graphical environment

If you’re running a graphical environment, take a look at ghex. ghex is a gnome (graphical hex) editor.Installing ghex on Debian is with:

debian:~# apt-get –yes install ghex
….

To run ghex from terminal type:

debian:~# ghex2

GHex2 GNOME hex binary editor screenshot

To install ghex on FreeBSD (and I assume other BSDs), install via port:

freebsd# cd /usr/ports/editors/ghex
freebsd# make install clean

Gnome hex editor have plenty of tools, useful for developers to debug binary files.

Some nice tools one can find are under the the menus:

Windows -> Character Table

This will show a complete list of each keyboard sent character in ASCII, Hex, Decimal, Octal and Binary

Screenshot ghex Character table Debian Linux

Another useful embedded tool in ghex is:

Windows -> Type Convertion Dialog

Ghex type convertion dialog screenshot

Note that if you want to use the Type Convertion Dialog tool to find the representing binary values of a text string you will have to type in the letters one by one and save the output within a text file and later you can go and use the same editor to edit the text string within the binary file you like.

I’m not a programmer but surely for programmers or people who want to learn some binary counting, this 2 ghex edmebbed tools are surely valuable.

To conclude even though there are plenty of softwares for hex editting in Linux and BSD, none of them is not so easy to use as the old DOS hexdedit tool, maybe it will be a nice idea if someone actually rewrites the DOS tool and they package it for various free operating systems, I’m sure many people will find it helpful to have a 1:1 equivalent to the DOS tool.

7. Shed pico like interfaced hex editor

For people, who use pico / nano as a default text editor in Linux shed will probably be the editor of choice as it follows the command shortcuts of picoOn Deb based distros to install it run:

debian:~# apt-get install –yes shed

shed pico like hex binary editor Linux

Shed has no BSD port as of time of writting.8. Okteta a KDE GUI hex editor

For KDE users, I found a program called okteta. It is available for Deb based Linuxes as deb to install it:

debian:~# apt-get –yes install okteta

Screenshot Okteta Debian GNU / Linux Squeeze

As of time of writting this article there is no okteta port for BSDs.
Okteta has plenty of functions and even has more of a functions than ghexedit. Something distinctive for it is it supports opening multiple files in tabs.

9. lfhex a large file text editor

lfhex is said to be a large (binary) file text editor, I have not tested it myself but just run it to see how it looks like. I don’t have a need to edit large binary files too, but I guess there are people with such requirements too 🙂

lfhex - Linux The Large file hex editor

To install lfhex on Debian:

debian:~# apt-get install –yes lfhex

lfhex has also a FreeBSD port installable via:

freebsd# cd /usr/ports/editors/lfhex
freebsd# make install clean

10. Bless a GUI tool for editting large hex (binary) files

Here is the description directly taken from the BSD port /usr/ports/editors/bless

Bless is a binary (hex) editor, a program that enables you to edit files asa sequence of bytes. It is written in C# and uses the Gtk# bindings for theGTK+ toolkit.

To install and use ot on deb based Linuxes:

debian:~# apt-get install –yes bless
….

On BSD installation is again from port:

freebsd# cd /usr/ports/editors/bless
freebsd# make install clean
….

Something that makes bless, maybe more desirable choice for GUI users than ghex is its availability of tabs. Opening multiple binaries in tabs will be useful only to few heavy debuggers.

Bless GUI hex editor Debian Linux tabs opened screenshot

11. Ghextris – an ultra hard hacker tetris game 🙂

For absolute, hacker / (geeks), there is a tetris game called ghextris. The game is the hardest tetris game I ever played in my life. It requires more than regular IQ and a lot of practice if you want to become really good in this game.

To enjoy it:

debian:~# apt-get –yes install ghextris

Ultra hrad hardcore hackers game ghextris screenshot

Unfortunately there is no native port of ghextris for BSD (yet). Anyhow, it can be probably run using the Linux emulation or even compiled from source.
Well that’s all I found for hexedit-ing, I’ll be happy to hear if someone can give me some feedback on his favourite editor.

What is the Pirate Party / Interview with Peter Sunde – The co-founder of ThePirateBay on CeBIT 2011

Monday, February 20th, 2012

The Piratebay website logo

Many people are using ThePiratebay to get all kind of data and share Movies, Games, Programs etc.)

ThePirateBay has become like a standard media for file sharing and still not many people are aware of who is behind ThePirateBay.
This is why I thought, by posting this article I can "rant" on the existence of the Swedish Pirate Party
The Pirate Party – Is a party fighting on behalf of information sharing freedom (or in other words a party fighting for our freedom as individuals in the digital era).

Swedish Pirate Party Official Party mascot
Pirate Party (Sweden) party official mascot

The Pirate party are swedish political party established in 2006, its members are activists lobbying on freedom of information as well as pointing at threats to our human freedoms imposed by new legislations and anti-freedom agreements like the recent ACTA treaty.

Since there creation in 2006, they have grown from a local Swedish party to an international party, locally existing in 33 countries arount the world.The ideology behind the Swedish PirateParty as well as the rest of existing Pirate Parties international is as I prior said Freedom of Information on the internet. PP is a party phenomenon, as they are neither a left or a right oriented party (on the traditional left-right scale). PP are independent bloc pursuing their political agenda colabborating with all mainstream parties, which stand behind the ideologies the party fights for.

After last European Parliament elections, PP received 7% of all Swedish votes for a country EU representative.
As of today according to Wikipedia , the party holds 2 EU deputee seats in EU parliament.

I've a friend who is taking membership in the Pirate Party and from him I know the PP party are big supporters of Free Software.

One major key figure part of Pirate Party is the swedish Peter Sunde. He is a front person who is behind ThePirateBay project as well as holds a very interesting revolutionary ideas. Many of his ideas are close to "the father of all free software" – Richard Mathew Stallman (RMS). Therefore the PP would be probably of interest to anyone who is interested in Free Software. Here is an interview with Peter Sunde's taken in Cebit 2011:

As long as I know the Pirate Party are along the only few organizations in the whole world, who are trying to fight on behalf of human freedom.I'll be glad to hear if someone know about other ornigazations apart from Pirate Party the Free Software Foundation – (FSF) lobbying on present worsening conditions of invidual human freedom.

Here is few curious facts proving how BIG and important thepiratebay.org is, the text below is again quote from Wikipedia

Currently ThePirateBay.org is ranked as the 80th most visited website in the world and 20th in Sweden by Alexa Internet, has over 5.5 million registered users and, as of February 2012, hosts more than 4 million torrent files.

According to the Los Angeles Times, The Pirate Bay is "one of the world's largest facilitators of illegal downloading" and "the most visible member of a burgeoning international anti-copyright or pro-piracy movement"

If you want to further check on what is today the Pirate Party standing for be sure to check ThePirateParty.Com – PP’s official website .

What is the real development costs of Debian GNU / Linux – How much costs the development of a Free Software projects

Friday, February 17th, 2012

Free Software (FS) is free as in freedom as well as free as in price. Free and Open Source Software (FOSS) is developed by geek hobbyist which voluntarily put their time and effort in writting, testing and sharing with anyone for free million of lines of programming code. This doesn't mean however the price of free software costs is 0 (zero). Though the "end product" –  Free Software developed is FREE, "real" software costs as with any other product costs huge money.

I've recently read on Jeb's blog an estimation on how much is the cost of one of the major Free Software project efforts – Debian GNU / Linux
According to James E. Brombergerthe whole Debian project was estimated to be at the shocking price of $19 billion – $19 000, 000, 000 !!!

Here is how JEB got the $19 billions, a quote taken from his blog:

"By using David A Wheeler’s sloccount tool and average wage of a developer of US$72,533 (using median estimates from Salary.com and PayScale.com for 2011) I summed the individual results to find a total of 419,776,604 source lines of code for the ‘pristine’ upstream sources, in 31 programming languages — including 429 lines of Cobol and 1933 lines of Modula3!

In my analysis the projected cost of producing Debian Wheezy in February 2012 is US$19,070,177,727 (AU$17.7B, EUR€14.4B, GBP£12.11B), making each package’s upstream source code worth an average of US$1,112,547.56 (AU$837K) to produce. Impressively, this is all free (of cost).

James has done incredible job with this great research and he deserves applause.
However I believe the numbers proposed by his research are slightly different if we speak about realistic cost of Debian GNU / Linux.
The real costs of the working software ready to install on a user PC are way higher, as according to Jeb's research only the software cost based on code line count is considered.

Hence James software estimation calculates only the programming costs and miss many, many factors that constitute the software end cost.
Some of the many, many REAL COST / expenses for developing a huge Free Software project like Debian GNU / Linux to be considered are:
 

a) bandwidth costs for hosting free software (on the server side)b) bandwidth cost for developers or FS users downloading the software

a) Time spend to spread the word of the great added value of Debian and bundled software (Mouth by Mouth Marketing)

b) Time spend to advertise Debian and its free software components on blogs, social networks (identi.ca, facebook, twitter) etc.(Voluntary online Marketing, SEO etc.)

c) Time spend on generating ideas on future program versions and reporting them to Debian FS community

d) Time on evaluation and feedback on software

e) Time spend on managing free software repository (download) servers voluntarily (by system administrators)

f) Time spend by users on Bug Tracking & Bug Reporting

g) Time spend on research and self-actualization by software developer)

h) Time spend on software Quality Assurance

This are most of the multiple factors which should probably influence the cost of any non-free (proprietary software) project. No matter this costs apply for non-free software, it perfectly applies for free software as well.With all said if if we assume the non-programming costs are equal to the programming costs of $ 19 000 000 000 (suggested by Jeb). This means the real cost of Debian will presumably be at least $32 000 000 000. Putting $ 19 billion for all this long list of "additional" costs (besides pure source) factors is probably still very under-scored number.
 

  • the developers use of their own computers (hardware depreciation)
  • electricity bill of the volunteer (developer) working on the program or project
  • electricity bills for servers where free software is stored and available for download
  • volunteer developer IT skills and tech knowledge (KNOW HOW)
  • Internet, network, dial up bandwidth cost
  • personal time put in FS development (programming, design, creativity etc.)! here the sub costs are long:
  • Costs for Project Management Leaders / Project Coordination
  • The complexity of each of the projects constituting Debian

Very interesting figure from Jeb's research is the Programming Languages break down by source code figure.
Jamesresearch reveals on the 4 major programming languages used in the 17000+ software projects (part of Debian GNU / Linux):

 

  • ANSI C with 168,536,758 – (40% of all projects source code)
  • C++ at 83,187,329 – (20% of all projects source)
  • Java 34,698,990 – (lines of code 8% of sources)
  • Lisp – (7% of all projects source code)

  His research also provides a general idea on how much the source code of some of the major FOSS projects costs. Here is a copy of his figures
 

Individual Projects

Other highlights by project included:

Project Version Thousands
of SLOC
Projected cost
at US$72,533/developer/year
Samba 3.6.1 2,000 US$101 (AU$93M)
Apache 2.2.9 693 US$33.5M (AU$31M)
MySQL 5.5.17 1,200 US$64.2M (AU$59.7M)
Perl 5.14.2 669 US$32.3M (AU$30M)
PHP 5.3.9 693 US$33.5M (AU$31.1M)
Bind 9.7.3 319 US$14.8M (AU$13.8M)
Moodle 1.9.9 396 US$18.6M (AU$17.3M)
Dasher 4.11 109 US$4.8M (AU$4.4M)
DVSwitch 0.8.3.6 6 US$250K (AU$232K)

 

As you can imagine all the source evaluation results, are highly biased and are open for discussion, since evaluating a free software project/s is a hard not to say impossible task. The "open" model of development makes a project very hard to track, open source model implies too many unexpected variables missing from the equation for clear calculation on costs. What is sure however if turned in money it is very expensive to produce.  At present moment Debian Project is sponsored only through donations. The usual yearly budget 5 years ago for Debian  was only $80 000 dollars a year!! You can check Debian Project annual reports throughout the years here , for year 2012 Debian Project budget is as low as $ 222, 677 (US Dollars)! The output price of the software the project provides is enormous high if compared to the low project expenses!

For us the free software users, price is not a concern, Debian is absolutely free both  as in freedom and free as in beer 😉
 

How to install OpenNTPD NTP server to synchronize system clock on FreeBSD for better security

Sunday, February 12th, 2012

FreeBSD, OpenBSD, NetBSD and Linux ntpd alternative server to synchronize server system time

Lately I've been researching on ntpd and wrote a two articles on how to install ntpd on CentOS, Fedora and how to install ntpd on FreeBSD and during my research on ntpd, I've come across OpenNTPD and decided to give it a go on my FreeBSD home router.
OpenBSD project is well known for it is high security standards and historically has passed the test of time for being a extraordinary secure UNIX like free operating system.
OpenBSD is developed in parallel with FreeBSD, however the development model of the two free operating systems are way different.

As a part of the OpenBSD to be independant in its basis of software from other free operating systems like GNU / Linux and FreeBSD. They develop the all around free software realm known OpenSSH. Along with OpenSSH, one interesting project developed for the main purpose of OpenBSD is OpenNTPD.

Here is how openntpd.org describes OpenNTPD:

"a FREE, easy to use implementation of the Network Time Protocol. It provides the ability to sync the local clock to remote NTP servers and can act as NTP server itself, redistributing the local clock."

OpenNTPD's accent just like OpenBSD's accent is security and hence for FreeBSD installs which targets security openntpd might be a good choice. Besides that the so popular classical ntpd has been well known for being historically "insecure", remote exploits for it has been released already at numerous times.

Another reason for someone to choose run openntpd instead of ntpd is its great simplicity. openntpd configuration is super simple.

Here are the steps I followed to have openntpd time server synchronize clock on my system using other public accessible openntpd servers on the internet.

1. Install openntpd through pkg_add -vr openntpd or via ports tree

a) For binar install with pkg_add issue:

freebsd# pkg_add -vr openntpd
...

b) if you prefer to compile it from source

freebsd# cd /usr/ports/net/openntpd
freebsd# make install clean
...

2. Enable OpenNTPD to start on system boot:

freebsd# echo 'openntpd_enable="YES"' >> /etc/rc.conf

3. Create openntpd ntpd.conf configuration file

There is a default sample ntpd.conf configuration which can be straight use as a conf basis:

freebsd# cp -rpf /usr/local/share/examples/openntpd/ntpd.conf /usr/local/etc/ntpd.conf

Default ntpd.conf works just fine without any modifications, if however there is a requirement the openntpd server to listen and accept time synchronization requests from only certain hosts add to conf something like:

listen on 192.168.1.2
listen on 192.168.1.3
listen on 2607:f0d0:3001:0009:0000:0000:0000:0001
listen on 127.0.0.1

This configuration will enable only 192.168.1.2 and 192.168.1.3 IPv4 addresses as well as the IPv6 2607:f0d0:3001:0009:0000:0000:0000:0001 IP to communicate with openntpd.

4. Start OpenNTPD service

freebsd# /usr/local/etc/rc.d/openntpd

5. Verify if openntpd is up and running

freebsd# ps axuww|grep -i ntp
root 31695 0.0 0.1 3188 1060 ?? Ss 11:26PM 0:00.00 ntpd: [priv] (ntpd)
_ntp 31696 0.0 0.1 3188 1140 ?? S 11:26PM 0:00.00 ntpd: ntp engine (ntpd)
_ntp 31697 0.0 0.1 3188 1088 ?? S 11:26PM 0:00.00 ntpd: dns engine (ntpd)
root 31700 0.0 0.1 3336 1192 p2 S+ 11:26PM 0:00.00 grep -i ntp

Its also good idea to check if openntpd has succesfully established connection with its peer remote openntpd time servers. This is necessery to make sure pf / ipfw firewall rules are not preventing connection to remote 123 UDP port:

freebsd# sockstat -4 -p 123
USER COMMAND PID FD PROTO LOCAL ADDRESS FOREIGN ADDRESS
_ntp ntpd 31696 4 udp4 83.228.93.76:54555 212.70.148.15:123
_ntp ntpd 31696 6 udp4 83.228.93.76:56666 195.69.120.36:123
_ntp ntpd 31696 8 udp4 83.228.93.76:49976 217.75.140.188:123

By default openntpd is also listening to IPv6 if IPv6 support is enabled in freebsd kernel.

6. Resolve openntpd firewall filtering issues

If there is a pf firewall blocking UDP requests to in/out port 123 within /etc/pf.conf rule like:

block in log on $EXT_NIC proto udp all

Before the blocking rule you will have to add pf rules:

# Ipv4 Open outgoing port TCP 123 (NTP)
pass out on $EXT_NIC proto tcp to any port ntp
# Ipv6 Open outgoing port TCP 123 (NTP)
pass out on $EXT_NIC inet6 proto tcp to any port ntp
# Ipv4 Open outgoing port UDP 123 (NTP)
pass out on $EXT_NIC proto udp to any port ntp
# Ipv6 Open outgoing port UDP 123 (NTP)
pass out on $EXT_NIC inet6 proto udp to any port ntp

where $EXT_NIC is defined to be equal to the external lan NIC interface, for example:
EXT_NIC="ml0"

Afterwards to load the new pf.conf rules firewall has to be flushed and reloaded:

freebsd# /sbin/pfctl -f /etc/pf.conf -d
...
freebsd# /sbin/pfctl -f /etc/pf.conf -e
...

In conclusion openntpd should be more secure than regular ntpd and in many cases is probably a better choice.
Anyhow bear in mind on FreeBSD openntpd is not part of the freebsd world and therefore security updates will not be issued directly by the freebsd dev team, but you will have to regularly update with the latest version provided from the bsd ports to make sure openntpd is 100% secure.

For anyone looking for more precise system clock synchronization and not so focused on security ntpd might be still a better choice. The OpenNTPD's official page states it is designed to reach reasonable time accuracy, but is not after the last microseconds.
 

TalkonAut Free Program to use Jabber protocol on Apple IPad (IBad)

Friday, February 3rd, 2012

 

TalkonAut Free Mobile Phones and Apple devices java program logo

I've installed Jabber as a platform for internal company communication for a company. It was a requirement for this jabber server to be accessed from a different type of computers / devices different in size, hardware and OS e.g. (Mac OS X, Windows, Linux, Ipad, mobile phones etc.)

Happily there is plenty of free software programs which allows access to jabber for free. On the major operating systems GNU / Linux, Mac OS X and Windows the client jabber accounts are working fine with the wonderful free software Pidgin The Universal Chat Client

One of the jabber clients, however was primary had to be used on Apple's proprietary IPad / IBad as Richard Stallman likes to call it 😉

The person who had to have the Jabber protocol working on IPad program was not a proficient user and therefore I had the task to find a program to be able to talk to the Jabber protocol for him.
A quick search in Google for jabber ipad client led me to few programs said to support Jabber on Ipad :
 

 

  • BeejiveIM for IPad
  • Jabba

Jabba was looking quite, nice but unfortunately costs $1.99 and should be purchased in Apple's App Store and it was preferrable not to spend money on a trivial thing as a Jabber client.
I gave BeejiveIM a try but it required some very complex registration, as well as again required to be purchased (if I remember correctly), so it wasn't an option either.

Then thanksfully, I found TalknOut which is free and it is a perfect jabber client for Apple Ipad
Talkonaut is a program also supporting both Jabber (XMPP) and GTalk2VoIP, hence supporting Google Talk and MSN/Live Messanger.
It is written Java and therefore works on any device that has Java installed. Talkonaut supports the following mobile architectures:
 

  • Apple's IPhone and Ipad
  • Google Android Phones
  • Many of the Nokia Symbian's S60 3rd and 5th edition "smart" phones
  • Windows Mobile 5.x and 6.x
  • Java J2ME based phones

Installing TalkonAut is a piece of cake from Ipod's Safari you click on the Install link and it gets installed. I will not get into details on how it is configured as this also is pretty easy.
Here is how it looks like on Ipad after configured and the user is logged in Jabber:
TalkonAut Jabber XMPP Mobile client Apple Ipad Screenshot

Something really unique and nice for the program is the way the chat dialogs gets ordered, the idea to place one person's sending on the left side and the replying one on right is innovative and something I've not seen in another chat client 😉

TalkonAut Sender and replier message on different side screenshotTalkonaut should also support VoIP (voice conversations), between mobile users, I'm curious if somebody used the program for VoIP and can share feedback?