Posts Tagged ‘function’

How to Avoid the 7 Most Frequent Mistakes in Python Programming

Monday, September 9th, 2019

python-programming-language-logo

Python is very appealing for Rapid Application Development for many reasons, including high-level built in data structures, dynamic typing and binding, or to use as glue to connect different components. It’s simple and easy to learn but new Python developers can fall in the trap of missing certain subtleties.

Here are 7 common mistakes that are harder to catch but that even more experienced Python developers have fallen for.

 

1. The misuse of expressions as function argument defaults

Python allows developers to indicate optional function arguments by giving them default values. In most cases, this is a great feature of Python, but it can create some confusion when the default value is mutable. In fact, the common mistake is thinking that the optional argument is set to whatever default value you’ve set every time the function argument is presented without a value. It can seem a bit complicated, but the answer is that the default value for this function argument is only evaluated at the time you’ve defined the function, one time only.  

how-to-avoid-the-7-most-frequent-mistakes-in-python-programming-3

 

2. Incorrect use of class variables

Python handles class variables internally as dictionaries and they will follow the Method Resolution Order (MRO). If an attribute is not found in one class it will be looked up in base classes so references to one part of the code are actually references to another part, and that can be quite difficult to handle well in Python. For class attributes, I recommend reading up on this aspect of Python independently to be able to handle them.

how-to-avoid-the-7-most-frequent-mistakes-in-python-programming-2

 

3. Incorrect specifications of parameters for exception blocks

There is a common problem in Python when except statements are provided but they don’t take a list of the exceptions specified. The syntax except Exception is used to bind these exception blocks to optional parameters so that there can be further inspections. What happens, however, is that certain exceptions are then not being caught by the except statement, but the exception becomes bound to parameters. The way to get block exceptions in one except statement has to be done by specifying the first parameter as a tuple to contain all the exceptions that you want to catch.

how-to-avoid-the-7-most-frequent-mistakes-in-python-programming-1
 

4. Failure to understand the scope rules

The scope resolution on Python is built on the LEGB rule as it’s commonly known, which means Local, Enclosing, Global, Built-in. Although at first glance this seems simple, there are some subtleties about the way it actually works in Python, which creates a more complex Python problem. If you make an assignment to a variable in a scope, Python will assume that variable is local to the scope and will shadow a variable that’s similarly named in other scopes. This is a particular problem especially when using lists.

 

5. Modifying lists during iterations over it

 

When a developer deletes an item from a list or array while iterating, they stumble upon a well known Python problem that’s easy to fall into. To address this, Python has incorporated many programming paradigms which can really simplify and streamline code when they’re used properly. Simple code is less likely to fall into the trap of deleting a list item while iterating over it. You can also use list comprehensions to avoid this problem.

how-to-avoid-the-7-most-frequent-mistakes-in-python-programming-8

       

6. Name clash with Python standard library

 

Python has so many library modules which is a bonus of the language, but the problem is that you can inadvertently have a name clash between your module and a module in the standard library. The problem here is that you can accidentally import another library which will import the wrong version. To avoid this, it’s important to be aware of the names in the standard library modules and stay away from using them.

how-to-avoid-the-7-most-frequent-mistakes-in-python-programming-5

 

7. Problems with binding variables in closures


Python has a late binding behavior which looks up the values of variables in closure only when the inner function is called. To address this, you may have to take advantage of default arguments to create anonymous functions that will give you the desired behavior – it’s either elegant or a hack depending on how you look at it, but it’s important to know.

 

 

how-to-avoid-the-7-most-frequent-mistakes-in-python-programming-6

Python is very powerful and flexible and it’s a great language for developers, but it’s important to be familiar with the nuances of it to optimize it and avoid these errors.

Ellie Coverdale, a technical writer at Essay roo and UK Writings, is involved in tech research and projects to find new advances and share her insights. She shares what she has learned with her readers on the Boom Essays blog.

Disable Windows hibernate on a work notebook or Desktop Gamers PC – Save a lot of Space on Windows C Drive, delete hidefil.sys howto

Thursday, May 18th, 2017

how-to-to-disable-stop-hibernate-windows-8-10-to-save-disk-space-and-get-rid-of-hbierfil.sys-misteriously-occupying-space-improve-windows-performance

Some Windows  laptop / desktop users prefer not to shutdown computers (especially those coming back from Mac OS backgound) at the end of the day but  hibernate instead.

Hibernate is a great thing but historically we know well that in Windows hibernate is working much worser than on Macs and it is common that after multiple hibernates you will face problems with missing  C: drive space is it might be "misteriously" decreasing in a way that the PC performance degrades as the C:hibfile.sys hidden file occupies few 16Gigas or so (the occupied space by hibfile.sys does resemble the installed RAM Memory on the computer, so if your PC has 16Gigas the hibfile.sys will be lets say approximately 15 Gigabytes)

However most users never use hibernate and might never use it for a life time, especially those on a Desktop Windows PCs, I use Windows as a WorkStation as an employee of DXC (the ex Hewlett Packard or Hewlett Packard Enterprise that merged with CSC) but to be honest I've used hibernate function very raraly on the notebook, thus I find the hibernate more or less useless feature, especially because at many times I try to wake-up the PC after hibernate the computer boots but the display stays dark and I have to restart the Computer before I can go back to normal work operations. Of course my Windows 7 hibernation issues might be caused do to the corporate software installed on my PC or because the fact the hard drive is encrypted but nomatter that in my case and I guess in case of many the hibernate function on Windows 7 / 8 / 10 might be totally useless.
 


Few works is Hiberfil.sys File and Why you might want to complete disable / delete it


On Windows 7 / 8 / 10 the hiberfil.sys file is being used to store the PC current state at time of hibernation, so if you have to move from a place to place within an organization / university / office without a charger hibernation is a really nice way to save battery power without later wasting time for additional PC boot (where a lot of power is wasted for Operationg System to load and re-opening the opened Browser etc.

So in short sleeping the PC with Hibernate function does cause the Computer to write into C:hiberfil.sys all data at the moment stored in the PC RAM (Memory), which is being cleared up at time of Computer being in Sleep mode.
Once the computer receives a Wake-up call from the hibernation in order to present with the Desktop at the same state hiberfile.sys stored information is being red and transferred to PC flushable RAM so the RAM memory is again filled with same bits it used to have right before the hibernation was made.

Because hiberfil.sys is a system file it has the hidden attribute and it can only be write / read by a Administrator Win account and usually it is not a good idea to touch it

Some people haven't shutdown Windows for 20-30 days and especially if Windows has disabled updates it happens for some users to use the hibernate function for weeks (re-hibernating and waking up thousand times) for long periods so the effect is the hiberfile.sys might become gigantic and if you take the time to check what is file or directory is wasting all your C:> drive with leys say WinDirStat or SpaceSniffer you will notice the lets say 15Gigas being eaten by Hiberfil.sys.

Disable of hibfile.sys is also a great tip for Gamers desktop PCs as most gamers won't use hibernate function at all.

I. How to Disable Hibernate Mode in Windows 10, 8, 7, or Vista


In order to get rid of the file across Windows 7 / 8 / 10

Open command prompt (as an Administartor, right click on the Command Prompt cmd.exe and choose Run as Administartor) and issue below cmd:

disable-hibernate-on-windows-7-8-10-powercfg-off-screenshot

C:> powercfg -h off

If later you decide you need the hibernate function again active on the PC or notebook do issue:

C:> powercfg -h on

You’re likely reading this because you noticed a gigantic hiberfil.sys file sitting on your system drive and you’re wondering if you can get rid of it to free up some space. Here’s what that file is and how you can delete it if you want to.

 

II. Disable Hibernate Mode in Windows XP

Hibernate function command is not present on Windows XP so in order to remove it on XP (hope you don't use XP any more and you're not a viction of the resent crypt catastrophic ransomware WannaCry 🙂

disable-hibernate-mode-windows-xp-screenshot

Control Panel -> Power Options

In the Power Options properties window, switch to the “Hibernate” tab and disable the “Enable hibernation” option.

After you disable hibernate mode, restart PC, and manually delete the hiberfil.sys file.

Now enjoy free-ing up few gigabytes of useless wasted C: hard drive space from your PC 🙂

Note: Removing hiberfil.sys is a precious thing to do on old Windows Computers which have been made with a little leys say 40Gigabyte partition drive C: whether with the time due to User profile use and Browsing caches the C: drive has left with leys say 1-2 Gigabyte of free space and the computers overall performance has fallen twice or so.

This post is in memoriam of Chriss Cornell (our generation used to grow with grunge and his music was one of the often listened by me and our generation)

R.I.P: Chriss Cornell (the head of SoundGarden and AudioSlave who passed away yesterday right on the day when we in Bulgarian Eastern Orthodox Church commemorate the memory of a great-martyr Nicolay Sofijski (Great Martyr Nicolas from Sofia martyred by Turkish Ottomans during year 1555).

I found surprising fact for me  that Chriss Cornell converted to Greek Eastern Orthodox faith under influence of his Greek Wife, below is paste from his Wikipedia page:

"

Chriss Cornell Personal life (Rest in Peace Chris)

Cornell was married to Susan Silver, the manager of Alice in Chains and Soundgarden.[123] They had a daughter, Lillian Jean, born in June 2000.[123] He and Silver divorced in 2004.[123] In December 2008, Cornell reported via his official website that he had finally won back his collection of 15 guitars after a four-year court battle with Silver.[124]

He was married to Vicky Karayiannis,[125] a Paris-based American publicist of Greek heritage. The union produced a daughter, Toni, born in September 2004, and a son, Christopher Nicholas, born in December 2005.[126] Cornell converted to the Greek Orthodox Church through her influence.[127]

When asked how Cornell beat all his addictions he stated, "It was a long period of coming to the realization that this way (sober) is better. Going through rehab, honestly, did help … it got me away from just the daily drudgery of depression and either trying to not drink or do drugs or doing them and you know, they give you such a simple message that any idiot can get and it's just over and over, but the bottom line is really, and this is the part that is scary for everyone, the individual kinda has to want it … not kinda, you have to want it and to not do that crap anymore or you will never stop and it will just kill you."[128]

In a 2011 interview,[129] Cornell said the major change with the reformed Soundgarden is a lack of alcohol: "The biggest difference I noticed … and we haven't even really talked about it: There are no bottles of Jack Daniel's around or beers. And we never talked about … it's just not there."


Enjoy!

 

Turn your Windows PC / notebook to Wireless Router with My Wifi Router 3.0 and TP-Link TL-WN722N

Wednesday, January 7th, 2015

my-wifi-router-3.0-turn-regular-windows-notebook-to-wifi-router

I've been to home of my wife's parents and for this Christmas, they have received second hand Acer Aspire notebook as a Christmas gift. So far they were using internet using their Windows XP PC which is getting internet here in Belarus via UTP network cable using ByFly ZTE ADSL router modem. As ADSL modem lacked wifh WI-FI Antenna (support) and there was already the Acer notebook which had to access the internet preferrably via Wireless connection, the option was to get a WI-FI router and connect it to the ADSL modem but as this would cost (20 EUR at minimum) and there was alreay Wireless (Receiver) adapter TP-LINK TL-WN722N unused, I decided to try make the TP-LINK Receiver and Windows XP PC to act as a small Home Made Wireless  (software) router.

Until I succeeded I've tried multiple softwares which all failed to turn the Windows PC to Wi-Fi Hotspot.
Here is list of few of the softwares I tried that didn't worked for some reason:

1. Wifi HotSpot Creator

WiFi-HotSpot-Creator-screenshot

Turn your PC into a Wi-Fi Router for Free!

Instantly share your internet connection with your friends and peers over Wi-Fi. Turns your PC into a Wi-Fi Router! And its Free! Wifi HotSpot Creator is said to be able to convert any Mac OS X and Windows PC to hotspot here is Wifi HotSpot Creator download website
Wifi HotSpot Creator is said to work with Windows Vista / XP / 7 / 8, however as I said it doesn't work for me on Windows XP.
 

2. Virtual Wifi router

Here is

With Virtual WiFi Router you can create a WiFi hotspot for WiFi Reverse Tethering on Windows 7 and for wifi supported mobiles and other wifi enabled computer to create a network and to share internet. Virtual Wifi Router in a minute converts your PC into a WiFi hot spot for free.

To make the program working it depends on .NET 4.5. Though the program looked like a superb it unfortunately was crashing on Windows XP. Below are few screenshots from program working on Windows 7.

virtual-wifi-router-screenshot

virtual-wifi-router-connected-device-screenshot-windows
3. My Wifi Router 3.0

Finally I've come across My Wifi Router 3.0 which is just another program that makes necessery Windows configuration to TP-Link TL-WN722N Wireless receiver Adapter to turn it into a homemade Wi-Fi router.
my-wifi-router-on-windows-xp-desktop-pc-noteboko-creenshot

By default Amiss_papp Wifi is configured, this can be changed, however in my case when I tried changing it there were some issues, so finally I've had to re-install My Wifi Router to make it working again.
Once configured My Wifi Router there is the green button (Activate / Deactivate Free Wifi) as seen from screenshot.

As you can see My Wifi Router also allows to Share Videos over WiFi. Once I've tested the program and confirmed it as working, I had to configure it to automatically start on Windows PC boot.

This is done from Settings (located on backward triangle button, next to minimize function).
I had to set check in to  Auto Start and Software Conflict Detection.

make-windows-pc-with-wireless-wifi-router-my-wifi-router-settings-screenshot

Once connected to the TP-Link TL-WN722N (USB) Wi-Fi (High Gain) Receiver adapter in Windows Tray a new indicator will popup that a device has been connected. I've tested My Wifi Router and it seem to be working fine with 3 remote connected Wi-Fi devices (1 Notebook and 2 Nokia Lumia mobile phones). The speed of internet was fast and if I didn't know the connection is done in a software way via such an improvised Windows XP Wi-Fi network router I would think it is just a regular Wi-Fi network router.

One more thing I had to do to make the internet working I had to share the the LAN Network (ethernet card) Interface's internet from
 

Control Panel -> Networking -> Local Area Connection (Properties) -> Advanced

https://www.pc-freak.net/images/windows-xp-internet-connection-sharing-how-to-networking-screenshot-make-network-address-translation-windows-xp.jpg

A downside of My Wifi Router is I couldn't find a way to save password while connection to the newly created WiFi router with it, so each time I had to login I had to manually type in the password (default my wifi router password is 123567890). Re-typing password on each login is annoying but if you have to do it once per day in the morning when you turn on your notebook it is not such a big deal.

Once connected to My Wifi Router in Connection Management in Friends (tab) you will see a list with connected devices.

my-wifi-router-on-windows-xp-with-htc-and-ipad-iphone-connected
As visible from above screenshot default IPs which will be assigned to new connected clients to My Wifi Router will be in local network IP range 192.168.23.2 – 192.168.23.254.

Now all left is to Enjoy  your new Software Wi-Fi router 🙂

PHP system(); hide command output – How to hide displayed output with exec();

Saturday, April 7th, 2012

I've recently wanted to use PHP's embedded system(""); – external command execute function in order to use ls + wc to calculate the number of files stored in a directory. I know many would argue, this is not a good practice and from a performance view point it is absolutely bad idea. However as I was lazy to code ti in PHP, I used the below line of code to do the task:

<?
echo "Hello, ";
$line_count = system("ls -1 /dir/|wc -l");
echo "File count in /dir is $line_count \n";
?>

This example worked fine for me to calculate the number of files in my /dir, but unfortunately the execution output was also visialized in the browser. It seems this is some kind of default behaviour in both libphp and php cli. I didn't liked the behaviour so I checked online for a solution to prevent the system(); from printing its output.

What I found as a recommendations on many pages is instead of system(); to prevent command execution output one should use exec();.
Therefore I used instead of my above code:

<?
echo "Hello, ";
$line_count = exec("ls -1 /dir/|wc -l");
echo "File count in /dir is $line_count \n";
?>

By the way insetad of using exec();, it is also possible to just use ` (backtick) – in same way like in bash scripting's .

Hence the above code can be also written for short like this:

<?
echo "Hello, ";
$line_count = `ls -1 /dir/|wc -l`;
echo "File count in /dir is $line_count \n";
?>

🙂

How to increase brightness on Fujitsu Siemens Amilo PI22515 notebook with Slackware Linux

Friday, March 9th, 2012

Increase LCD screen brightness on Fujitsu Siemens Amilo laptop with Linux Slackware

A friend of mine has Fujitsu Siemens Amilo laptop and is full time using his computer with Slackware Linux.

He is quite happy with Slackware Linux 13.37 on the laptop, but unfortunately sometimes his screen brightness lowers. One example when the screen gets darkened is when he switch the computer on without being plugged in the electricity grid. This lowered brightness makes the screen un-user friendly and is quite tiring for the eye …

By default the laptop has the usual function keys and in theory pressing Function (fn) + F8 / F7 – should increase / decrease the brightness with no problems, however on Slackware Linux (and probably on other Linuxes too?), the function keys are not properly recognized and not responding whilst pressed.
I used to have brigtness issues on my Lenovo notebook too and remember how irritating this was.
After a bit of recalling memories on how I solved this brightness issues I remembered the screen brigthness on Linux is tunable through /proc virtual (memory) filesystem.

The laptop (Amilo) Fujitsu Siemens video card is:

lspci |grep -i vga
00:02.0 VGA compatible controller: Intel Corporation Mobile GM965/GL960 Integrated Graphics Controller (primary) (rev 03)

I took a quick look in /proc and found few files called brightness:
 

  • /proc/acpi/video/GFX0/DD01/brightness
  • /proc/acpi/video/GFX0/DD02/brightness
  • /proc/acpi/video/GFX0/DD03/brightness
  • /proc/acpi/video/GFX0/DD04/brightness
  • /proc/acpi/video/GFX0/DD05/brightness

cat-ting /proc/acpi/video/GFX0/DD01/brightness, /proc/acpi/video/GFX0/DD03/brightness, /proc/acpi/video/GFX0/DD04/brightness all shows not supported and therefore, they cannot be used to modify brightness:

bash-4.1# for i in $(/proc/acpi/video/GFX0/DD0{1,3,4,5}/brightness); do \
cat $i;
done
<not supported>
<not supported>
<not supported>
<not supported>

After a bit of testing I finally succeeded in increasing the brightness.
Increasing the brightness on the notebook Intel GM965 video card model is done, through file:

/proc/acpi/video/GFX0/DD02/brightness

To see all the brightness levels the Fujitsu LCD display supports:

bash-4.1# cat /proc/acpi/video/GFX0/DD02/brightness
levels: 13 25 38 50 63 75 88 100
current: 25

As you can see the dark screen was caused cause the current: brightness is set to a low value of 25.
To light up the LCD screen and make the screen display fine again, I increased the brightness to the maximum level 100, e.g.:

bash-4.1# echo '100' > /proc/acpi/video/GFX0/DD02/brigthness

Just for the fun, I've written also a two lines script which gradually increases LCDs brightness 🙂

bash-4.1# echo '13' > /proc/acpi/video/GFX0/DD02/brightness;
bash-4.1# for i in \
$(cat /proc/acpi/video/GFX0/DD02/brightness|grep 'levels'|sed -e 's#levels:##g'); do \
echo $i > /proc/acpi/video/GFX0/DD02/brightness; sleep 1; \done

fujitsu_siemens_brightness_fun.sh script is fun to observe in changing the LCD screen gradually in one second intervals 🙂

Here is also a tiny program that reduces and increases the notebook laptop brightness written in C. My friend Dido, coded it in just few minutes just for the fun 🙂
To permanently solve the issues with darkened screen on boot time it is a good idea to include echo '100' > /proc/acpi/video/GFX0/DD02/brigthness in /etc/rc.local:

bash-4.1# echo '100' > /proc/acpi/video/GFX0/DD02/brigthness

I've also written another Universal Linux Increase laptop screen brightness Shell script which should be presumable also working for all Laptop models running Linux 🙂

My maximize_all_linux_laptops_brightness.sh "universal increase Linux brightness" script is here
I'll be glad to hear from people who had tested the script on other laptops and can confirm it works fine for them.
 

How to convert SVG to PNG graphic formats (using GUI and console) on GNU / Linux and FreeBSD

Wednesday, December 7th, 2011

SVG to PNG Convert on GNU / Linux FreeBSD using command and GUI

I’ve playing trying to learn InkScapeThe Open Source vector graphic editor .By so far I’m quite impressed on how easy this program is learned and how easy graphical manipulation with this nifty program can be done.
The default format in which InkScape saves its files is SVG (Scalable Vectors Graphics). For all those unfamiliar with SVG – SVG is an open (free format) format developed in 1999 which insetad of containing binary data like PNG or JPEG does contain plain XML content. SVG being consisted of plain XML has multiple advantageous, the most important one makes it easy for text and visual data to be displayed among different program svg readers in absolutely identical way. Besides that the format if read with plain text editor like vim or emacs can be altered directly via the source.
Being multi system interoperable makes SVG as a great format for text and visual data storage in HTML5, actually SVG is already a part of the HTML5 html coding standard. And most probably its adoption rate will raise up drastically as soon as HTML5 starts substitute HTML4 and lower web standards.

Anyways I’m slipping away from the aim of this post so I’ll stop blabbering on how great SVG is and let people check it out for themselves (if not already).

Going back to the aim of my article to show How to convert SVG to PNG graphical extension on GNU / Linux and FreeBSD

After producing a bunch of files with InkScape I realized the default format in which Inkscape stores its files is SVG , this was okay with me but since I wanted to have my experimental produced content in PNG I needed a way to convert SVG to
My first logical guess was that The Gimp will be able to handle the situation and after opening my SVG file with GIMP and used the gimp File -> Save As option and give the SVGfile an extension of PNG , Gimp succesfully converted the file to PNG.

However I wanted to dig further and check out what is the standard accepted way to convert SVG files using a plain command. This will possibly be handy to me if I had to do something online (let’s say a website) which will accept SVG and will require the SVG files to be converted and also stored in PNG or other Graphic file formats.

After checking online, I’ve found a post which pointed me to librsvg2 which contains RSVG(Turn SVG files into raster images.)

librsvg is available as a package in most mainstream Linux distributions nowdays, Fedora, Debian etc., as well as contains a port inside the FreeBSD ports system. Since I’m using Debian on my notebook where I installed and tested the command line SVG to PNG convertion the way I did it is:

noah:/home/hipo/Desktop# apt-get --yes install librsvg2-bin
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
librsvg2-bin
0 upgraded, 1 newly installed, 0 to remove and 16 not upgraded.
Need to get 72.5 kB of archives.
After this operation, 180 kB of additional disk space will be used.
Get:1 http://ftp.nl.debian.org/debian/ squeeze/main librsvg2-bin amd64 2.26.3-1 [72.5 kB]
Fetched 72.5 kB in 0s (184 kB/s)
Selecting previously deselected package librsvg2-bin.
(Reading database ... 376046 files and directories currently installed.)
Unpacking librsvg2-bin (from .../librsvg2-bin_2.26.3-1_amd64.deb) ...
Processing triggers for man-db ...
Setting up librsvg2-bin (2.26.3-1) ...

Afterwards the exact convertion of my Inscape SVG file drawing.svg to drawing.png using rsvg I’ve done like so:

hipo@noah:~/Desktop$ rsvg drawing.svg drawing.png

The convertion results for me was 100% uniqueness between the file converted and the output PNG. Some people might wonder why I didn’t used Inkscape’s Export to Bitmap function and then use convert command part of ImageMagick in order to convert the produced Inkscape bitmap to PNG.

 

One other thing worthy to mention is on  Debian,  librsvg2-bin contains 2 more executable besides rsvg. One is the rsvg-view command which allows one to view SVG files using command line or Graphic enviroment, the other one is rsvg-convert which supports again SVG convertion to PDF and to PNG

Before proceeding with the other described ways to convert SVG to PNG earlier in this article, I give a try to Inkscape’s Export to Bitmap embedded function but the produced bitmap did not resembled the original SVG file so I decided to completely abandon this method
Maybe there is some particular reason of the chaotic way I’ve tested Inkscape to place random images sometimes going out of the field of a paper etc. which influenced the improper generation of Bitmap using Inkscape, despite that it seems InkScape needs some more development until the bugs in Bitmap producing get fixed and it can be freely used to produce Bitmaps.

Maybe there is some particular reason for the failure of Inkscape to produce a good BMP file, like for example the chaotic way I’ve tested Inkscape to place random images sometimes going out of the field of a paper borders etc.This should have influenced the improper generation of Bitmap using Inkscape, anyhow it seems InkScape needs some more development until the bugs in Bitmap creation get fixed.

By the way if you’re wondering how to convert PNG to bitmap BMP after, once having converted SVG to PNG this is easily doable with convert command, like so:

hipo@noah:~/Desktop$ convert drawing.png drawing.bmp

Maybe in future releases it will be a good idea if InkScape developers integrate a convertion to other formats this will make it handy and make surely these nice program more popular among users. Hope this is helpful. Cheers and as RMS likes to say Happy Hacking 😉

Fix of “Unable to allocate memory for pool.” PHP error messages

Saturday, October 15th, 2011

Since some time, I don’t know exactly where, after some updates of my WordPress running on a small server with FreeBSD 7.2. I’ve started getting a lot of Apache crashes. Often the wordpress scripts stopped working completely and I got only empty pages when trying to process the wordpress blog in a browser.

After a bunch of reading online, I’ve figured out that the cause might be PHP APC stands for Alternative PHP Cache .

I was not sure if the PHP running on the server had an APC configured at all so I used a phpinfo(); script to figure out if I had it loaded. I saw the APC among the loaded to show off in the list of loaded php modules, so this further led me to the idea the APC could be really causing the unexpected troubles.

Thus first I decided to disable the APC on a Virtualhost level for the domain where the crashing wordpress was hosted, to do I placed in the VirtualHost section in the Apache configuration /usr/local/etc/apache2/httpd.conf the following config directive:

php_flag apc.cache_by_default Off

These get me rid of the multiple errors:

PHP Warning: require_once() [function.require-once]: Unable to allocate memory for pool. in /usr/local/www/data-dist/blog/wp-content/plugins/tweet-old-post/top-admin.php on line 6

which constantly were re-occuring in php_error.log:

Further after evaluating all the websites hosted on the server and making sure none of which was really depending on APC , I’ve disabled the APC completely for PHP. To do so I issued:

echo 'apc.enabled = 0' >> /usr/local/etc/php.ini

Similarly on GNU/Linux to disable globally APC from PHP only the correct location to php.ini should be provided on Debian this is /etc/php5/apache2/php.ini .

Filter messages in Qmail with unwanted words, get rid of the Viagra annoying spam with Qtrap

Sunday, September 4th, 2011

Drop qmail received mail containing banned / unwanted words to get rid of Viagra and Sex related spam

Recently the annoying Viagra spam has emerged again. Therefore I decided to clean up some of the mails received to one of the qmail servers to protect users emailbox from this viagra peril.

To do so I’ve remember about an old script which used to be part of qmailrocks.org qmail install, the script is called qtrap and is able to filter emails based on list of specific mail contained words.
Since qmailrocks.org is gone (down) for some time and its still available only on few mirrored locations which by the way are not too easy to find I decided to write a little post on how qtrap.sh could be integrated quick & easy with any Qmail + Vpopmail install out there.

Hereby I include the description for qtrap.sh given by the script author:

“qtrap.sh script is applied on a per domain basis and serves as a “bad word” scanner to catch any spam that Spamassassin may have missed. This filter serves as the last defense against SPAM before it arrived in your inbox. I like this filter because it helps to get rid of any SPAM that happens to make it by Spamassassin. Without any protection at all, my mailbox gets a shit ton of SPAM every day. Within the first 3 months I enacted the Qtrap filter, Qtrap logged over 9,000 deleted SPAM messages, none of which were legitimate e-mails. My keyboard’s delete key was very appreciated the extra rest.

Any emails that are scanned and contain a banned word will be automatically deleted and logged by the qtrap script. A whitelist feature now exists so that individual addresses or domains can be exempt from the qtrap scan.”

Now as one might have general idea on what the script does. Here is the step by step qtrap.sh integration;

1. Create necessery qtrap directory and logs and set proper permissions

If the vpopmail is installed in /home/vpopmail , issue the following commands.

debian:~# cd /home/vpopmail
debian:~# mkdir -p qtrap/logs
debian:/home/vpopmail/qtrap# cd qtrap
debian:/home/vpopmail/qtrap# wget https://www.pc-freak.net/files/qtrap.sh
...
debian:/home/vpopmail/qtrap# cd ~
debian:~# touch /home/vpopmail/qtrap/logs/qtrap.logdebian:~# chown -R vpopmail:vchkpw /home/vpopmail/qtrapdebian:~# chmod -R 755 /home/vpopmail/qtrap

On older qmail installations it could be vpopmail is installed in /var/vpopmail if that’s the case, link /var/vpopmail to /home/vpopmail and go back to step 1. To link:

debian:~# ln -sf /var/vpopmail/ /home/vpopmail

2. Edit qtrap.sh to whitelist email addresses and build a ban words list

a) Include the email addresses mail arriving from which would not be checked by qtrap.sh

Inside qtrap.sh in line 63, there is a shell function whitelist_check(), the function looks like so:

whitelist_check () {
case $WHITELIST in
address@somewhere.com|address@somewhereelse.com)
echo $SENDER found in whitelist on `date "+%D %H:%M:%S"` >> /home/vpopmail/qtrap/logs/qtrap.log
exit 0;;
*)
;;
esac
}

By default the script has just two sample mails which gets whitelisted this is the line reading:

address@somewhere.com|address@somewhereelse.com

The whitelisted emails should be separated with a pipe, thus to add two more sample emails to get whitelisted by script the line should be changed like:

address@somewhere.com|address@somewhereelse.com|hipod@mymailserver.com|hipo@gmail.com

In order to whitelist an entire domain let’s say yahoo.com add a line to the above code like:

address@somewhere.com|address@somewhereelse.com|hipod@mymailserver.com|hipo@gmail.com|*yahoo.com

b) Defining the bad words ban list, mails containing them should not be delivery by qmail

The function that does check for the ban word list inside the script is checkall();, below is a paste from the script function:

checkall () {
case $BANNED_WORDS in
porn|PORN|Sex|SEX)
printout $BANNED_WORDS
echo MESSAGE DROPPED from $SENDER because of $BANNED_WORDS on `date "+%D %H:%M:%S"` >> /home/vpopmail/qtrap/logs/qtrap.log
exit 99;;
*)
;;
esac
}

checkall(); is located on line 74 in qtrap.sh, the exact list of banned words which the script should look for is located on line 76, the default qtrap.sh filters only mails containing just 4 words, e.g.:

porn|PORN|Sex|SEX)

To add the Viagra and VIAGRA common spam words to the list, modify it and expand like so:

porn|PORN|Sex|SEX|viagra|Viagra)

The delimiter is again | , so proceed further and add any unwanted spam words that are not common for any legit mails.

3. Install qtrap.sh to process all emails delivered to vpopmail

If its necessery to install the dropping of mails based on word filtering only to a single vpopmail virtualdomain do it with cmd:

debian:~# cd /home/vpopmail/domains/yourdomain.com
debian:/home/vpopmail/domains/yourdomain.com# touch .qmail-default.new
debian:/home/vpopmail/domains/yourdomain.com# echo '| /home/vpopmail/qtrap/qtrap.sh' >> .qmail-default.new
debian:/home/vpopmail/domains/yourdomain.com# echo "| /home/vpopmail/bin/vdelivermail '' bounce-no-mailbox" >> .qmail-default.new
debian:/home/vpopmail/domains/yourdomain.com# chown vpopmail:vchkpw .qmail-default.new
debian:/home/vpopmail/domains/yourdomain.com# cp -rpf .qmail-default .qmail-default.bak; mv .qmail-default.new .qmail-default
If however qtrap.sh needs to get installed for all existing vpopmail virtualdomains on the qmail server, issue a one liner bash script:

debian:~# cd /home/vpopmail/domains
debian:/var/vpopmail/domains# for i in *; do cd $i; echo "| /home/vpopmail/qtrap/qtrap.sh" >> $i/.qmail-default.new;
echo "| /home/vpopmail/bin/vdelivermail '' bounce-no-mailbox" >> $i/.qmail-default.new;
chown vpopmail:vchkpw .qmail-default.new; mv .qmail-default .qmail-default.old; mv .qmail-default.new .qmail-default; cd ..; done

This for loop will add ‘| /home/vpopmail/qtrap/qtrap.sh’ to all .qmail-default for all vpopmail domains.

Afterwards the .qmail-default file should contain the following two lines:

| /home/vpopmail/qtrap/qtrap.sh
| /home/vpopmail/bin/vdelivermail '' delete

A very important thing here you should consider that adding some common words, as let’s say hello or mail etc. could easily drop almost all the emails the qmail hands in to vpopmail.

Caution!! Never ever implement common words in the list of words !!
Always make sure the banned words added to qtrap.sh are words that are never enter an everyday legit email.

Another thing to keep in mind is that qtrap.sh doesn’t make a copy of the received message ,though it can easily be modified to complete this task.
Any mail that matches the banned words list will be dropped and lost forever.

4. Check if qtrap.sh is working

To check, if qtrap is working send mail to some mailbox located on the qmail server containing inside subject or mail message body the unwanted word defined inside qtrap.sh.

The mail should not be received in the mailbox to which its sent, if qtrap is working moreover qtrap.sh should log it inside it’s log file:

debian:~# cat /home/vpopmail/qtrap/logs/qtrap.log
MESSAGE DROPPED from hipo@mytestmail.com because of viagra on 09/03/11 11:34:19
MESSAGE DROPPED from support@mymailserver.com because of Viagra on 09/03/11 11:39:29

If the qtrap.log contains records similar to the one above, and the mail matching the banned word is not delivered, qtrap.sh is properly configured. If any issues check in qmail logs, they should have a good pointer on what went wrong with qtrap.sh invokation.

Note that I’ve integrated qtrap.sh to custom qmail install running on Debian Lenny 5.0 GNU/Linux.
If I have time I’ll soon test if its working fine on the latest stable Debian Squeeze and will report here in comments.
If however someone is willing to test if the script works on Debian Squeeze 6.0 or have tested it already please drop a comment to report if it works fine.
qtrap.sh, is a bit oldish and is not written to work too optimal therefore on some heavy loaded mail servers it can create some extra load and a bit delay the mail delivery. Thus when implementang one needs to consider the downsides of putting it in.

Also I was thinking tt might be nice if the script is rewritten to read the ban words and whitelist mails from files instead of as it is now as the words are hard coded in the script.
If I have some free time, I’ll probably do this, though I’m not sure if this is a too good idea as this might have a negative performance impact on the script execution time, as each instance of the script invoked should do one more operation of reading a file storing the ban words.

Well that’s pretty much it, enjoy 😉

How to compile latest qmailadmin (qmailadmin 1.2.15) on Debian Squeeze Linux

Thursday, August 11th, 2011

I’ve completed a qmail installation few days ago on a fresh installed Debian Squeeze 64 bit server. All is configured and works fine, except qmailadmin and vqadmin.
As the mail server was missing any kind of web mail administration panel, I needed to make at least one of the two above to make with qmail.

I decided to concentrate on qmailadmin and took the time to make it work. I used the following command lines and got the compile failure during make compilation:

debian:/usr/local/src/qmailadmin-1.2.15# ./configure --enable-cgibindir=/usr/lib/cgi-bin --enable-htmldir=/var/www/qmailadmin/ --enable-modify-quota
...
debian:/usr/local/src/qmailadmin-1.2.15# make
...

The source make failed with the following error:

In file included from template.c:45:
qmailadmin.h:37:1: warning: "MAX_FILE_NAME" redefined
In file included from template.c:28:
/home/vpopmail/include/vpopmail.h:146:1: warning: this is the location of the previous definition
template.c: In function "send_template_now":
template.c:505: error: "VERSION" undeclared (first use in this function)
template.c:505: error: (Each undeclared identifier is reported only once
template.c:505: error: for each function it appears in.)
make[1]: *** [template.o] Error 1
make[1]: Leaving directory `/usr/local/src/qmailadmin-1.2.15'
make: *** [all] Error 2

To workaround these compile issues, I’ve had to modify the C source file belonging to qmailadmin ( template.c ), e.g.:

debian:/usr/local/src/qmailadmin-1.2.15# vim template.c

In the file I had to add besides the line:

#include "util.h"

The code:

#define VERSION ""

Aterwards qmailadmin’s compile and install via make && make install-strip succeeded and now works perfectly fine 😉