Posts Tagged ‘distributions’

How to test RAM Memory for errors in Linux / UNIX OS servers. Find broken memory RAM banks

Friday, December 3rd, 2021

test-ram-memory-for-errors-linux-unix-find-broken-memory-logo

 

1. Testing the memory with motherboard integrated tools
 

Memory testing has been integral part of Computers for the last 50 years. In the dawn of computers those older perhaps remember memory testing was part of the computer initialization boot. And this memory testing was delaying the boot with some seconds and the user could see the memory numbers being counted up to the amount of memory. With the increased memory modern computers started to have and the annoyance to wait for a memory check program to check the computer hardware memory on modern computers this check has been mitigated or completely removed on some hardware.
Thus under some circumstances sysadmins or advanced computer users might need to check the memory, especially if there is some suspicion for memory damages or if for example a home PC starts crashing with Blue screens of Death on Windows without reason or simply the PC or some old arcane Linux / UNIX servers gets restarted every now and then for now apparent reason. When such circumstances occur it is an idea to start debugging the hardware issue with a simple memory check.

There are multiple ways to test installed memory banks on a server laptop or local home PC both integrated and using external programs.
On servers that is usually easily done from ILO or IPMI or IDRAC access (usually web) interface of the vendor, on laptops and home usage from BIOS or UEFI (Unified Extensible Firmware Interface) acces interface on system boot that is possible as well.

memtest-hp
HP BIOS Setup

An old but gold TIP, more younger people might not know is the

 

Prolonged SHIFT key press which once held with the user instructs the machine to initiate a memory test before the computer starts reading what is written in the boot loader.

So before anything else from below article it might be a good idea to just try HOLD SHIFT for 15-20 seconds after a complete Shut and ON from the POWER button.

If this test does not triggered or it is triggered and you end up with some corrupted memory but you're not sure which exact Memory bank is really crashing and want to know more on what memory Bank and segments are breaking up you might want to do a more thorough testing. In below article I'll try to explain shortly how this can be done.


2. Test the memory using a boot USB Flash Drive / DVD / CD 
 

Say hello to memtest86+. It is a Linux GRUB boot loader bootable utility that tests physical memory by writing various patterns to it and reading them back. Since memtest86+ runs directly off the hardware it does not require any operating system support for execution. Perhaps it is important to mention that memtest86 (is PassMark memtest86)and memtest86+ (An Advanced Memory diagnostic tool) are different tools, the first is freeware and second one is FOSS software.

To use it all you'll need is some version of Linux. If you don't already have some burned in somewhere at your closet, you might want to burn one.
For Linux / Mac users this is as downloading a Linux distribution ISO file and burning it with

# dd if=/path/to/iso of=/dev/sdbX bs=80M status=progress


Windows users can burn a Live USB with whatever Linux distro or download and burn the latest versionof memtest86+ from https://www.memtest.org/  on Windows Desktop with some proggie like lets say UnetBootIn.
 

2.1. Run memtest86+ on Ubuntu

Many Linux distributions such as Ubuntu 20.0 comes together with memtest86+, which can be easily invoked from GRUB / GRUB2 Kernel boot loader.
Ubuntu has a separate menu pointer for a Memtest.

ubuntu-grub-2-04-boot-loader-memtest86-menu-screenshot

Other distributions RPM based distributions such as CentOS, Fedora Linux, Redhat things differ.

2.2. memtest86+ on Fedora


Fedora used to have the memtest86+ menu at the GRUB boot selection prompt, but for some reason removed it and in newest Fedora releases as of time such as Fedora 35 memtest86+ is preinstalled and available but not visible, to start on  already and to start a memtest memory test tool:

  •   Boot a Fedora installation or Rescue CD / USB. At the prompt, type "memtest86".

boot: memtest86

2.3 memtest86+ on RHEL Linux

The memtest86+tool is available as an RPM package from Red Hat Network (RHN) as well as a boot option from the Red Hat Enterprise Linux rescue disk.
And nowadays Red Hat Enterprise Linux ships by default with the tool.

Prior redhat (now legacy) releases such as on RHEL 5.0 it has to be installed and configure it with below 3 commands.

[root@rhel ~]# yum install memtest86+
[root@rhel ~]# memtest-setup
[root@rhel ~]# grub2-mkconfig -o /boot/grub2/grub.cfg


    Again as with CentOS to boot memtest86+ from the rescue disk, you will need to boot your system from CD 1 of the Red Hat Enterprise Linux installation media, and type the following at the boot prompt (before the Linux kernel is started):

boot: memtest86

memtestx86-8gigabytes-of-memory-boot-screenshot
memtest86+ testing 5 memory slots

As you see all on above screenshot the Memory banks are listed as Slots. There are a number of Tests to be completed until
it can be said for sure memory does not have any faulty cells. 
The

Pass: 0
Errors: 0 

Indicates no errors, so in the end if memtest86 does not find anything this values should stay at zero.
memtest86+ is also usable to detecting issues with temperature of CPU. Just recently I've tested a PC thinking that some memory has defects but it turned out the issue on the Computer was at the CPU's temperature which was topping up at 80 – 82 Celsius.

If you're unfortunate and happen to get some corrupted memory segments you will get some red fields with the memory addresses found to have corrupted on Read / Write test operations:

memtest86-returning-memory-address-errors-screenshot


2.4. Install and use memtest and memtest86+ on Debian / Mint Linux

You can install either memtest86+ or just for the fun put both of them and play around with both of them as they have a .deb package provided out of debian non-free /etc/apt/sources.list repositories.


root@jeremiah:/home/hipo# apt-cache show memtest86 memtest86+
Package: memtest86
Version: 4.3.7-3
Installed-Size: 302
Maintainer: Yann Dirson <dirson@debian.org>
Architecture: amd64
Depends: debconf (>= 0.5) | debconf-2.0
Recommends: memtest86+
Suggests: hwtools, memtester, kernel-patch-badram, grub2 (>= 1.96+20090523-1) | grub (>= 0.95+cvs20040624), mtools
Description-en: thorough real-mode memory tester
 Memtest86 scans your RAM for errors.
 .
 This tester runs independently of any OS – it is run at computer
 boot-up, so that it can test *all* of your memory.  You may want to
 look at `memtester', which allows testing your memory within Linux,
 but this one won't be able to test your whole RAM.
 .
 It can output a list of bad RAM regions usable by the BadRAM kernel
 patch, so that you can still use you old RAM with one or two bad bits.
 .
 This is the last DFSG-compliant version of this software, upstream
 has opted for a proprietary development model starting with 5.0.  You
 may want to consider using memtest86+, which has been forked from an
 earlier version of memtest86, and provides a different set of
 features.  It is available in the memtest86+ package.
 .
 A convenience script is also provided to make a grub-legacy-based
 floppy or image.

Description-md5: 0ad381a54d59a7d7f012972f613d7759
Homepage: http://www.memtest86.com/
Section: misc
Priority: optional
Filename: pool/main/m/memtest86/memtest86_4.3.7-3_amd64.deb
Size: 45470
MD5sum: 8dd2a4c52910498d711fbf6b5753bca9
SHA256: 09178eca21f8fd562806ccaa759d0261a2d3bb23190aaebc8cd99071d431aeb6

Package: memtest86+
Version: 5.01-3
Installed-Size: 2391
Maintainer: Yann Dirson <dirson@debian.org>
Architecture: amd64
Depends: debconf (>= 0.5) | debconf-2.0
Suggests: hwtools, memtester, kernel-patch-badram, memtest86, grub-pc | grub-legacy, mtools
Description-en: thorough real-mode memory tester
 Memtest86+ scans your RAM for errors.
 .
 This tester runs independently of any OS – it is run at computer
 boot-up, so that it can test *all* of your memory.  You may want to
 look at `memtester', which allows to test your memory within Linux,
 but this one won't be able to test your whole RAM.
 .
 It can output a list of bad RAM regions usable by the BadRAM kernel
 patch, so that you can still use your old RAM with one or two bad bits.
 .
 Memtest86+ is based on memtest86 3.0, and adds support for recent
 hardware, as well as a number of general-purpose improvements,
 including many patches to memtest86 available from various sources.
 .
 Both memtest86 and memtest86+ are being worked on in parallel.
Description-md5: aa685f84801773ef97fdaba8eb26436a
Homepage: http://www.memtest.org/

Tag: admin::benchmarking, admin::boot, hardware::storage:floppy,
 interface::text-mode, role::program, scope::utility, use::checking
Section: misc
Priority: optional
Filename: pool/main/m/memtest86+/memtest86+_5.01-3_amd64.deb
Size: 75142
MD5sum: 4f06523532ddfca0222ba6c55a80c433
SHA256: ad42816e0b17e882713cc6f699b988e73e580e38876cebe975891f5904828005
 

 

root@jeremiah:/home/hipo# apt-get install –yes memtest86+

root@jeremiah:/home/hipo# apt-get install –yes memtest86

Reading package lists… Done
Building dependency tree       
Reading state information… Done
Suggested packages:
  hwtools kernel-patch-badram grub2 | grub
The following NEW packages will be installed:
  memtest86
0 upgraded, 1 newly installed, 0 to remove and 21 not upgraded.
Need to get 45.5 kB of archives.
After this operation, 309 kB of additional disk space will be used.
Get:1 http://ftp.de.debian.org/debian buster/main amd64 memtest86 amd64 4.3.7-3 [45.5 kB]
Fetched 45.5 kB in 0s (181 kB/s)     
Preconfiguring packages …
Selecting previously unselected package memtest86.
(Reading database … 519985 files and directories currently installed.)
Preparing to unpack …/memtest86_4.3.7-3_amd64.deb …
Unpacking memtest86 (4.3.7-3) …
Setting up memtest86 (4.3.7-3) …
Generating grub configuration file …
Found background image: saint-John-of-Rila-grub.jpg
Found linux image: /boot/vmlinuz-4.19.0-18-amd64
Found initrd image: /boot/initrd.img-4.19.0-18-amd64
Found linux image: /boot/vmlinuz-4.19.0-17-amd64
Found initrd image: /boot/initrd.img-4.19.0-17-amd64
Found linux image: /boot/vmlinuz-4.19.0-8-amd64
Found initrd image: /boot/initrd.img-4.19.0-8-amd64
Found linux image: /boot/vmlinuz-4.19.0-6-amd64
Found initrd image: /boot/initrd.img-4.19.0-6-amd64
Found linux image: /boot/vmlinuz-4.19.0-5-amd64
Found initrd image: /boot/initrd.img-4.19.0-5-amd64
Found linux image: /boot/vmlinuz-4.9.0-8-amd64
Found initrd image: /boot/initrd.img-4.9.0-8-amd64
Found memtest86 image: /boot/memtest86.bin
Found memtest86+ image: /boot/memtest86+.bin
Found memtest86+ multiboot image: /boot/memtest86+_multiboot.bin
File descriptor 3 (pipe:[66049]) leaked on lvs invocation. Parent PID 22581: /bin/sh
done
Processing triggers for man-db (2.8.5-2) …

 

After this both memory testers memtest86+ and memtest86 will appear next to the option of booting a different version kernels and the Advanced recovery kernels, that you usually get in the GRUB boot prompt.

2.5. Use memtest embedded tool on any Linux by adding a kernel variable

Edit-Grub-Parameters-add-memtest-4-to-kernel-boot

2.4.1. Reboot your computer

# reboot

2.4.2. At the GRUB boot screen (with UEFI, press Esc).

2.4.3 For 4 passes add temporarily the memtest=4 kernel parameter.
 

memtest=        [KNL,X86,ARM,PPC,RISCV] Enable memtest
                Format: <integer>
                default : 0 <disable>
                Specifies the number of memtest passes to be
                performed. Each pass selects another test
                pattern from a given set of patterns. Memtest
                fills the memory with this pattern, validates
                memory contents and reserves bad memory
                regions that are detected.


3. Install and use memtester Linux tool
 

At some condition, memory is the one of the suspcious part, or you just want have a quick test. memtester  is an effective userspace tester for stress-testing the memory subsystem.  It is very effective at finding intermittent and non-deterministic faults.

The advantage of memtester "live system check tool is", you can check your system for errors while it's still running. No need for a restart, just run that application, the downside is that some segments of memory cannot be thoroughfully tested as you already have much preloaded data in it to have the Operating Sytstem running, thus always when possible try to stick to rule to test the memory using memtest86+  from OS Boot Loader, after a clean Machine restart in order to clean up whole memory heap.

Anyhow for a general memory test on a Critical Legacy Server  (if you lets say don't have access to Remote Console Board, or don't trust the ILO / IPMI Hardware reported integrity statistics), running memtester from already booted is still a good idea.


3.1. Install memtester on any Linux distribution from source

wget http://pyropus.ca/software/memtester/old-versions/memtester-4.2.2.tar.gz
# tar zxvf memtester-4.2.2.tar.gz
# cd memtester-4.2.2
# make && make install

3.2 Install on RPM based distros

 

On Fedora memtester is available from repositories however on many other RPM based distros it is not so you have to install it from source.

[root@fedora ]# yum install -y memtester

 

3.3. Install memtester on Deb based Linux distributions from source
 

To install it on Debian / Ubuntu / Mint etc. , open a terminal and type:
 

root@linux:/ #  apt install –yes memtester

The general run syntax is:

memtester [-p PHYSADDR] [ITERATIONS]


You can hence use it like so:

hipo@linux:/ $ sudo memtester 1024 5

This should allocate 1024MB of memory, and repeat the test 5 times. The more repeats you run the better, but as a memtester run places a great overall load on the system you either don't increment the runs too much or at least run it with  lowered process importance e.g. by nicing the PID:

hipo@linux:/ $ nice -n 15 sudo memtester 1024 5

 

  • If you have more RAM like 4GB or 8GB, it is upto you how much memory you want to allocate for testing.
  • As your operating system, current running process might take some amount of RAM, Please check available Free RAM and assign that too memtester.
  • If you are using a 32 Bit System, you cant test more than 4 GB even though you have more RAM( 32 bit systems doesnt support more than 3.5 GB RAM as you all know).
  • If your system is very busy and you still assigned higher than available amount of RAM, then the test might get your system into a deadlock, leads to system to halt, be aware of this.
  • Run the memtester as root user, so that memtester process can malloc the memory, once its gets hold on that memory it will try to apply lock. if specified memory is not available, it will try to reduce required RAM automatically and try to lock it with mlock.
  • if you run it as a regular user, it cant auto reduce the required amount of RAM, so it cant lock it, so it tries to get hold on that specified memory and starts exhausting all system resources.


If you have 8 Gigas of RAM plugged into the PC motherboard you have to multiple 1024*8 this is easily done with bc (An arbitrary precision calculator language) tool:

root@linux:/ # bc -l
bc 1.07.1
Copyright 1991-1994, 1997, 1998, 2000, 2004, 2006, 2008, 2012-2017 Free Software Foundation, Inc.
This is free software with ABSOLUTELY NO WARRANTY.
For details type `warranty'. 
8*1024
8192


 for example you should run:

root@linux:/ # memtester 8192 5

memtester version 4.3.0 (64-bit)
Copyright (C) 2001-2012 Charles Cazabon.
Licensed under the GNU General Public License version 2 (only).

pagesize is 4096
pagesizemask is 0xfffffffffffff000
want 8192MB (2083520512 bytes)
got  8192MB (2083520512 bytes), trying mlock …Loop 1/1:
  Stuck Address       : ok        
  Random Value        : ok
  Compare XOR         : ok
  Compare SUB         : ok
  Compare MUL         : ok
  Compare DIV         : ok
  Compare OR          : ok
  Compare AND         : ok
  Sequential Increment: ok
  Solid Bits          : ok        
  Block Sequential    : ok        
  Checkerboard        : ok        
  Bit Spread          : ok        
  Bit Flip            : ok        
  Walking Ones        : ok        
  Walking Zeroes      : ok        
  8-bit Writes        : ok
  16-bit Writes       : ok

Done.

 

4. Shell Script to test server memory for corruptions
 

If for some reason the machine you want to run a memory test doesn't have connection to the external network such as the internet and therefore you cannot configure a package repository server and install memtester, the other approach is to use a simple memory test script such as memtestlinux.sh
 

#!/bin/bash
# Downloaded from https://www.srv24x7.com/memtest-linux/
echo "ByteOnSite Memory Test"
cpus=`cat /proc/cpuinfo | grep processor | wc -l`
if [ $cpus -lt 6 ]; then
threads=2
else
threads=$(($cpus / 2))
fi
echo "Detected $cpus CPUs, using $threads threads.."
memory=`free | grep 'Mem:' | awk {'print $2'}`
memoryper=$(($memory / $threads))
echo "Detected ${memory}K of RAM ($memoryper per thread).."
freespace=`df -B1024 . | tail -n1 | awk {'print $4'}`
if [ $freespace -le $memory ]; then
echo You do not have enough free space on the current partition. Minimum: $memory bytes
exit 1
fi
echo "Clearing RAM Cache.."
sync; echo 3 > /proc/sys/vm/drop_cachesfile
echo > dump.memtest.img
echo "Writing to dump file (dump.memtest.img).."
for i in `seq 1 $threads`;
do
# 1044 is used in place of 1024 to ensure full RAM usage (2% over allocation)
dd if=/dev/urandom bs=$memoryper count=1044 >> dump.memtest.img 2>/dev/null &
pids[$i]=$!
echo $i
done
for pid in "${pids[@]}"
do
wait $pid
done

echo "Reading and analyzing dump file…"
echo "Pass 1.."
md51=`md5sum dump.memtest.img | awk {'print $1'}`
echo "Pass 2.."
md52=`md5sum dump.memtest.img | awk {'print $1'}`
echo "Pass 3.."
md53=`md5sum dump.memtest.img | awk {'print $1'}`
if [ “$md51” != “$md52” ]; then
fail=1
elif [ “$md51” != “$md53” ]; then
fail=1
elif [ “$md52” != “$md53” ]; then
fail=1
else
fail=0
fi
if [ $fail -eq 0 ]; then
echo "Memory test PASSED."
else
echo "Memory test FAILED. Bad memory detected."
fi
rm -f dump.memtest.img
exit $fail

Nota Bene !: Again consider the restults might not always be 100% trustable if possible restart the server and test with memtest86+

Consider also its important to make sure prior to script run,  you''ll have enough disk space to produce the dump.memtest.img file – file is created as a test bed for the memory tests and if not scaled properly you might end up with a full ( / ) root directory!

 

4.1 Other memory test script with dd and md5sum checksum

I found this solution on the well known sysadmin site nixCraft cyberciti.biz, I think it makes sense and quicker.

First find out memory site using free command.
 

# free
             total       used       free     shared    buffers     cached
Mem:      32867436   32574160     293276          0      16652   31194340
-/+ buffers/cache:    1363168   31504268
Swap:            0          0          0


It shows that this server has 32GB memory,
 

# dd if=/dev/urandom bs=32867436 count=1050 of=/home/memtest


free reports by k and use 1050 is to make sure file memtest is bigger than physical memory.  To get better performance, use proper bs size, for example 2048 or 4096, depends on your local disk i/o,  the rule is to make bs * count > 32 GB.
run

# md5sum /home/memtest; md5sum /home/memtest; md5sum /home/memtest


If you see md5sum mismatch in different run, you have faulty memory guaranteed.
The theory is simple, the file /home/memtest will cache data in memory by filling up all available memory during read operation. Using md5sum command you are reading same data from memory.


5. Other ways to test memory / do a machine stress test

Other good tools you might want to check for memory testing is mprime – ftp://mersenne.org/gimps/ 
(https://www.mersenne.org/ftp_root/gimps/)

  •  (mprime can also be used to stress test your CPU)

Alternatively, use the package stress-ng to run all kind of stress tests (including memory test) on your machine.
Perhaps there are other interesting tools for a diagnosis of memory if you know other ones I miss, let me know in the comment section.

Install postgresql on Debian Squeeze / How to install PostGreSQL on Obsolete Debian installation

Friday, June 10th, 2016

how-to-install-postgresql-on-obsolete-old-debian-squeeze-tutorial

If you're in position like me to be running an old version of Debian (Squeeze) and you need to install PostgreSQL you will notice that the Debian 6.0 standard repositories are no longer active and apt-get update && apt-get upgrade are returning errors, thus because this Debian release is already too old and even the LTS repositories are inactive it is impossible to install postgresql with the usual.

To get around the situation first thing I did was to try to add followin Debian  repositories. to /etc/apt/sources.list
 

deb http://ftp.debian.net/debian-backports squeeze-backports-sloppy main
deb http://archive.debian.org/debian-archive/debian/ squeeze main contrib non-free
deb http://archive.debian.org/debian-archive/debian/ squeeze-lts main contrib non-free

After adding it I continued getting missing package errors while trying:
 

# apt-get update && apt-get install postgresql postgresql-client
….
…..

 

E: Some index files failed to download. They have been ignored, or old ones used instead.


Thus I googled a bit and I found the following PostgreSQL instructions working Debian 7.0 Wheeze and decided to try it 1 in 1 just changing the repository package wheezy word with squeeze
in original tutorial postgre's deb repositories are:

 

deb http://apt.postgresql.org/pub/repos/apt/ wheezy-pgdg main


I've only changed that one with:

 

deb http://apt.postgresql.org/pub/repos/apt/ squeeze-pgdg main

 

I guess though this worked for Debian Squeeze installing current versions such as Debian 8.0 Jessis and newer wouldn't be a prolem if you just change the debian version keyword witht he distribution for which you need the postgresql package


Here is all the consequential steps I took to make the PostgreSQL 9.5 running on my old and unsupported Debian 6.0 Squeeze

Create /etc/apt/sources.list.d/pgdg.list. The distributions are called codename-pgdg. In the example, replace wheezy with the actual distribution you are using:

# vim /etc/apt/sources.list.d/pgdg.list

 

deb http://apt.postgresql.org/pub/repos/apt/ squeeze-pgdg main

debian:~# apt-get –yes install wget ca-certificates debian:~# wget –quiet -O – https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add – debian:~# apt-get update debian:~# apt-get upgrade debian:~# apt-get –yes install postgresql-9.5 pgadmin3

Next step is to connect to PostGreSQL and create database user and a database # su – postgres $ psql

Create a new database user and a database:

postgres=# CREATE USER mypguser WITH PASSWORD 'mypguserpass'; postgres=# CREATE DATABASE mypgdatabase OWNER mypguser;

 

or

# createuser mypguser #from regular shell # createdb -O mypguser mypgdatabase

Quit from the database

postgres=# q

Connect as user mypguser to new database

# su – mypguser $ psql mypgdatabase

or

# psql -d mypgdatabase -U mypguser

If you get errors like:

psql: FATAL: Ident authentication failed for user "mypguser"

edit pg_hba.conf in /etc/postgresql/9.5.Y/main/pg_hba.conf

 

local all all trust # replace ident or peer with trust

reload postgresql

/etc/init.d/postgresql reload …

 


To make sure that PostGreSQL is running on the system check the following processes are present on the server:

 

 

 

root@pcfreak:/var/www/images# ps axu|grep -i post postgres 9893 0.0 0.0 318696 16172 ? S 15:20 0:00 /usr/lib/postgresql/9.5/bin/postgres -D /var/lib/postgresql/9.5/main -c config_file=/etc/postgresql/9.5/main/postgresql.conf postgres 9895 0.0 0.0 318696 1768 ? Ss 15:20 0:00 postgres: checkpointer process postgres 9896 0.0 0.0 318696 2700 ? Ss 15:20 0:00 postgres: writer process postgres 9897 0.0 0.0 318696 1708 ? Ss 15:20 0:00 postgres: wal writer process postgres 9898 0.0 0.0 319132 2564 ? Ss 15:20 0:00 postgres: autovacuum launcher process postgres 9899 0.0 0.0 173680 1652 ? Ss 15:20 0:00 postgres: stats collector process root 14117 0.0 0.0 112404 924 pts/1 S+ 16:09 0:00 grep -i post

 

 


Well that's all folks now you will have the postgresql running on its default port 5433:

 

debian:/etc/postgresql/9.5/main# grep -i port postgresql.conf
port = 5433 # (change requires restart)
# supported by the operating system:
# supported by the operating system:
# ERROR REPORTING AND LOGGING # %r = remote host and port

 

 


Well that's it folks thanks The Lord Jesus Christ grace by the prayers of John The Baptist and Saint Sergij Radonezhki it works 🙂

 

 

Happy Birthday Debian! – Rejoice, Debian GNU / Linux turns 20 :)

Wednesday, August 28th, 2013

Debian Happy birthday cake with debian logo spiral - Debian Linux becomes 20 years old

 A bit outdated news but still worthy to mention as Debian GNU / Linux is important part of my life. On 16 of August this year Debian turned 20 years! I'm actively using Debian Linux for servers and Desktops over the past 13 years and for this time I've seen right before my eyes how debian grow and from buggy hobbyinst Linux distribution became a robust and rock-solid OS. Moreover Debian is now practically the most important Linux distribution around. Thanks to it currently a thousands of other world changing distributions like Ubuntu, Arch Linux, Knoppix Linux LiveCD, Linux Mint etc. Debian is truly multi platform as of time of writting supports 10 hardware architecture (platforms) – in this number Embedded devices like ARM processors, has translation of most shipped software to 73 languages and comes with about 20 000 installable software packages. Contribution of Debian GNU Linux for Free software community is immerse, hundreds of millions or even billion Debian servers or some kind of Debian based OSes are running all around the net. Besides that Debian is one of the largest if not the biggest and most influential Open Source Project. By its essence existing of Debian is just a miracle. 
Though out of date again,  lets great each other with Happy Debian Anniversary and Wish Debian a many and healthy years of successful development!

Alternative way to enter as administrator in MySQL if you forgot MySQL root password on Debian Linux

Wednesday, July 10th, 2013

Forgot MySQL password root alternative way to enter as administrator in MyQL MySQL logo with 2 dolphins

Whether you have to administrate a bunch of chaotic organized MySQL servers and amount of work is more than you can bear it is very common you make stupid mistakes, like loosing MySQL root adminsitrator password. There is way to recover password by stopping SQL server and starting it with –skip-grant-tables options via SSH , however if you do it that way there is at least few seconds of down time and as its not a good idea on productive servers Debian and Ubuntu Linux admins have better way to do it by using MySQL default user used to check whether all is fine with database on MySQL server initialization via /etc/init.d/mysql. User with GRANT PRIVILEGES, (all MySQL administrator users have grant privileges) on Debian based distributions is debian-sys-maint and whether you have root access to server you can easily obtain password with:

# grep -i -E 'user|pass' /etc/mysql/debian.cnf |uniq

user = debian-sys-maint

password = k6x6tBUBfHN3ZxHv

Using this password then you can login via mysql cli or via PhpMyAdmin, whether installed and do any normal SQL operation you do as root. Of course having this password in plain text file can be very dangerous, by default it is configured to be only red by root be careful not to change this permissions by default as anyone who has access to system can then access your SQL as administrator.
To reset MySQL root password once logged in run:

UPDATE USER set password=PASSWORD('NEW_PASS_WORD') where USER='root';
FLUSH PRIVILEGES;

Enjoy 😉
 

ZenMap Nmap multi platform Graphical frontend for checking port security

Saturday, June 15th, 2013

graphic program to scan remote network server port security on GNU Linux and Windows ZenMap

Recently I wrote little article with some examples for scanning server port security with Nmap. I forgot to mention in the article that there is also Nmap frontend GUI program called ZenMap. ZenMap port is available for both Windows and Linux. In Debian, Ubuntu, Mint and other debian derivative distributions ZenMap is available from standard package repositories;

 noah:~# apt-cache show zenmap|grep -i description -A 3

Description-en: The Network Mapper Front End
 Zenmap is an Nmap frontend. It is meant to be useful for advanced users
 and to make Nmap easy to use by beginners. It was originally derived
 from Umit, an Nmap GUI created as part of the Google Summer of Code.
Description-md5: 4e4e4c6aeaa4441484054473e97b7168
Tag: implemented-in::python, interface::x11, network::scanner, role::program,
 uitoolkit::gtk, use::scanning, x11::application
Section: net

To install  ZenMap on Debian / Ubuntu Linux:

noah:~# apt-get install --yes zenmap
...

In Fedora, CentOS and other RPM based Linux-es to install ZenMap run:

noah:~# yum -y install nmap-frontend nmap
...

To use Nmap's Frontend full functionality, you have to run it as (root) superuser:

hipo@noah:~$ sudo su
[sudo] password for hipo:
noah:~# zenmap

Zenmap saves, a lot of time as there is no need to  remember Nmap's arguments or run few Nmap scans until you get essential information for remote scanned machine.
It automatically gives details on Remote server running services (fingerprint)

Zenmap remote server security services scan with services software version

Very useful report it makes as well is network (and host) topology diagram,

network scanner remote host Linux Windows toplogy guess ZenMap screenshot

ZenMap is just Nmap frontend and under the GUI it does use Nmap with various arguments to do produce scan results. In Nmap Output tab, you can see a lot of verbose info.

Zenmap Linux Windows GUI port scanne nmap output tab screen Debian / Ubuntu Linux

Happy scanning 🙂

Remote Desktop client – Remmina, Connect Remote to MS Windows VNC hosts from Linux

Friday, May 31st, 2013

remmina remote connet to windows linux vnc client logo

If you're system administrator, who use Linux as Desktop. You surely want to check out Remmina – The GTK+ Remote Desktop Client.

As far as I tested among all VNC Linux clients I know Remmina is definitely the one of choice in terms of Interface simplicity / stability and remote connection level of responsibility.

Before finding out about Remmina existence, I tried xtightvncviewer, xvnc4viewer, gvncviewer, gtkvncviewer. xtightvncviewer, xvncviewer and gvncviewer are more for console geeks and hence either they lack GUI interface or user interface looks terrible.

GTKVncViewer's interface is also not bad but still not со nice as Remmina's.

gtkvncviewer Debian GNU Linux Wheezy screenshot Linux VNC simple client

As you see in above shot, gtkvncviewer lacks any configuration. The only thing it can do is connect to remote host and you have option to configure nothing related how remote connection will respond, what type of Resolution to use etc.

I know of of no other Linux VNC Clients that has configurability and GUI interface of Remmina.

 

As of time of writting Remmina is at stable version 1.0 and supports following Remote connection protocols:

  • VNC
  • VNC
  • RDP
  • RDPF
  • RDPS
  • SFTP
  • SSH

Remmina is available across mostly all Linux mainstream distributions:

To install Remmina on Debian / Ubuntu and deb derivatives:

debian:~# apt-get –yes install remmina
….

On Redhats (Fedora, CentOS, RHEL – RPM based Linuxes) install via:

[root@centos ~]# yum –yes install remmina

Below are few screenshots of Remmina:

Remmina Linux remote vnc connect best software gui frontend screenshot

Linux VNC best VNC connect tool Remmina preferences screenshot

One of best Remmina feature is it supports Tabbing just like in Firefox. You can open a number of Remote VNC connects to different Windows hosts and manage them all by switching from tab to tab.

Remmina best vnc linux desktop client screenshot with tabs / What is best VNC client for Linux

BSD (Berkley Software Distribituion) use by distribution type (FreeBSD, NetBSD, OpenBSD, DrangflyBSD) use percantage charts

Saturday, February 18th, 2012

I've hit an interesting article in Wikipedia called Comparison of BSD operating systems
The article explains basic difference between different BSD (Berkley Software Distributions) and what is the primary accent of each of the BSD (free software OS) distributions. It also reveals basic details about the history and how each of the BSD's came to existence. I recommend to anyone interested in free software as it is just a great reading for everybody interested in FOSS.

The most interesting part of the wiki thread is a bar chart, provided by BSD Certification Group research conducted in September 2005.

FreeBSD, OpenBSD, NetBSD, Dragonflybsd usage statistics

The above diagram is showing the proportion of users of each BSD variant from the BSD usage survey prior conducted

The research is already 6 years old, and unfortunately as of time of writting seems to be the only publicly available. Though being outdated, I believe generally the bar charts distributions along different BSD variants would be mostly true. The only big difference will be probably in PC-BSD which is not even on the diagram should have outbeaten DragonflyBSD's use. Since there is no public data available for 2012 and the years 2005 – 2012 for the use percantage of each of the BSD distributions, I've thought about a pseudo way to get some general statistics on each of the BSD distributions popularity. The methodology to gather the required statistics comes to simply, type in Google each of the BSD variant "code names" (e.g. freebsd, netbsd, openbsd etc.) and look at the number of results returned. It seems logical the more results distribution keyword searched returns, the bigger the probability of more users to be involved in developing or using the respective BSD variant.

Below you see the results, I've gathered in my quick "google research":

FreeBSD NetBSD OpenBSD BSD variant (users) use diagram based on Google searches of keywords 2012

As you can see in the above data FreeBSD is still probably leading the BSD use, the public interest to OpenBSD – BSD focused on security has significantly grow since the last 6 years. Next it is seen the PC-BSD users base has probably tremendously increased and according to the Google results returned it is probably on a 3rd place by users interest (use?) followed by NetBSD with only 1.47% of all the BSD users. Lastly with only 0.99%, orders Dragonfly BSD which no longer is so popular as a Desktop BSD based OS as it used to be back in 2005.
Again the presented diagram results are based on only on the factor of Google BSD variant popularity and hence shouldn't be consired too trustworthy, still I'm sure it gives a general idea on how used is each of the BSD variants as of Jan 2012.

How to find out all programs bandwidth use with (nethogs) top like utility on Linux

Friday, September 30th, 2011

Just run across across a super nice top like, program for system administrators, its called nethogs and is definitely entering my “l337” admin outfit next to tools like iftop, nettop, ettercap, darkstat htop, iotop etc.

nethogs is ultra easy to use, to get immediately in console statistics about running processes UPLOAD and DOWNLOAD bandwidth consumption just run it:

linux:~# nethogs

Nethogs screenshot on Linux Server with Nginx
Nethogs running on Debian GNU/Linux serving static web content with Nginx

If you need to check what program is using what amount of network bandwidth, you will definitely love this tool. Having information of bandwidth consumption is also viewable partially with iftop, however iftop is unable to track the bandwidth consumption to each process using the network thus it seems nethogs is unique at what it does.

Nethogs supports IPv4 and IPv6 as well as supports network traffic over ppp. The tool is available via package repositories for Debian GNU/Lenny 5 and Debian Squeeze 6.

To install Nethogs on CentOS and Fedora distributions, you will have to install it from source. On CentOS 5.7, latest nethogs which as of time of writting this article is 0.8.0 compiles and installs fine with make && make install commands.

In the manner of thoughts of network bandwidth monitoring, another very handy tool to add extra understanding on what kind of traffic is crossing over a Linux server is jnettop
jnettop shows which hosts/ports is taking up the most network traffic.
It is available for install via apt in Debian 5/6).

Here is a screenshot on jnettop in action:

Jnettop check network traffic in console

To install jnettop on latest Fedoras / CentOS / Slackware Linux it has to be download and compiled from source via jnettop’s official wiki page
I’ve tested jnettop install from source on CentOS release 5.7 and it seems to compile just fine using the usual compile commands:

[root@prizebg jnettop-0.13.0]# ./configure
...
[root@prizebg jnettop-0.13.0]# make
...
[root@prizebg jnettop-0.13.0]# make install

If you need to have an idea on the network traffic passing by your Linux server distringuished by tcp/udp/icmp network protocols and services like ssh / ftp / apache, then you will definitely want to take a look at nettop (if of course not familiar with it yet).
Nettop is not provided as a deb package in Debian and Ubuntu, where it is included as rpm for CentOS and presumably Fedora?
Here is a screenshot on nettop network utility in action:

Nettop server traffic division by protocol screenshot
FreeBSD users should be happy to find out that jnettop and nettop are part of the ports tree and the two can be installed straight, however nethogs would not work on FreeBSD, I searched for a utility capable of what Nethogs can, but couldn’t find such.
It seems the only way on FreeBSD to track bandwidth back and from originating process is using a combination of iftop and sockstat utilities. Probably there are other tools which people use to track network traffic to the processes running on a hos and do general network monitoringt, if anyone knows some good tools, please share with me.

How to Split files on Linux FreeBSD, NetBSD and OpenBSD

Sunday, July 31st, 2011

Split large files in pieces Scissors

Did you have the need to sometimes split an SQL extra large files to few pieces in order to be able to later upload it via phpmyadmin?
Did you needed an extra large video or data file to be cut in few pieces in order to transfer it in few pieces over an USB stick?
Or just to give you an another scenario where I sometimes need to have an enormous file let’s say 3G split in few pieces, in order to later read it in vim or mcedit .
I sometimes need to achieve this on FreeBSD and Linux hosts thus I thought it will be helpful to somebody to give a very quick tutorial on the way large files can be cut in pieces on Linux and BSD hosts.

GNU/Linux and FreeBSD are equipped with the split command. The purpose of this command is exactly the cutting of a file to a number of pieces.

On Linux the split command comes by default install to the system with the coreutils package on most Debian (deb) based and Redhat based (rpm) distributions, theerefore Linux’s version of split is GNU/split since it’s part of the GNU Coreutils package. An interesting fact about Linux split is that one of the two programmers who has coded it is Richard Stallman 😉

On BSD Unix split is the AT&T UNIX (BSD) split

In the past splitting files in pieces was much more needed than today, as people used floppy drives to transfer data, though today with the bloom of Internet and the improve of the data carriers transferring even an extra large files from one place to another is a way more trivial task still at many occasions splitting it in pieces is needed.

Even though today splitting file is very rarely required, still there are times when being able to split a file in X number of parts is very much needed.
Maybe the most common use of splitting a file today is necessery when a large SQL file dumps, like let’s say 200 MBytes of info database needs to be moved from ane hosting provider to another one.
Many hosting providers does disallow direct access with standard mySQL client programs to the database directly and only allow a user to connect only via phpMyAdmin or some other web interface like Cpanel to improve data into the SQL or PostgreSQL server.

In such times, having knowledge on the Unix split command is a priceless asset.

Even though on Linux and BSD the code for the split command is not identical and GNU/split and BSD/split has some basic differences, the use of split on both of these Unices is identical.
The way to split a file in few pieces using on both Linux and BSD OSes is being done with one and the same command, here is how:

1. Splitting file in size of 40 mb On Linux

linux:~# split -b 40m SQL-Backup-Data.sql SQL-Backup-Data_split

2. Splitting file in size of 40mb on BSD (FreeBSD, OpenBSD, NetBSD)

freebsd# split -b 40m SQL-Backup-Data.sql SQL-Backup-Data_split

The Second argument the split command takes is actually called a prefix, the prefix is used as a basis name for the creation of the newly generated files cut in pieces file based on SQL-Backup-Data.sql.

As I said identical command will split the SQL-Backup-Data.sql files in a couple of parts which of it will be sized 40 megas.

These command will generate few files output like:

freebsd# ls -1 SQL-Backup-Dat*SQL-Backup-Data.sql
SQL-Backup-Dataa
SQL-Backup-Dataab
SQL-Backup-Dataac
SQL-Backup-Dataad
SQL-Backup-Dataae

As you see the SQL-Backup-Data.sql with size 200MB is being split in four files each of which is sized 40mbytes.

After the files are transfered to another Linux or BSD host, they can easily be again united in the original file with the command:

linux:~# for i in $(ls -1 SQL-Backup-Data_split*); echo $i >> SQL-Backup-Data.sql

Alternatively in most Unices also using cat should be enough to collect back the pieces into the original file, like so:

freebsd# cat SQL-Backup-Data_split* >> SQL-Backup-Data.sql

Enjoy splitting