Posts Tagged ‘statistics’

Top AIX UNIX Performance tracking commands every Linux admin / user should know

Monday, March 16th, 2015

IBM_AIX_UNIX-Performance-Tracking-every-commands-Linux-sysadmin-and-user-should-know-AIX_logo

Though IBM AIX is basicly UNIX OS and many of the standard Linux commands are same or similar to AIX's if you happen to be a Linux sysadmin and you've been given some 100 AIX servers,  you will have to invest some time to read on AIX, however as a starter you should be aware to at least be able to do performance tracking on system to prevent system overloads. If that's the case I advise you check thoroughfully below commands documentation.

fcstat – Displays statistics gathered by the specified Fibre Channel device driver

filemon – Performance statistics for files, logical/physical volumes and virtual memory segments

fileplace – Displays the placement of file blocks within logical or physical volumes.

entstat – Displays the statistics gathered by the specified Ethernet device driver

iostat – Statistics for ttys, disks and cpu ipcs – Status of interprocess communication facilities

lsps – Statistics about paging space

netstat – Shows network status

netpmon – Performance statistics for CPU usage, network device-driver I/O, socket calls & NFS

nfsstat – Displays information about NFS and RPC calls

pagesize – Displays system page size ps – Display status of current processes

pstat – Statistics about system attributes

sar – System Activity Recorder

svmon – Captures a snapshot of the current contents of both real and virtual memory

traceroute – intended for use in network testing, measurement, and management.

tprof – Detailed profile of CPU usage by an application vmstat – Statistics about virtual memory and cpu/hard disk usage

topas – AIX euqivalent of Linux top command

Here are also useful examples use of above AIX performance tracking commands

To display the statistics for Fiber Channel device driver fcs0, enter:

fcstat fcs0

To monitor the activity at all file system levels and write a verbose report to the fmon.out file, enter:

filemon -v -o fmon.out -O all

To display all information about the placement of a file on its physical volumes, enter:

fileplace -piv data1

To display a continuous disk report at two second intervals for the disk with the logical name disk1, enter the following command:

iostat -d disk1 2

To display extended drive report for all disks, enter the following command:

iostat -D

To list the characteristics of all paging spaces, enter:

lsps -a

List All Ports (both listening and non listening ports)

netstat -a | more

The netpmon command uses the trace facility to obtain a detailed picture of network activity during a time interval.

netpmon -o /tmp/netpmon.log -O all;

netpfmon is very much like AIX Linux equivalent of tcpdump To print all of the supported page size with an alphabetical suffix, enter:

pagesize -af

To display the i-nodes of the system dump saved in the dumpfile core file

pstat -i dumpfile

To report current tty activity for each 2 seconds for the next 40 seconds, enter the following command:

sar -y -r 2 20

To watch system unit for 10 minutes and sort data, enter the following command:

sar -o temp 60 10

To report processor activity for the first two processors, enter the following command:

sar -u -P 0,1

To display global statistics for virtual memory in a one line format every minute for 30 minutes, enter the following command:

svmon -G -O summary=longreal -i 60 30

The traceroute command is intended for use in network testing, measurement, and management. While the ping command confirms IP network reachability, you cannot pinpoint and improve some isolated problems

traceroute aix1

Basic global program and thread-level summary / Reports processor usage

prof -x sleep 10

Single process level profiling

tprof -u -p workload -x workload

Reports virtual memory statistics

vmstat 10 10

To display fork statistics, enter the following command:

vmstat -f

To display the count of various events, enter the following command: vmstat -s To display the count of various events, enter the following command:

vmstat -s

To display time-stamp next to each column of output of vmstat, enter the following command:

vmstat -t

To display the I/O oriented view with an alternative set of columns, enter the following command:

vmstat -I

To display all the VMM statistics available, enter the following command:

vmstat -vs


If you already have some experience with some BSD (OpenBSD or FreeBSD) you will feel much more confortable with AIX as both operating system share common ancestor OS (UNIX System V), actually IBM AIX is U. System V with 4.3 BSD compatible extensions. As AIX was the first OS to introduce file system journalling, journalling capabilities on AIX are superb. AIX was and is still widely used by IBM for their mainframes, on IBM RS/6000 series (in 1990s), nowdays it runs fine on PowerPC-based systems and IA-64 systems.
For GUI loving users which end up on AIX try out SMIT (System Management Interface tool for AIX). AIX was using bash shell in prior versions up to AIX 3 but in recent releases default shell is Korn Shell (ksh88).
Nowdays AIX just like HP-UX and rest of commercial UNICes are loosing ground as most of functionalities is provided by commercial Linux distributions like RHEL so most of clients including Banks and big business clients are migrating to Linux.


Happy AIX-ing ! 🙂

Microsoft Windows most secure OS for 2014 ? – Short OS and Application Security report for 2014

Tuesday, February 24th, 2015

windows-more-secure-OS-for-2014-than-Linux-and-Mac-OSX-and-iOS-operating-systems-short-security-report-2014

It is shocking news for me and probably to many that according to security specialists at National Vulnerability Database, at present moment for year 2014 Windows looks like more secure than both Apple's (iOS and Mac OS X) as well as to Linux.

Windows has been  bullied for its bad OS design and easier to breach Security compared to Linux, there was a constant hype also of Mac OS users claiming the invulnerability of their BSD based OS, but it seems security breach statistics given by  National Vulnerability Database security breach evaluation reports tell us security issues for 2014 Windows OSes while compared to other OS vulnerabilities in different operating systems such as Linux.

statistics-of-Operating-System-security-issues-vulnerabilities-for-2014-windows-most-secure-OS-2014-source-national-vulnerability-database
I will have to disappoint Apple Mac fans but in 2014 Mac OS X was found to be riddled with the greatest number of security problems147 in total, including 64 rated as high severity, and 67 as medium.

iOS's security was also ranked poor with 127 vulnerabilities including 32 high and 72 with a medium rating.

For comparison the latest Windows 8.1 had only 36 vulnerabilities, and its predecessors — Windows 8 and 7 — both had same number.
In Enterprise World (users) Windows Server 2007 and 2008 both have 38 vulnerabilities. Reported vulnerabilities were mainly of middle and high severity.

high-severity-vulnerabilities-graph-of-operating-systems-year-2014

Overall statistics also show there has been a huge increase in the number security vulnerabilities in the NVD security reports database.
In 2013 the number of all logged vulnerabilities were 4,794 while this jumped to 7,038 in y. 2014. The good news is lower percentage of all logged in security issues were rated of critical security importance.
It is mostly third party software not part of OS which contain security issues, 83% of all reported vulnerabilities were laying in 3rd party applications, only 13% percantage were OS specific and 4% hardware related.
Though overall statistics shows Microsoft products more secure than Apple Inc. Products and (Open Source) Linux, though still M$ Internet Explorer is the most insecure web browser, for 2014,  Internet Explorer had  242 vulnerabilities while Google Chrome had 124 security issues and the most secure browser rated for 2014 is (surprising for me) Mozilla Firefox.
It is important to say such statistics are not completely relevant because, for example you can rarely see a Linux desktop user infected with Malware but almost everyone around using Windows OS is malware infected, same goes for Mac OS users, there are plenty of vulnerabilities for Mac but overall security of Mac OS is better as I haven't still met Mac OS users with Viruses and Spyware but I fixed about (30!!) of Microsoft PCs and notebooks infected with various Viruses and badware throughout 2014. Also it should be considered that many securitty bugs are kept secret and actively exploited for a long time by blackhats like it happened recently with Heartbleed and ShellShock vulnerabilities
For those interested, below is a list of top vulnerable applications for 2014

security-issues-vulnerability-report-2012-2013-2014_graph_windows-most-secure-operating-system-for-2014

Disable php notice logging / stop variable warnings in error.log on Apache / Nginx / Lighttpd

Monday, July 28th, 2014

disable_php_notice_warnings_logging_in-apache-nginx-lighttpd
At one of companies where I administrate few servers, we are in process of optimizing the server performance to stretch out the maximum out of server hardware and save money from unnecessery hardware costs and thus looking for ways to make server performance better.

On couple of web-sites hosted on few of the production servers, administrating, I've noticed dozens of PHP Notice errors, making the error.log quickly grow to Gigabytes and putting useless hard drive I/O overhead. Most of the php notice warnings are caused by unitialized php variables.

I'm aware having an unitialized values is a horrible security hole, however the websites are running fine even though the notice warnings and currently the company doesn't have the necessery programmers resource to further debug and fix all this undefined php vars, thus what happens is monthly a couple of hundreds megabytes of useless same php notice warnings are written in error.log.

That  error.log errors puts an extra hardship for awstats which is later generating server access statistics while generating the 404 errors statistics and thus awstats script has to read and analyze huge files with plenty of records which doesn't have nothing to do with 404 error

We found this PHP Notice warnings logged is one of the things we can optimize had to be disabled.

Here is how this is done:
On the servers running Debian Wheezy stable to disable php notices.

I had to change in /etc/php5/apache2/php.ini error_reporting variable.

Setting was to log everything (including PHP critical errors, warning and notices) like so:
 

vi /etc/php5/apache2/php.ini

error_reporting = E_ALL & ~E_DEPRECATED

to

error_reporting = E_COMPILE_ERROR|E_ERROR|E_CORE_ERROR


On CentOS, RHEL, SuSE based servers, edit instead /etc/php.ini.

This setting makes Apache to only log in error.log critical errors, php core dump (thread) errors and php code compilation (interpretation errors)

To make settings take affect on Debian host Apache webserver:

/etc/init.d/apache2 restart

On CentOS, RHEL Linux, had to restart Apache with:

/etc/init.d/httpd restart

For other servers running Nginx and Lighttpd webservers, after changing php.ini:

service nginx reload
service lighttpd restart

To disable php notices errors only on some websites, where .htaccess enabled, you can use also place in website DocumentRoot .htaccess:
 

php_value error_reporting 2039


Other way to disable via .htaccess is by adding to it code:
 

php_flag display_errors off


Also for hosted websites on some of the servers, where .htaccess is disabled, enabling / disabling php notices can be easily triggered by adding following php code to index.php

define('DEBUG', true);

if(DEBUG == true)
{
    ini_set('display_errors', 'On');
    error_reporting(E_ALL);
}
else
{
    ini_set('display_errors', 'Off');
    error_reporting(0);
}

 

How to check Microsoft Windows uptime – Check server uptime in Windows server

Wednesday, May 21st, 2014

how-to-check-windows-uptime-windows-server-uptime-logo
In Linux to check uptime there is the uptime command, so how is it possible to check your system uptime – e.g. check when was last time Windows host was rebooted?

Or in other words what is Windows server equivalent to Linux's uptime command?

To check uptime on Windows OS, there is the:

net statistics server

command a shorter reference to this command is net stats srv

To run it quickest way is to press Windows (button)+r type cmd.exe and exec command in Windows command prompt:

 

C:UsersGeorgi>net statistics server
Server Statistics for \SM07862

Statistics since 21.05.2014 09:55:21

Sessions accepted 1
Sessions timed-out 0
Sessions errored-out 0

Kilobytes sent 0
Kilobytes received 0

Mean response time (msec) 0

System errors 0
Permission violations 0
Password violations 0

Files accessed 0
Communication devices accessed 0
Print jobs spooled 0

Times buffers exhausted

Big buffers 0
Request buffers 0

The command completed successfully.

C:UsersGeorgi>

Statistics since 21.05.2014 09:55:21 – shows when system booted last time, so to check the difference between current time and when system booted last – you need to check current time with time command

 


C:UsersGeorgi>time
The current time is: 16:59:26,60
Enter the new time:

Alternative command to check when Windows system booted is:

C:UsersGeorgi>systeminfo|findstr "System Boot Time"
System Boot Time: 21.05.2014, 09:54:11
System Manufacturer: HP
System Model: ProLiant BL460c G7
System Type: x64-based PC
System Directory: C:Windowssystem32
Boot Device: DeviceHarddiskVolume1
System Locale: de;German (Germany)
Time Zone: (UTC+01:00) Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna

C:UsersGeorgi>

If you want to check Windows boot time "the Windows way" through the GUI, launch Windows Task Manager – run taskmgr command and go to Performance tab

images/check-windows-server-uptime-with-taskmanager-performance-tab-screenshot

 

IQ world rank by country and which are the smartest nations

Friday, March 14th, 2014

IQ_world_rank_by_country_world_distribution_of_intelligence
In a home conversation with my wife who is Belarusian and comparison between Bulgarian and Belarusian nation, the interesting question arised – Which nation is Smarter Bulgarian or Belarusian?

This little conversation pushed me to intriguing question What is the IQ World rank if compared by country? Since a moment of my life I'm trying to constantly prove to myself I'm smart enough. For years my motivation was to increase my IQ. I had periods when studied hard history, philosophy and literature then I had periods to put all my efforts in music and mysticism then there was my fascination about IT and informatics and hacking, I had periods with profound interest in Biology and neourosciences, then of course psychology and social sciences and since last 10 years as I belived in God, I'm deeply interested in world religions and more particularly in Christniaty. All this is connected with my previous IQ (Intelligence Quotient) and my desire to develop my IQ. I'm quite aware that IQ statistics can never be 100% reliable as there is deviation (standard error) and its a very general way to find out about a person psychology. But anyways it is among the few methods to compare people's intelligence… I've done an IQ test in distant 2008 and I scored about 118 out of 180  – meaning my  IQ level is a little bit above average. The IQ conversation triggered my curiousity so I decided to check if my current IQ has changed over the last 6 years. Here is results from test I took March, 2013 on free-iqtest.net

IQ Test
IQtest just prooved, my IQ kept almost same, still a little bit above avarage.
Further on, I did investgation online to see if I can prove to my wife the thesis Bulgarians overall IQ is higher than Belarusian. I googled for IQ world rank by Country
Here is what I found ;

 

Nations Intelligence as sorted by Country

Rank
——–

Country
———————–

%
————-

1

Singapore

108

2

South Korea

106

3

Japan

105

4

Italy

102

5

Iceland

101

5

Mongolia

101

6

Switzerland

101

7

Austria

100

7

China

100

7

Luxembourg

100

7

Netherlands

100

7

Norway

100

7

United Kingdom

100

8

Belgium

99

8

Canada

99

8

Estonia

99

8

Finland

99

8

Germany

99

8

New Zealand

99

8

Poland

99

8

Sweden

99

9

Andorra

98

9

Australia

98

9

Czech Republic

98

9

Denmark

98

9

France

98

9

Hungary

98

9

Latvia

98

9

Spain

98

9

United States

98

10

Belarus

97

10

Malta

97

10

Russia

97

10

Ukraine

97

11

Moldova

96

11

Slovakia

96

11

Slovenia

96

11

Uruguay

96

12

Israel

95

12

Portugal

95

13

Armenia

94

13

Georgia

94

13

Kazakhstan

94

13

Romania

94

13

Vietnam

94

14

Argentina

93

14

Bulgaria

93

15

Greece

92

15

Ireland

92

15

Malaysia

92

16

Brunei

91

16

Cambodia

91

16

Cyprus

91

16

FYROM

91

16

Lithuania

91

16

Sierra Leone

91

16

Thailand

91

17

Albania

90

17

Bosnia and Herzegovina

90

17

Chile

90

17

Croatia

90

17

Kyrgyzstan

90

17

Turkey

90

18

Cook Islands

89

18

Costa Rica

89

18

Laos

89

18

Mauritius

89

18

Serbia

89

18

Suriname

89

19

Ecuador

88

19

Mexico

88

19

Samoa

88

20

Azerbaijan

87

20

Bolivia

87

20

Brazil

87

20

Guyana

87

20

Indonesia

87

20

Iraq

87

20

Myanmar (Burma)

87

20

Tajikistan

87

20

Turkmenistan

87

20

Uzbekistan

87

21

Kuwait

86

21

Philippines

86

21

Seychelles

86

21

Tonga

86

22

Cuba

85

22

Eritrea

85

22

Fiji

85

22

Kiribati

85

22

Peru

85

22

Trinidad and Tobago

85

22

Yemen

85

23

Afghanistan

84

23

Bahamas, The

84

23

Belize

84

23

Colombia

84

23

Iran

84

23

Jordan

84

23

Marshall Islands

84

23

Micronesia, Federated States of

84

23

Morocco

84

23

Nigeria

84

23

Pakistan

84

23

Panama

84

23

Paraguay

84

23

Saudi Arabia

84

23

Solomon Islands

84

23

Uganda

84

23

United Arab Emirates

84

23

Vanuatu

84

23

Venezuela

84

24

Algeria

83

24

Bahrain

83

24

Libya

83

24

Oman

83

24

Papua New Guinea

83

24

Syria

83

24

Tunisia

83

25

Bangladesh

82

25

Dominican Republic

82

25

India

82

25

Lebanon

82

25

Madagascar

82

25

Zimbabwe

82

26

Egypt

81

26

Honduras

81

26

Maldives

81

26

Nicaragua

81

27

Barbados

80

27

Bhutan

80

27

El Salvador

80

27

Kenya

80

28

Guatemala

79

28

Sri Lanka

79

28

Zambia

79

29

Congo, Democratic Republic of the

78

29

Nepal

78

29

Qatar

78

30

Comoros

77

30

South Africa

77

31

Cape Verde

76

31

Congo, Republic of the

76

31

Mauritania

76

31

Senegal

76

32

Mali

74

32

Namibia

74

33

Ghana

73

34

Tanzania

72

35

Central African Republic

71

35

Grenada

71

35

Jamaica

71

35

Saint Vincent and the Grenadines

71

35

Sudan

71

36

Antigua and Barbuda

70

36

Benin

70

36

Botswana

70

36

Rwanda

70

36

Togo

70

37

Burundi

69

37

Cote d'Ivoire

69

37

Ethiopia

69

37

Malawi

69

37

Niger

69

38

Angola

68

38

Burkina Faso

68

38

Chad

68

38

Djibouti

68

38

Somalia

68

38

Swaziland

68

39

Dominica

67

39

Guinea

67

39

Guinea-Bissau

67

39

Haiti

67

39

Lesotho

67

39

Liberia

67

39

Saint Kitts and Nevis

67

39

Sao Tome and Principe

67

40

Gambia, The

66

41

Cameroon

64

41

Gabon

64

41

Mozambique

64

42

Saint Lucia

62

43

Equatorial Guinea

59

 

North Korea

N/A

 

– Countries are ranked highest to lowest national IQ score.

Above statistics are taken from a work carried out earlier this decade by Richard Lynn, a British psychologist, and Tatu Vanhanen, a Finnish political scientist. To extract statistics they analized  IQ studies from 113 countries.  

For my surprise it appeared Belarusian (ranking 10th in the world) have generally higher IQ than Bulgarians (ordering 14th). Anyways being 14th in world IQ Ranking is not bad at all as we still rank in the top 20 smartest nations.

IQ is a relative way to measure intelligence, so I don't believe these statistics are revelant but they give some very general idea about world IQs.

I learned there are some claims that in more developed economies people have higher IQs than less developed. If we take in consideration above statistics its obvious such claims are dubious as you can see there are countries in top 5 countries with highest IQ, and surely Mongolia is not to be ordered in countries with high economic development.

There are plenty of other interesting researches like "Does IQ relates to people Income?", Does Religious people score higher than atheists? According to research done in U.S. Atheists score 6 IQ points higher than Religious people. However most "religous" people IQ tested were from protestant origin so results are relative (I'm sure Orthodox Christian would score higher 🙂 ). The IQ nation world ranks fail in a way that, a social, economic and historical factors are not counted. According to Gallups research, the world poorest people tend to be the most religious, a fact supporting well the saying of all saints who say that for saintly life people who preferred deliberately to live as poor people.

 

Linux: Understanding uptime command Load Avarage statistics / When load avarage is high?

Wednesday, May 8th, 2013

GNU / Linux load avarage explained load avarage from top command

There is probably no Linux system administrator who, don't have idea about system  Load Avarage. Most of admins however does have some brought idea about what kind of load avarage is critical but doesn't have good understanding on the 3 digits returned as a load avarage i.e. – load average: 2.47, 2.27, 2.02 shown in above ascii graphs ( generated by tload command).
 

What is Load Avarage ?

  •  The number of blocking processes in the run queue averaged over a certain time period.

A blocking process is a process that is waiting for something to continue. Typically, a process is waiting to use:
 

  •  CPU Time,  Disk Input / Output oper. or Network I / O

Thus logically the higher the Load Avarage, the more processes has to wait for access to CPU, HDD and Network I/O.

The most two common commands used where load avarage appear are;
 

w – who

and

uptime

mx:/home/hipo# w
 11:07:56 up 513 days,  1:04,  1 user,  load average: 1.92, 1.95, 1.84
USER     TTY      FROM              LOGIN@   IDLE   JCPU   PCPU WHAT
hipo     pts/0    pc-freak.net     Thu19    0.00s  0.06s  0.02s sshd: hipo [priv]

mail:/home/hipo# uptime
 11:03:59 up 513 days,  1:00,  2 users,  load average: 2.11, 1.91, 1.81

Other common place to check load avarage is in top cmd:

mail:/home/hipo# top

Linux top command load avarage showing server system load 3 digits of load avarage explained

a) Optimum machine use – Load Avarage 1 

So what does load avarage: 1.74, 1.90, 1.83 really means? The 3 digits are showing system load avarage over the last 1, 5 and 15 minutes time. Meaning;

 

– before 1 minute system had a load of 1.74
– 5 minutes before it was 1.90
– and 15 minutes back 1.83

Usually Load Avarage of more than 1 is considered critical. If a system is working with a load avarage of 1 this means the system is working capacity. In best cases in terms of optimizing processes on server with hardware it is good the system is working in load of 0.70 or 0.80. Whether a traffic the machine gets is planned in most cases a load avarage of exactly 1 means machine hardware is properly utilized. However whether the load avarage is hitting over 1 this usually means you have to think about moving server to new hardware. It is general rule of thumb that if system load is exceeeding 0.70 it is time to migrate to better hardware.

b) Load avarage on Multi-core / Multiprocessor servers

Load avarage of 4 on 4 CPU cores server hardware is optimum one. Each core / CPU on machine should get maximum of load avarage 1. Load avarage of 1 means CPU is utilized in 100%. Load avarage of 4 on 4 CPU server hardware means all 4 processors are working in their maximum power of 100%. For people who have multi processor server the best way to show utilization is by running htop. There all 4 CPUs will show idle of 0%.

Hence rule to calculate normal load avarage for server is;

1 Load Avarage per CPU. Therefore for 24 CPU Intel Xeon hardware. Load Avarage under 1*24.00 = 24.00 is considered normal. On such a server whether load avarage jumps to 50.00 / 70.00 or above server becomes totally irresponsive and it is very likely to hang because of over-heating. Even if it continues working it will work extremely slow and even simply operations like ssh to it will become hardly possible and sometimes even access via ssh will be not possible.

Therefore Rule of Thumb for calculating which load avarage is okay for a server is;

Number of CPU / Cores should not exceed digit returned in Load Avarage stats

c) Critical – Load avarage >5 – A sure sign for unresponsive or soon to hang server

On Computers with just 1 CPU, load avarage of 5 is sure sign running services will lag brutally and server will become inaccessible. For multicore / multiprocessor servers big troubles can be expected, whether load avarage is about  1/2 of the maximum number of of Load Avarage; (for 8 CPU Multicore hardware). A load avarage of 8 + ( 1/2 * 8 ) = 12 is sure sign system is stoned and running services inaccessible.

d) load avarage: 1.74, 1.90, 1.83 – Is 1 / 5 or 15 minutes LA numbers more important to consider?

All are important however 5 and 15 minutes load avg. give better indication on what's happening with machine as current load can peak for just a second to a higher number, being misleading.

To get number of CPU / Cores use cmd;

mail:/home/hipo# grep 'model name' /proc/cpuinfo | wc -l
24

For more precise info on CPU type and model use;

mail:/home/hipo# cat /proc/cpuinfo

processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 44
model name      : Intel(R) Xeon(R) CPU           E5645  @ 2.40GHz
stepping        : 2
cpu MHz         : 2400.094
cache size      : 12288 KB
physical id     : 0
siblings        : 12
core id         : 0
cpu cores       : 6
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 11
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida arat tpr_shadow vnmi flexpriority ept vpid
bogomips        : 4800.18
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:
……
………
 

If you hit abnormal high load avarage, it is useful to check in top process list what is top process / processes causing highest system load. It is useful to run ps with following arguments

mail:/home/hipo# ps axuwwf

Look in STAT column. Processes in STAT have 3 states;

  • R – Running
  • S – Sleeping
  • D – Waiting for something

Usually processes with status of D – are ones causing problems. If you get D STAT-ed processes check further what's wrong with them and fix it. If there are none,  simply, number of clients using machine pop-ed up meaning you need to quickly move to better hardware host.

e) Getting notified via email whether load avarage exceeds certain value

A good way to get notified or do certain action like restarting Apache WebServer or other common process causing high loads is through monit. Monit is very usefukl for notifying on high load avarages or even better for Restarting processes imposing high loads.

You can always use a few liners shell script to mail to email or SMS2Email mailbox similar to this tiny shell script to restart apache on high load.There is also a Ruby lang tool – Scout to monitor and report high load avarages

How to install Awstats Apache weblog statistics on Debian Squeeze GNU Linux

Monday, October 8th, 2012

I like using Webalizer to keep an eye in web of my access.log, however since the information it shows is a bit chaotic and much less than one in Awstats statistics, I decided to install awstats. I haven’t installed awstats for a long time so I have no exact memory how I previously did it and hence run quick search too see if there is information on specifics concerning Debian Squeeze. I did not find any specific article and therefore decided to write this short one to document how awstats install is done on Debian Squeeze Linux.

1. Installing awstats deb package

There is already a deb package so no need to hunt for specific perl CPAN modules and manually fulfill dependencies.

Installation is as straight as any other deb package:


debian:~# apt-get install --yes awstats
...

2. Change basic awstats.conf configurations to make it properly working

First thing to do immediately after install is to set the primary SiteDomain= for which Awstats will process site statistics.

For that in the beginning (first line) of /etc/awstats/awstats.conf add directive:


SiteDomain="www.your-domain-name.com"

Substitution www.your-domain-name.com with whatever your primary domain will be.

Next in /etc/awstats/awstats.conf change value for DNSLookup. By default DNSLookup is 1, which means each of the IP request in /var/log/apache2/access.log is attempted be resolved via separate DNS request; Most IP Addresses that have quieried Apache webserver however miss proper PTR DNS record and hence attempts to be resolved fail after 10 to 20 seconds.
The overall result of this is processing execution of /var/log/apache2/access.log takes hours in case access.log is >100MB or so. This slow processing slowness is due to failed DNS requests. Besides that it does useless hundreds of queries to DNS servers which take up bandwidth for nothing …

To prevent this I disabled immediately DNSLookup right after install by substituting


DNSLookup=1

with:


DNSLookup=0

Other thing is by default Awstats is set to use LogFormat=4. As you can read in awstats.conf (Comments section) 4 stands for:


# 4 - Apache or Squid native common log format (NCSA common/CLF log format)

However in Debian Linux Apache2 default config is done in a way that Apache keeps logged requests formatted in combined (not in common log

Here is LogFormat directive extracted from /etc/apache2/apache2.conf:


LogFormat "%v:%p %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" vhost_combined
LogFormat "%h %l %u %t \"%r\" %>s %O" common
LogFormat "%{Referer}i -> %U" referer
LogFormat "%{User-agent}i" agent

With that said in awstats.conf to match (combined) Apache set logging change LogFormat to 1:


LogFormat=1

3. Generate manual AWStats access.log statistics for first time

You will have to run as superuser following cmd:


debian:~# /usr/lib/cgi-bin/awstats.pl -config=/etc/awstats/awstats.conf
Create/Update database for config "/etc/awstats/awstats.conf" by AWStats version 6.95 (build 1.943)
From data in log file "/var/log/apache2/access.log"...
Phase 1 : First bypass old records, searching new record...
Searching new records from beginning of log file...
Phase 2 : Now process new records (Flush history on disk after 20000 hosts)...
Flush history file on disk (unique url reach flush limit of 5000)
Flush history file on disk (unique url reach flush limit of 5000)
Flush history file on disk (unique url reach flush limit of 5000)
Flush history file on disk (unique url reach flush limit of 5000)
Flush history file on disk (unique url reach flush limit of 5000)
Flush history file on disk (unique url reach flush limit of 5000)
Flush history file on disk (unique url reach flush limit of 5000)
Flush history file on disk (unique url reach flush limit of 5000)
Flush history file on disk (unique url reach flush limit of 5000)
Flush history file on disk (unique url reach flush limit of 5000)
Jumped lines in file: 0
Parsed lines in file: 602983
Found 8 dropped records,
Found 5 corrupted records,
Found 0 old records,
Found 602970 new qualified records.

4. Access awstats statistics in Web Browser

Once the command execution completes, open in Epiphany, Firefox or whatever browser you like URL:


http://www.your-domain-name.com/cgi-bin//awstats.pl?config

If all is okay you should see some numbers on Unique Visitors like in below browser screenshot:

Screenshot Awstats example Statistics for pc-freak.net in Epiphany

5. Set ScriptAlias for easier awstats access path and set directory permissions

In /etc/apache2/apache2.conf or in VirtualHost file, lets say /etc/apache2/sites-enabled/your-domain-name.com, place following configs:


Alias /awstats-icon/ /usr/share/awstats/icon/
ScriptAlias /awstats/ /usr/lib/cgi-bin/

Options None
AllowOverride None
Order allow,deny
Allow from all

For new configs to take effect as usual Apache should be restarted:


debian:~# /etc/init.d/apache2 restart
....

From now on Awstats can be accessed via the much easier to remember access URL:


http://your-domain-name.com/awstats/awstats.pl

6. Protecting Awstats statistics with Apache HTACCESS password

It is a must to protect awstats statistics with password via .htaccess and htpasswd

a.) Use htpasswd to generate user/pass:


linux:~# htpasswd -c /etc/apache2/awstats.passwd admin
New password:
Re-type new password:
Adding password for user admin

b.) Create /usr/lib/cgi-bin/.htaccess with following content:


linux:~# vim /usr/lib/cgi-bin/.htaccess

AuthType Basic
AuthUserFile /etc/apache2/awstats.passwd
AuthGroupFile /dev/null
AuthName "Please Enter Password to access AWstat"
AuthType Basic
Require valid-user

7. Set awstats to generate statistics via daily cron job:

awstats binary deb automatically installs a cron job in /etc/cron.d/awstats:


linux:~# cat /etc/cron.d/awstats
*/10 * * * * www-data [ -x /usr/share/awstats/tools/update.sh ] && /usr/share/awstats/tools/update.sh
# Generate static reports:
10 03 * * * www-data [ -x /usr/share/awstats/tools/buildstatic.sh ] && /usr/share/awstats/tools/buildstatic.sh

I prefer not to use it but use a custom root cron job. To stop /etc/cron.d/awstats from executing I move it to /root:


mv /etc/cron.d/awstats /root

Then I set a new root user cron job to process Apache access.log. The reason I use root user crontab, instead of Apache’s www-data is with www-data user, /var/log/apache2/access.log is unreadable ,…


linux:~# crontab -u root -e
8,18,27,38,48,59 * * * * [ -x /usr/lib/cgi-bin/awstats.pl -a -f /etc/awstats/awstats.conf -a -r /var/log/apache2/access.log ] && /usr/lib/cgi-bin/awstats.pl -config=awstats -update >/dev/null

This cron makes awstats web statistics be refreshed every our in minutes 8,18,27,38,48,59.

That’s it. Enjoy 🙂

Did you that every single day there are …

Saturday, April 7th, 2012

Do you know that every single day there are:

  • 150 Million Blogs being read
  • 60 Million New Facebook Status Updates
  • 140 Million New "Treets" …
  • 2 Million New Videos Posted to Youtube per day
  • 3.3 Billion! Product and Information Searches
  • 188 Billion! Emails Sent …

 

This statistics were sent to me just few weeks, ago by a friend. I have no clue where he got the statistics, but the numbers are really amazing. From a business perspective point of view this numbers are tremendously HIGH! Nowadays only about 2.5 billion people are actively using the internet daily.

This means more than half of the humanity is still about to join using the internet in the 10 or 15 years to come. Though the continuous use of internet has a very bad impact on us. It is a tremendously big business opportunity field. With this said definitely business innovative people and enterpreneurs should reconsider, there strategic plans for potential products and extend or include the internet in anything they do in order to maximize profits.

All this is just news for anyone who is involved somehow actively with the net (like me), so system admins, web designers, programmers, use your brains and start making money from the internet. If one doesn't start with monetarizing his high tech skills, its quite likely some bad tied suit guy took the lead and made his millions or billions  on our back 🙂

diskinfo Linux hdparm FreeBSD equivalent command for disk info and benchmarking

Thursday, March 8th, 2012

FreeBSD Linux hdparm equivalent is diskinfo artistic logo

On Linux there is the hdparm tool for various hard disk benchmarking and extraction of hard disk operations info.
As the Linux manual states hdparmget/set SATA/IDE device parameters

Most Linux users should already know it and might wonder if there is hdparm port or equivalent for FreeBSD, the aim of this short post is to shed some light on that.

The typical use of hdparm is like this:

linux:~# hdparm -t /dev/sda8

/dev/sda8:
Timing buffered disk reads: 76 MB in 3.03 seconds = 25.12 MB/sec
linux:~# hdparm -T /dev/sda8
/dev/sda8:
Timing cached reads: 1618 MB in 2.00 seconds = 809.49 MB/sec

The above output here is from my notebook Lenovo R61i.
If you're looking for alternative command to hdparm you should know in FreeBSD / OpenBSD / NetBSD, there is no exact hdparm equivalent command.
The somehow similar hdparm equivallent command for BSDs (FreeBSD etc.) is:
diskinfo

diskinfo is not so feature rich as linux's hdparm. It is just a simple command to show basic information for hard disk operations without no possibility to tune any hdd I/O and seek operations.
All diskinfo does is to show statistics for a hard drive seek times I/O overheads. The command takes only 3 arguments.

The most basic and classical use of the command is:

freebsd# diskinfo -t /dev/ad0s1a
/dev/ad0s1a
512 # sectorsize
20971520000 # mediasize in bytes (20G)
40960000 # mediasize in sectors
40634 # Cylinders according to firmware.
16 # Heads according to firmware.
63 # Sectors according to firmware.
ad:4JV48BXJs0s0 # Disk ident.

Seek times:
Full stroke: 250 iter in 3.272735 sec = 13.091 msec
Half stroke: 250 iter in 3.507849 sec = 14.031 msec
Quarter stroke: 500 iter in 9.705555 sec = 19.411 msec
Short forward: 400 iter in 2.605652 sec = 6.514 msec
Short backward: 400 iter in 4.333490 sec = 10.834 msec
Seq outer: 2048 iter in 1.150611 sec = 0.562 msec
Seq inner: 2048 iter in 0.215104 sec = 0.105 msec

Transfer rates:
outside: 102400 kbytes in 3.056943 sec = 33498 kbytes/sec
middle: 102400 kbytes in 2.696326 sec = 37978 kbytes/sec
inside: 102400 kbytes in 3.178711 sec = 32214 kbytes/sec

Another common use of diskinfo is to measure hdd I/O command overheads with -c argument:

freebsd# diskinfo -c /dev/ad0s1e
/dev/ad0s1e
512 # sectorsize
39112312320 # mediasize in bytes (36G)
76391235 # mediasize in sectors
75784 # Cylinders according to firmware.
16 # Heads according to firmware.
63 # Sectors according to firmware.
ad:4JV48BXJs0s4 # Disk ident.

I/O command overhead:
time to read 10MB block 1.828021 sec = 0.089 msec/sector
time to read 20480 sectors 4.435214 sec = 0.217 msec/sector
calculated command overhead = 0.127 msec/sector

Above diskinfo output is from my FreeBSD home router.

As you can see, the time to read 10MB block on my hard drive is 1.828021 (which is very high number),
this is a sign the hard disk experience too many read/writes and therefore needs to be shortly replaced with newer faster one.
diskinfo is part of the basis bsd install (bsd world). So it can be used without installing any bsd ports or binary packages.

For the purpose of stress testing hdd, or just some more detailed benchmarking on FreeBSD there are plenty of other tools as well.
Just to name a few:
 

  • rawio – obsolete in FreeBSD 7.x version branch (not available in BSD 7.2 and higher)
  • iozone, iozone21 – Tools to test the speed of sequential I/O to files
  • bonnie++ – benchmark tool capable of performing number of simple fs tests
  • bonnie – predecessor filesystem benchmark tool to bonnie++
  • raidtest – test performance of storage devices
  • mdtest – Software to test metadata performance on filsystems
  • filebench – tool for micro-benchmarking storage subsystems

Linux hdparm allows also changing / setting various hdd ATA and SATA settings. Similarly, to set and change ATA / SATA settings on FreeBSD there is the:

  • ataidle

tool.

As of time of writting ataidle is in port path /usr/ports/sysutils/ataidle/

To check it out install it as usual from the port location:

FreeBSD also has also the spindown port – a small program for handling automated spinning down ofSCSI harddrive
spindown is useful in setting values to SATA drives which has problems with properly controlling HDD power management.

To keep constant track on hard disk operations and preliminary warning in case of failing hard disks on FreeBSD there is also smartd service, just like in Linux.
smartd enables you to to control and monitor storage systems using the Self-Monitoring, Analysisand Reporting Technology System (S.M.A.R.T.) built into most modern ATA and SCSI hard disks.
smartd and smartctl are installable via the port /usr/ports/sysutils/smartmontools.

To install and use smartd, ataidle and spindown run:

freebsd# cd /usr/ports/sysutils/smartmontools
freebsd# make && make install clean
freebsd# cd /usr/ports/sysutils/ataidle/
freebsd# make && make install clean
freebsd# cd /usr/ports/sysutils/spindown/
freebsd# make && make install clean

Check each one's manual for more info.

Fix vnstat error “eth0: Not enough data available yet.” on Debian GNU / Linux

Monday, November 21st, 2011

Vnstat GNU Linux console terminal traffic statistics logo

After installing vnstat to keep an eye on server IN and OUT traffic on a Debian Squeeze server. I used the usual:

debian:~# vnstat -u -i eth0

In order to generate the initial database for the ethernet interface used by vnstat to generate its statistics.

However even though /var/lib/vnstat/eth0 got generated with above command statistics were not further generated and trying to check them with command:

debian:~# vnstat --days

Returned the error message:

eth0: Not enough data available yet.

To solve the eth0: Not enough data available yet. message I tried completely removing vnstat package by purging the package e.g.:

debian:~# apt-get --yes remove vnstat
...
debian:~# dpkg --purge vnstat
...

Even though dpkg –purge was invoked /var/lib/vnstat/ refused to be removed since it contained vnstat’s db file eth0

Therefore I deleted by hand before installing again vnstat:

debian:~# rm -rf /var/lib/vnstat/

Tried installing once again vnstat “from scratch”:

debian:~# apt-get install vnstat
...

After that I tried regenerating the vnstat db file eth0 once again with vnstat -u -i eth0 , hoping this should fix the error but it was no go and after that the error:

debian:~# vnstat --hours
eth0: Not enough data available yet.

persisted.

I checked in Debian bugs mailing lists and I found, some people complaining about the same issue with some suggsetions on how the error can be work arouned, anyways none of the suggestions worked for me.

Being irritated I further removed / purged once again vnstat and decided to give it a try by installing vnstat from source
As of time of writting this article, the latest stable vnstat version is 1.11 .
Therefore to install vnstat from source I issued:

debian:~# cd /usr/local/src
debian:/usr/local/src# wget http://humdi.net/vnstat/vnstat-1.11.tar.gz
...
debian:/usr/local/src# tar -zxvvf vnstat-1.11.tar.gz
debian:/usr/local/src# cd vnstat-1.11
debian:/usr/local/src/vnstat-1.11# make & make all & make install
debian:/usr/local/src/vnstat-1.11# cp examples/vnstat.cron /etc/cron.d/vnstat
debian:/usr/local/src/vnstat-1.11# vnstat -u -i eth0
Error: Unable to read database "/var/lib/vnstat/eth0".
Info: -> A new database has been created.

As a last step I put on root crontab to execute:

debian:~# crontab -u root -e

*/5 * * * * /usr/bin/vnstat -u >/dev/null 2>&1

This line updated vnstat db eth0 database, every 5 minutes. After the manual source install vnstat works, just fine 😉