Posts Tagged ‘Human’

Get dmesg command kernel log report with human date / time timestamp on older Linux distributions

Friday, June 18th, 2021

how-to-get-dmesg-human-readable-timestamp-kernel-log-command-linux-logo

If you're a sysadmin you surely love to take a look at dmesg kernel log output. Usually on many Linux distributions there is a setup that dmesg keeps logging to log files /var/log/dmesg or /var/log/kern.log. But if you get some inherited old Linux servers it is quite possible that the previous machine maintainer did not enable the output of syslog to get logged in /var/log/{dmesg,kern.log,kernel.log}  or even have disabled the kernel log for some reason. Even though that in dmesg output you might find some interesting events reporting issues with Hard drives on its way to get broken / a bad / reads system processes crashing or whatever of other interesting information that could help you prevent severe servers downtimes or problems earlier but due to an old version of Linux distribution lets say Redhat 5 / Debian 6 or old CentOS / Fedora, the version of dmesg command shipped does not support the '-T' option that is present in util-linux package shipped with newer versions of  Redhat 7.X  / 8.X / SuSEs etc.  

 -T, –ctime
              Print human readable timestamps.  The timestamp could be inaccurate!

To illustrate better what I mean, here is an example from the non-human readable timestamp provided by older dmesg command

root@web-server~:# dmesg |tail -n 5
[4505913.361095] hid-generic 0003:1C4F:0002.000E: input,hidraw1: USB HID v1.10 Device [SIGMACHIP USB Keyboard] on usb-0000:00:1d.0-1.3/input1
[4558251.034024] Process accounting resumed
[4615396.191090] r8169 0000:03:00.0 eth1: Link is Down
[4615397.856950] r8169 0000:03:00.0 eth1: Link is Up – 100Mbps/Full – flow control rx/tx
[4644650.095723] Process accounting resumed

Thanksfully using below few lines of shell or perl scripts the dmesg -T  functionality could be added to the system , so you can easily get the proper timestamp out of the obscure default generated timestamp in the same manner as on newer distros.

Here is how to do with it with bash script:

#!/bin/sh paste in .bashrc and use dmesgt to get human readable timestamp
dmesg_with_human_timestamps () {
    FORMAT="%a %b %d %H:%M:%S %Y"

 

    now=$(date +%s)
    cputime_line=$(grep -m1 "\.clock" /proc/sched_debug)

    if [[ $cputime_line =~ [^0-9]*([0-9]*).* ]]; then
        cputime=$((BASH_REMATCH[1] / 1000))
    fi

    dmesg | while IFS= read -r line; do
        if [[ $line =~ ^\[\ *([0-9]+)\.[0-9]+\]\ (.*) ]]; then
            stamp=$((now-cputime+BASH_REMATCH[1]))
            echo "[$(date +”${FORMAT}” –date=@${stamp})] ${BASH_REMATCH[2]}"
        else
            echo "$line"
        fi
    done
}

Copy the script somewhere under lets say /usr/local/bin or wherever you like on the server and add into your HOME ~/.bashrc some alias like:
 

alias dmesgt=dmesg_with_timestamp.sh


You can get a copy dmesg_with_timestamp.sh of the script from here

Or you can use below few lines perl script to get the proper dmeg kernel date / time

 

#!/bin/perl
# on old Linux distros CentOS 6.0 etc. with dmesg (part of util-linux-ng-2.17.2-12.28.el6_9.2.x86_64) etc. dmesg -T not available
# workaround is little pl script below
dmesg_with_human_timestamps () {
    $(type -P dmesg) "$@" | perl -w -e 'use strict;
        my ($uptime) = do { local @ARGV="/proc/uptime";<>}; ($uptime) = ($uptime =~ /^(\d+)\./);
        foreach my $line (<>) {
            printf( ($line=~/^\[\s*(\d+)\.\d+\](.+)/) ? ( “[%s]%s\n", scalar localtime(time – $uptime + $1), $2 ) : $line )
        }'
}


Again to make use of the script put it under /usr/local/bin/check_dmesg_timestamp.pl

alias dmesgt=dmesg_with_human_timestamps

root@web-server:~# dmesgt | tail -n 20

[Sun Jun 13 15:51:49 2021] usb 2-1.3: USB disconnect, device number 9
[Sun Jun 13 15:51:50 2021] usb 2-1.3: new low-speed USB device number 10 using ehci-pci
[Sun Jun 13 15:51:50 2021] usb 2-1.3: New USB device found, idVendor=1c4f, idProduct=0002, bcdDevice= 1.10
[Sun Jun 13 15:51:50 2021] usb 2-1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[Sun Jun 13 15:51:50 2021] usb 2-1.3: Product: USB Keyboard
[Sun Jun 13 15:51:50 2021] usb 2-1.3: Manufacturer: SIGMACHIP
[Sun Jun 13 15:51:50 2021] input: SIGMACHIP USB Keyboard as /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.3/2-1.3:1.0/0003:1C4F:0002.000D/input/input25
[Sun Jun 13 15:51:50 2021] hid-generic 0003:1C4F:0002.000D: input,hidraw0: USB HID v1.10 Keyboard [SIGMACHIP USB Keyboard] on usb-0000:00:1d.0-1.3/input0
[Sun Jun 13 15:51:50 2021] input: SIGMACHIP USB Keyboard Consumer Control as /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.3/2-1.3:1.1/0003:1C4F:0002.000E/input/input26
[Sun Jun 13 15:51:50 2021] input: SIGMACHIP USB Keyboard System Control as /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.3/2-1.3:1.1/0003:1C4F:0002.000E/input/input27
[Sun Jun 13 15:51:50 2021] hid-generic 0003:1C4F:0002.000E: input,hidraw1: USB HID v1.10 Device [SIGMACHIP USB Keyboard] on usb-0000:00:1d.0-1.3/input1
[Mon Jun 14 06:24:08 2021] Process accounting resumed
[Mon Jun 14 22:16:33 2021] r8169 0000:03:00.0 eth1: Link is Down
[Mon Jun 14 22:16:34 2021] r8169 0000:03:00.0 eth1: Link is Up – 100Mbps/Full – flow control rx/tx

Squid Proxy log timestamp human readable / Convert and beautify Proxy unixtime logs in human-readable form howto

Thursday, February 21st, 2019

ccze-squid-access-log-colorized-with-log-analizer-linux-tool

If you have installed Squid Cache Proxy recently and you need to watch who is accessing the proxy and what Internet (website is viewed) under /var/log/squid/access.log /var/log/store.log /var/log/access.log etc., you will be unpleasently surprised the log's records are logged in a weird human unreadable format called UTC as Squid Proxy server does not store the date / year / hour time information in a human readable format.

Squid uses the format:
<UNIX timestamp>.<Centiseconds> and you have to be a robot of a kind or a math genious to read it 🙂

To display Squid Proxy log in a human readable, luckily you can use below one-liner  regular expression.
 

cat access.log | perl -p -e 's/^([0-9]*)/”[“.localtime($1).”]"/e'


If you have to review squid logs multiple times and on a regular basis you can either set some kind of cmd alias in $HOME/.bashrc such as:

alias readproxylog='cat access.log | perl -p -e 's/^([0-9]*)/”[“.localtime($1).”]"/e'

Or for those who prefer beauty install and use a log beatifier / colorizer such as ccze
 

root@pcfreak:/home/hipo# apt-cache show ccze|grep -i desc -A 3
Description-en: robust, modular log coloriser
 CCZE is a robust and modular log coloriser, with plugins for apm,
 exim, fetchmail, httpd, postfix, procmail, squid, syslog, ulogd,
 vsftpd, xferlog and more.

Description-md5: 55cd93dbcf614712a4d89cb3489414f6
Homepage: https://github.com/madhouse/ccze
Tag: devel::prettyprint, implemented-in::c, interface::commandline,
 role::program, scope::utility, use::checking, use::filtering,

root@pcfreak:/home/hipo# apt-get install –yes ccze

 

tail -f /var/log/squid/access.loc | ccze -CA


ccze is really nice to view /var/log/syslog errors and make your daily sysadmin life a bit more colorful

 

tail -f -n 200 /var/log/messages | ccze


tail-ccze-syslog-screenshot viewing in Colors your Linux logs

For a frequent tail + ccze usage with ccze you can add to ~/.bashrc following shell small function
 

tailc () { tail $@ | ccze -A }


Below is a list of supported syntax highlighting colorizer:

$ ccze -l
Available plugins:

Name      | Type    | Description
————————————————————
apm       | Partial | Coloriser for APM sub-logs.
distcc    | Full    | Coloriser for distcc(1) logs.
dpkg      | Full    | Coloriser for dpkg logs.
exim      | Full    | Coloriser for exim logs.
fetchmail | Partial | Coloriser for fetchmail(1) sub-logs.
ftpstats  | Full    | Coloriser for ftpstats (pure-ftpd) logs.
httpd     | Full    | Coloriser for generic HTTPD access and error logs.
icecast   | Full    | Coloriser for Icecast(8) logs.
oops      | Full    | Coloriser for oops proxy logs.
php       | Full    | Coloriser for PHP logs.
postfix   | Partial | Coloriser for postfix(1) sub-logs.
procmail  | Full    | Coloriser for procmail(1) logs.
proftpd   | Full    | Coloriser for proftpd access and auth logs.
squid     | Full    | Coloriser for squid access, store and cache logs.
sulog     | Full    | Coloriser for su(1) logs.
super     | Full    | Coloriser for super(1) logs.
syslog    | Full    | Generic syslog(8) log coloriser.
ulogd     | Partial | Coloriser for ulogd sub-logs.
vsftpd    | Full    | Coloriser for vsftpd(8) logs.
xferlog   | Full    | Generic xferlog coloriser.

At many cases for sysadmins like me that prefer clarity over obscurity, even a better solution is to just change in /etc/squid/squid.conf
the logging to turn it in human-readable form
, to do so add to config somewhere:

 

Logformat squid %tl.%03tu %6tr %>a %Ss/%03Hs %


You will get log output in format like:

 

18/Feb/2019:18:38:47 +0200.538 4787 y.y.y.y TCP_MISS/200 41841 GET https://google.com – DIRECT/x.x.x.x text/html


SQUID's format recognized parameters in above example are as follows:

 

%    a literal % character
>a    Client source IP address
>A    Client FQDN
>p    Client source port
la    Local IP address (http_port)
lp    Local port number (http_port)
sn    Unique sequence number per log line entry
ts    Seconds since epoch
tu    subsecond time (milliseconds)
tl    Local time. Optional strftime format argument
default %d/%b/%Y:%H:%M:%S %z
tg    GMT time. Optional strftime format argument
default %d/%b/%Y:%H:%M:%S %z
tr    Response time (milliseconds)
dt    Total time spent making DNS lookups (milliseconds)

 

Books on Human Resource Management

Friday, September 18th, 2009

Here are two books on Human Resource Management Human Resource Management (MGT – 501)
Strategic Human Resource Management View
Strategic Human Resources Management – Aligning with the Mission END—–

A funny caricature representing our human ways

Thursday, March 12th, 2009

A friend of mine (Necrolea) had just sent me a funny drawingrepresenting our human major tendencies in life. here Here it is END—–

HRQM Games

Sunday, September 23rd, 2007

The last 3 days in the college we used to have a guest from the Arnhem Business Scholl: Job Thinke.He is a teacher in HRQM (Human Resource and Quality Management). We played a simulator. We choose abusiness to be in. Job, Bozhidar, and two graduating students from Arnhem ( one black girl and one guy from South Africa )were the teachers. We choose an industry (our group was some Open cars producing company). Every peson of the teamhad to participate in taking decisions and writting down the decisions in a decision forms every form was aboutmoney we would invest in things which would lead to accomplish a specific goals we had setted in the beginning of the game,things like hiring more employees promoting some of them etc. etc. were our work. The game was played 3 days and it waslike if we had run a 2 years long business. Our group mistake was that we always thought about things in short terms.In the end of the 2nd year of the game we had a discussion meetings with our higher level employees and it turns outthat we are the poorest team because we always thought about the company in short term instead of long term.During this 3 days there was also some funny games for developing concentration and thinking and improvingcommunication skills. In general I think the whole think was a great loss of time, but I’m a strange person …Most of the ppl liked involved from IBMS and Hotel Management liked the game very much.END—–