multiple times Archives - ☩ Walking in Light with Christ - Faith, Computing, Diary ☩ Walking in Light with Christ

Posts Tagged ‘multiple times’

Debugging routing and network issues on Linux common approaches. A step by step guide to find out why routing or network service fails

Thursday, November 30th, 2023

For system administrators having a Network issue is among the Hell-ish stuff that can happen every now and then. That is especially true in Heterogenous / Hybrid and complicated Network topologies (with missing well crafted documentation), that were build without an initial overview "on the fly".
Such a networking connectivity or routing issues are faced by every novice, mid or even expert system administrators as the Company's Network IT environments are becoming more and more complicated day by day.

When the "Disaster" of being unable to connect two servers or at times home laptops / PCs to see each other even though on the Physical layer / Transport Layer (Hardware such as external Switches / Routers / Repeaters / Cabling etc.) is Present machines are connected and everything on the 1 Physical Layer from OSI layears is present happens, then it is time to Debug it with some software tools and methods.

To each operating system the tools and methods to test networking connection and routings is a bit different but generally speaking most concepts are pretty much the same across different types of operating systems (Linux ditros / OpenBSD / FreeBSD / Mac OS / Android / iOS / HP-UX / IBM AIX / DOS / Windows etc.).

Debugging network issues across separate operating systems has its variations but in this specific (ideas) are much close to this article. As the goal at that guide will be to point out how to debug network issues on Linux, in future if I have the time or need to debug other OS-es from Linux, I'll try to put an article on how to debug Network issues on Windows when have some time to do it.

Consider to look for the issue following the basic TCP / IP OSI Level model, every system administrator should have idea about it already, it is part of most basic networking courses such as Cisco's CCNA

1. Check what is the Link status of the Interface with ethtool

root@freak:~# ethtool eno1
Settings for eno1:
   Supported ports: [ TP ]
   Supported link modes: 10baseT/Half 10baseT/Full
   100baseT/Half 100baseT/Full
   1000baseT/Full
   Supported pause frame use: Symmetric
   Supports auto-negotiation: Yes
   Supported FEC modes: Not reported
   Advertised link modes: 10baseT/Half 10baseT/Full
   100baseT/Half 100baseT/Full
   1000baseT/Full
   Advertised pause frame use: Symmetric
   Advertised auto-negotiation: Yes
   Advertised FEC modes: Not reported
   Speed: 100Mb/s
   Duplex: Full
   Auto-negotiation: on
   Port: Twisted Pair
   PHYAD: 1
   Transceiver: internal
   MDI-X: on (auto)
   Supports Wake-on: pumbg
   Wake-on: g
Current message level: 0x00000007 (7)
drv probe link
   Link detected: yes

root@freak:~# ethtool eno2
Settings for eno2:
   Supported ports: [ TP ]
   Supported link modes: 10baseT/Half 10baseT/Full
   100baseT/Half 100baseT/Full
   1000baseT/Full
   Supported pause frame use: Symmetric
   Supports auto-negotiation: Yes
   Supported FEC modes: Not reported
   Advertised link modes: 10baseT/Half 10baseT/Full
   100baseT/Half 100baseT/Full
   1000baseT/Full
   Advertised pause frame use: Symmetric
   Advertised auto-negotiation: Yes
   Advertised FEC modes: Not reported
   Speed: 1000Mb/s
   Duplex: Full
   Auto-negotiation: on
   Port: Twisted Pair
   PHYAD: 1
   Transceiver: internal
   MDI-X: on (auto)
   Supports Wake-on: pumbg
   Wake-on: g
Current message level: 0x00000007 (7)
drv probe link
   Link detected: yes

For example lets check only if Cable of Network card is plugged in and detected to have a network connection to remote node or switch and show the connection speed on which the 'autoneg on' (autonegiation option) of the LAN card has detected the network exat maximum speed:

root@pcfreak:~# ethtool eth0|grep -i 'link detected'; ethtool eth0 |grep 'Speed: '
Link detected: yes
Speed: 100Mb/s

1. Check ip command network configuration output

root@freak:~# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master xenbr0 state UP group default qlen 1000
link/ether 70:e2:84:13:44:15 brd ff:ff:ff:ff:ff:ff
altname enp7s0
3: eno2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master xenbr1 state UP group default qlen 1000
link/ether 70:e2:84:13:44:17 brd ff:ff:ff:ff:ff:ff
altname enp8s0
4: xenbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 70:e2:84:13:44:13 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.7/24 brd 192.168.1.255 scope global dynamic xenbr0
valid_lft 7361188sec preferred_lft 7361188sec
5: xenbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 70:e2:84:13:44:15 brd ff:ff:ff:ff:ff:ff
inet 192.168.0.5/24 brd 192.168.0.255 scope global dynamic xenbr1
valid_lft 536138sec preferred_lft 536138sec
10: vif2.0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master xenbr0 state UP group default qlen 2000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
11: vif2.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master xenbr1 state UP group default qlen 2000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
12: vif3.0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master xenbr0 state UP group default qlen 2000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
13: vif3.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master xenbr1 state UP group default qlen 2000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
14: vif4.0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master xenbr0 state UP group default qlen 2000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
15: vif4.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master xenbr1 state UP group default qlen 2000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
16: vif5.0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master xenbr0 state UP group default qlen 2000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
17: vif5.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master xenbr1 state UP group default qlen 2000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
18: vif6.0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master xenbr0 state UP group default qlen 2000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
19: vif6.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master xenbr0 state UP group default qlen 2000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
30: vif17.0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master xenbr0 state UP group default qlen 2000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
31: vif17.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master xenbr1 state UP group default qlen 2000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
34: vif21.0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master xenbr0 state UP group default qlen 2000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
35: vif21.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master xenbr1 state UP group default qlen 2000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
48: vif25.0-emu: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master xenbr0 state UNKNOWN group default qlen 1000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
49: vif25.1-emu: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master xenbr1 state UNKNOWN group default qlen 1000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
50: vif25.0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master xenbr0 state DOWN group default qlen 2000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
51: vif25.1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master xenbr1 state DOWN group default qlen 2000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
118: vif47.0-emu: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master xenbr0 state UNKNOWN group default qlen 1000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
119: vif47.1-emu: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master xenbr1 state UNKNOWN group default qlen 1000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
120: vif47.0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master xenbr0 state DOWN group default qlen 2000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
121: vif47.1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master xenbr1 state DOWN group default qlen 2000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
root@freak:~#

ip a s (is a also a shortcut command alias) you can enjoy if you have to deal with ip command frequently.

2. Check the status of the interfaces

Old fashioned way is to just do:

/sbin/ifconfig

root@freak:~# ifconfig
eno1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 70:e2:84:13:44:15 txqueuelen 1000 (Ethernet)
RX packets 52366502 bytes 10622469320 (9.8 GiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 242622195 bytes 274688121244 (255.8 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device memory 0xfb200000-fb27ffff

eno2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 70:e2:84:13:44:17 txqueuelen 1000 (Ethernet)
RX packets 220995454 bytes 269698276095 (251.1 GiB)
RX errors 0 dropped 7 overruns 0 frame 0
TX packets 192319925 bytes 166233773782 (154.8 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device memory 0xfb100000-fb17ffff

lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
loop txqueuelen 1000 (Local Loopback)
RX packets 2553 bytes 147410 (143.9 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 2553 bytes 147410 (143.9 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

vif17.0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 14517375 bytes 133226551792 (124.0 GiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 139688950 bytes 145111993017 (135.1 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

vif17.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 86113294 bytes 156944058681 (146.1 GiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 181513904 bytes 267892940821 (249.4 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

vif2.0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 1521875 bytes 88282472 (84.1 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 152691174 bytes 278372314505 (259.2 GiB)
TX errors 0 dropped 3 overruns 0 carrier 0 collisions 0

vif2.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 454915 bytes 81069760 (77.3 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 266953989 bytes 425692364876 (396.4 GiB)
TX errors 0 dropped 26 overruns 0 carrier 0 collisions 0

vif21.0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 20043711 bytes 1283926794 (1.1 GiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 141580485 bytes 277396881113 (258.3 GiB)
TX errors 0 dropped 3 overruns 0 carrier 0 collisions 0

vif21.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 73004 bytes 3802174 (3.6 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 267151006 bytes 425621892663 (396.3 GiB)
TX errors 0 dropped 14 overruns 0 carrier 0 collisions 0

vif25.0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

vif25.1: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

vif25.0-emu: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 1000 (Ethernet)
RX packets 2736348 bytes 295661367 (281.9 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 260385509 bytes 265751226663 (247.5 GiB)
TX errors 0 dropped 200 overruns 0 carrier 0 collisions 0

vif25.1-emu: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 1000 (Ethernet)
RX packets 145387 bytes 36011655 (34.3 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 370314760 bytes 394725961081 (367.6 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

vif3.0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 55382861 bytes 130042280927 (121.1 GiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 99040097 bytes 147929196318 (137.7 GiB)
TX errors 0 dropped 1 overruns 0 carrier 0 collisions 0

vif3.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 5132631 bytes 295493762 (281.8 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 262314199 bytes 425416945203 (396.2 GiB)
TX errors 0 dropped 16 overruns 0 carrier 0 collisions 0

vif4.0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 4902015 bytes 615387539 (586.8 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 149342891 bytes 277802504143 (258.7 GiB)
TX errors 0 dropped 1 overruns 0 carrier 0 collisions 0

vif4.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 276927 bytes 30720101 (29.2 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 267132395 bytes 425745668273 (396.5 GiB)
TX errors 0 dropped 14 overruns 0 carrier 0 collisions 0

vif47.0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

vif47.1: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

vif47.0-emu: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 1000 (Ethernet)
RX packets 208745 bytes 20096596 (19.1 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 110905731 bytes 110723486135 (103.1 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

vif47.1-emu: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 1000 (Ethernet)
RX packets 140517 bytes 14596061 (13.9 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 150831959 bytes 162931572456 (151.7 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

vif5.0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 2030528 bytes 363988589 (347.1 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 152264264 bytes 278131541781 (259.0 GiB)
TX errors 0 dropped 1 overruns 0 carrier 0 collisions 0

vif5.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 4169244 bytes 1045889687 (997.4 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 263561100 bytes 424894400987 (395.7 GiB)
TX errors 0 dropped 7 overruns 0 carrier 0 collisions 0

vif6.0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 300242 bytes 16210963 (15.4 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 153909576 bytes 278461295620 (259.3 GiB)
TX errors 0 dropped 2 overruns 0 carrier 0 collisions 0

vif6.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 43 bytes 1932 (1.8 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 154205631 bytes 278481298141 (259.3 GiB)
TX errors 0 dropped 2 overruns 0 carrier 0 collisions 0

xenbr0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.1.8 netmask 255.255.255.0 broadcast 192.168.1.255
ether 70:e2:84:13:44:11 txqueuelen 1000 (Ethernet)
RX packets 13689902 bytes 923464162 (880.6 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 12072932 bytes 1307055530 (1.2 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

xenbr1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.0.3 netmask 255.255.255.0 broadcast 192.168.0.255
ether 70:e2:84:13:44:12 txqueuelen 1000 (Ethernet)
RX packets 626995 bytes 180026901 (171.6 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 12815 bytes 942092 (920.0 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

root@freak:~# ifconfig
eno1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 70:e2:84:13:44:11 txqueuelen 1000 (Ethernet)
RX packets 52373358 bytes 10623034427 (9.8 GiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 242660000 bytes 274734018669 (255.8 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device memory 0xfb200000-fb27ffff

eno2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 70:e2:84:13:44:12 txqueuelen 1000 (Ethernet)
RX packets 221197892 bytes 269978137472 (251.4 GiB)
RX errors 0 dropped 7 overruns 0 frame 0
TX packets 192573206 bytes 166491370299 (155.0 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device memory 0xfb100000-fb17ffff

lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
loop txqueuelen 1000 (Local Loopback)
RX packets 2553 bytes 147410 (143.9 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 2553 bytes 147410 (143.9 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

vif17.0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 14519247 bytes 133248290251 (124.0 GiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 139708738 bytes 145135168676 (135.1 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

vif17.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 86206104 bytes 157189755115 (146.3 GiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 181685983 bytes 268170806613 (249.7 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

vif2.0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 1522072 bytes 88293701 (84.2 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 152712638 bytes 278417240910 (259.2 GiB)
TX errors 0 dropped 3 overruns 0 carrier 0 collisions 0

vif2.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 454933 bytes 81071616 (77.3 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 267218860 bytes 426217224334 (396.9 GiB)
TX errors 0 dropped 26 overruns 0 carrier 0 collisions 0

vif21.0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 20045530 bytes 1284038375 (1.1 GiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 141601066 bytes 277441739746 (258.3 GiB)
TX errors 0 dropped 3 overruns 0 carrier 0 collisions 0

vif21.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 73010 bytes 3802474 (3.6 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 267415889 bytes 426146753845 (396.8 GiB)
TX errors 0 dropped 14 overruns 0 carrier 0 collisions 0

vif25.0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

vif25.1: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

vif25.0-emu: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 1000 (Ethernet)
RX packets 2736576 bytes 295678097 (281.9 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 260429831 bytes 265797660906 (247.5 GiB)
TX errors 0 dropped 200 overruns 0 carrier 0 collisions 0

vif25.1-emu: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 1000 (Ethernet)
RX packets 145425 bytes 36018716 (34.3 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 370770440 bytes 395263409640 (368.1 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

vif3.0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 55392503 bytes 130064444520 (121.1 GiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 99052116 bytes 147951838129 (137.7 GiB)
TX errors 0 dropped 1 overruns 0 carrier 0 collisions 0

vif3.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 5133054 bytes 295517366 (281.8 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 262578665 bytes 425941777243 (396.6 GiB)
TX errors 0 dropped 16 overruns 0 carrier 0 collisions 0

vif4.0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 4902949 bytes 615496460 (586.9 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 149363618 bytes 277847322538 (258.7 GiB)
TX errors 0 dropped 1 overruns 0 carrier 0 collisions 0

vif4.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 276943 bytes 30721141 (29.2 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 267397268 bytes 426270528575 (396.9 GiB)
TX errors 0 dropped 14 overruns 0 carrier 0 collisions 0

vif47.0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

vif47.1: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

vif47.0-emu: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 1000 (Ethernet)
RX packets 208790 bytes 20100733 (19.1 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 110950236 bytes 110769932971 (103.1 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

vif47.1-emu: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 1000 (Ethernet)
RX packets 140551 bytes 14599509 (13.9 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 151287643 bytes 163469024604 (152.2 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

vif5.0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 2030676 bytes 363997181 (347.1 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 152285777 bytes 278176471509 (259.0 GiB)
TX errors 0 dropped 1 overruns 0 carrier 0 collisions 0

vif5.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 4169387 bytes 1045898303 (997.4 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 263825846 bytes 425419251935 (396.2 GiB)
TX errors 0 dropped 7 overruns 0 carrier 0 collisions 0

vif6.0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 300266 bytes 16212271 (15.4 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 153931212 bytes 278506234302 (259.3 GiB)
TX errors 0 dropped 2 overruns 0 carrier 0 collisions 0

vif6.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 43 bytes 1932 (1.8 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 154227291 bytes 278526238467 (259.3 GiB)
TX errors 0 dropped 2 overruns 0 carrier 0 collisions 0

xenbr0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.1.8 netmask 255.255.255.0 broadcast 192.168.1.255
ether 70:e2:84:13:44:11 txqueuelen 1000 (Ethernet)
RX packets 13690768 bytes 923520126 (880.7 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 12073667 bytes 1307127765 (1.2 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

xenbr1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.0.3 netmask 255.255.255.0 broadcast 192.168.0.255
ether 70:e2:84:13:44:12 txqueuelen 1000 (Ethernet)
RX packets 627010 bytes 180028847 (171.6 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 12815 bytes 942092 (920.0 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

To see ethernet interfaces that seem up and then do a ifconfig -a to check whether some interfaces are down (e.g. not shown in the simple ifconfig list).
/sbin/ifconfig -a

! Please note that some virtual IP configurations might not appear and noly be visible in an (ip addr show) command.

3. Check iproute2 for special rt_tables (Routing Tables) rules

By default Linux distributions does not have any additional rules in /etc/iproute2/rt_tables however some Linux router machines, needs to have a multiple Gateways. Perhaps the most elegant way to do multiple routings with Linux is to use iproute2's routing tables rt_tables.

Here is example of an OpenXEN system that has 2 Internet providers attached and routes different traffic via

root@freak:~# cat /etc/iproute2/rt_tables
#
# reserved values
#
255   local
254   main
253   default

100   INET1
200 INET2
0   unspec
#
# local
#
#1   inr.ruhep

root@freak:~# ip rule list
0:   from all lookup local
32762:   from all to 192.168.1.8 lookup INET2
32763:   from 192.168.1.8 lookup INET2
32764:   from all to 192.168.0.3 lookup INET1
32765:   from 192.168.0.3 lookup INET1
32766:   from all lookup main
32767:   from all lookup default
root@freak:~#

4. Using ip route get to find out traffic route (path)

root@freak:~# ip route get 192.168.0.1
192.168.0.1 via 192.168.0.1 dev xenbr1 src 192.168.0.3 uid 0
cache

root@freak:~# /sbin/route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.1.1 0.0.0.0 UG 0 0 0 xenbr0
192.168.0.0 192.168.0.1 255.255.255.0 UG 0 0 0 xenbr1
192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 xenbr1
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 xenbr0
root@freak:~#

root@freak:~# ip route show
default via 192.168.1.1 dev xenbr0
192.168.0.0/24 via 192.168.0.1 dev xenbr1
192.168.0.0/24 dev xenbr1 proto kernel scope link src 192.168.0.3
192.168.1.0/24 dev xenbr0 proto kernel scope link src 192.168.1.8

If you find that gateway is missing you might want to add it with:

root@freak:~# ip route add default via 192.168.5.1

If you need to add a speicic network IP range via separate gateways, you can use commands like:

To add routing for 192.168.0.1/24 / 192.168.1.1/24 via 192.168.0.1 and 192.168.1.1

# /sbin/route add -net 192.168.1.0 netmask 255.255.255.0 gw 192.168.1.1 dev eth1
# /sbin/route add -net 192.168.0.0 netmask 255.255.255.0 gw 192.168.0.1 dev eth1

If you need to delete a configured wrong route with ip command

# ip route del 192.168.1.0/24 via 0.0.0.0 dev eth1
# ip route del 192.168.0.0/24 via 0.0.0.0 dev eth1

5. Use ping (ICMP protocol) the Destionation IP

root@freak:~# ping -c 3 192.168.0.1
PING 192.168.0.1 (192.168.0.1) 56(84) bytes of data.
64 bytes from 192.168.0.1: icmp_seq=1 ttl=64 time=0.219 ms
64 bytes from 192.168.0.1: icmp_seq=2 ttl=64 time=0.295 ms
64 bytes from 192.168.0.1: icmp_seq=3 ttl=64 time=0.270 ms

— 192.168.0.1 ping statistics —
3 packets transmitted, 3 received, 0% packet loss, time 2048ms
rtt min/avg/max/mdev = 0.219/0.261/0.295/0.031 ms
root@freak:~# ping -c 3 192.168.0.39
PING 192.168.0.39 (192.168.0.39) 56(84) bytes of data.
From 192.168.1.80: icmp_seq=2 Redirect Host(New nexthop: 192.168.0.39)
From 192.168.1.80: icmp_seq=3 Redirect Host(New nexthop: 192.168.0.39)
From 192.168.1.80 icmp_seq=1 Destination Host Unreachable

— 192.168.0.39 ping statistics —
3 packets transmitted, 0 received, +1 errors, 100% packet loss, time 2039ms
pipe 3

Note that sometimes you might get 100% traffic loss but still have connection to the destionation in case if the ICMP protocol is filtered for security.

However if you get something like Network is unreachable that is usually an indicator of some routing problem or wrongly configured network netmask.

root@freak:~# ping 192.168.0.5
ping: connect: Network is unreachable

Test network with different packet size. To send 8972 bytes of payload in a Ethernet frame without fragmentation, the following command can be used:

root@pcfreak:~# ping -s 8972 -M do -c 4 freak
PING xen (192.168.1.8) 8972(9000) bytes of data.
ping: local error: message too long, mtu=1500
ping: local error: message too long, mtu=1500
ping: local error: message too long, mtu=1500
^C
— xen ping statistics —
3 packets transmitted, 0 received, +3 errors, 100% packet loss, time 2037ms

root@pcfreak:~#

-M pmtudisc_opt
Select Path MTU Discovery strategy. pmtudisc_option may be either do (prohibit fragmentation, even local one), want (do PMTU discovery, fragment locally when packet size is
large), or dont (do not set DF flag).

root@pcfreak:~# ping -s 8972 -M want -c 4 freak
PING xen (192.168.1.8) 8972(9000) bytes of data.
8980 bytes from xen (192.168.1.5): icmp_seq=1 ttl=64 time=2.18 ms
8980 bytes from xen (192.168.1.5): icmp_seq=2 ttl=64 time=1.90 ms
8980 bytes from xen (192.168.1.5): icmp_seq=3 ttl=64 time=2.10 ms
^C
— xen ping statistics —
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 1.901/2.059/2.178/0.116 ms
root@pcfreak:~#

-M do: prohibit fragmentation
-s 8972 8972 bytes of data
ICMP header: 8 bytes
IP header: 20 bytes (usually, it can be higher)
8980 bytes of bytes is the IP payload

These commands can be used to capture for MTU (maximum transmition units) related issues between hosts that are preventing for hosts to properly send traffic between themselves.
A common issue for Linux hosts to be unable to see each other on the same network is caused by Jumbo Frames (MTU 9000) packets enabled on one of the sides and MTU of 1500 on the other side.
Thus it is always a good idea to thoroughully look up all configured MTUs for all LAN Devices on each server.

6. Check traceroute path to host

If there is no PING but ip route get shows routing is properly configured and the routes existing in the Linux machine routing tables, next step is to check the output of traceroute / tracepath / mtr

raceroute to 192.168.0.1 (192.168.0.1), 30 hops max, 60 byte packets
1 pcfreak (192.168.0.1) 0.263 ms 0.166 ms 0.119 ms
root@freak:~# tracepath 192.168.1.1
1?: [LOCALHOST] pmtu 1500
1: vivacom-gigabit-router 0.925ms reached
1: vivacom-gigabit-router 0.835ms reached
Resume: pmtu 1500 hops 1 back 1

It might be useful to get a frequent output of the command (especially on Linux hosts) where mtr command is not installed with:

root@freak:~# watch -n 0.1 traceroute 192.168.0.1

root@freak:~# traceroute -4 google.com
traceroute to google.com (172.217.17.110), 30 hops max, 60 byte packets
1 vivacom-gigabit-router (192.168.1.1) 0.657 ms 1.280 ms 1.647 ms
2 213.91.190.130 (213.91.190.130) 7.983 ms 8.168 ms 8.097 ms
3 * * *
4 * * *
5 212-39-66-222.ip.btc-net.bg (212.39.66.222) 16.613 ms 16.336 ms 17.151 ms
6 * * *
7 142.251.92.65 (142.251.92.65) 18.808 ms 13.246 ms 209.85.254.242 (209.85.254.242) 15.541 ms
8 142.251.92.3 (142.251.92.3) 14.223 ms 142.251.227.251 (142.251.227.251) 14.507 ms 142.251.92.3 (142.251.92.3) 15.328 ms
9 ams15s29-in-f14.1e100.net (172.217.17.110) 14.097 ms 14.909 ms 142.251.242.230 (142.251.242.230) 13.481 ms
root@freak:~#

If you have MTR then you can get plenty of useful additional information such as the Network HOP name or the Country location of the HOP.

To get HOP name:

root@freak:~# mtr -z google.com

To get info on where (which Country) exactly network HOP is located physically:

root@freak:~# mtr -y 2 google.com

7. Check iptables INPUT / FORWARD / OUTPUT rules are messing with something

# iptables -L -n
…

# iptables -t nat -L -n
…

Ideally you would not have any firewall

# iptables -L -n

Chain INPUT (policy ACCEPT)
target prot opt source destination

Chain FORWARD (policy ACCEPT)
target prot opt source destination

Chain OUTPUT (policy ACCEPT)
target prot opt source destination

# iptables -t nat -L -n
Chain PREROUTING (policy ACCEPT)
target prot opt source destination

Chain INPUT (policy ACCEPT)
target prot opt source destination

Chain OUTPUT (policy ACCEPT)
target prot opt source destination

Chain POSTROUTING (policy ACCEPT)
target prot opt source destination

In case if something like firewalld is enabled as a default serviceto provide some modern Linux firewall as Ubuntu and Redhat / CentOS / Fedoras has it often turned on as a service stop and disable the service

# systemctl stop firewalld

# systemctl disable firewalld

8. Debug for any possible MAC address duplicates

root@pcfrxen:~# arp -an
? (192.168.1.33) at 00:16:3e:59:96:9e [ether] on eth0
? (192.168.1.1) at 18:45:93:c6:d8:00 [ether] on eth1
? (192.168.0.1) at 8c:89:a5:f2:e8:d9 [ether] on eth1
? (192.168.1.1) at 18:45:93:c6:d8:00 [ether] on eth0
? (192.168.1.11) at 7c:0a:3f:89:b6:fa [ether] on eth1
? (192.168.1.17) at <incomplete> on eth0
? (192.168.1.37) at 00:16:3e:ea:05:ce [ether] on eth0
? (192.168.1.80) at 8c:89:a5:f2:e7:d8 [ether] on eth0
? (192.168.1.11) at 7c:0a:3f:89:a5:fa [ether] on eth0
? (192.168.1.30) at 00:16:3e:bb:46:45 [ether] on eth1
? (192.168.0.210) at 00:16:3e:68:d9:55 [ether] on eth1
? (192.168.1.30) at 00:16:3e:bb:46:45 [ether] on eth0
? (192.168.1.18) at 00:16:3e:0d:40:05 [ether] on eth1
? (192.168.0.211) at 00:16:3e:4d:41:05 [ether] on eth1
? (192.168.1.35) at 00:16:3e:d1:8f:77 [ether] on eth0
? (192.168.1.18) at 00:16:3e:0d:43:05 [ether] on eth0
? (192.168.1.28) at 00:16:3e:04:12:1c [ether] on eth1
? (192.168.0.3) at 70:e2:84:13:43:12 [ether] on eth1
? (192.168.0.208) at 00:16:3e:51:de:9c [ether] on eth1
? (192.168.0.241) at 00:16:3e:0d:48:06 [ether] on eth1
? (192.168.1.28) at 00:16:3e:04:12:1c [ether] on eth0
? (192.168.1.33) at 00:16:3e:59:97:8e [ether] on eth1
? (192.168.0.241) at 00:16:3e:0d:45:06 [ether] on eth0
? (192.168.0.209) at 00:16:3e:5c:df:96 [ether] on eth1

root@pcfrxen:~# ip neigh show
192.168.1.33 dev eth0 lladdr 00:16:3e:59:96:9e REACHABLE
192.168.1.1 dev eth1 lladdr 18:45:93:c6:d8:00 STALE
192.168.0.1 dev eth1 lladdr 8c:89:a5:f2:e8:d9 REACHABLE
192.168.1.1 dev eth0 lladdr 18:45:93:c6:d9:01 REACHABLE
192.168.1.11 dev eth1 lladdr 7c:0a:3f:89:a6:fb STALE
192.168.1.17 dev eth0 FAILED
192.168.1.37 dev eth0 lladdr 00:16:3e:ea:06:ce STALE
192.168.1.80 dev eth0 lladdr 8c:89:a5:f2:e8:d9 REACHABLE
192.168.1.11 dev eth0 lladdr 7c:0a:3f:89:a7:fa STALE
192.168.1.30 dev eth1 lladdr 00:16:3e:bb:45:46 STALE
192.168.0.210 dev eth1 lladdr 00:16:3e:68:d8:56 REACHABLE
192.168.1.30 dev eth0 lladdr 00:16:3e:bb:45:46 STALE
192.168.1.18 dev eth1 lladdr 00:16:3e:0d:48:04 STALE
192.168.0.211 dev eth1 lladdr 00:16:3e:4d:40:04 STALE
192.168.1.35 dev eth0 lladdr 00:16:3e:d2:8f:76 STALE
192.168.1.18 dev eth0 lladdr 00:16:3e:0d:48:06 STALE
192.168.1.28 dev eth1 lladdr 00:16:3e:04:11:2c STALE
192.168.0.3 dev eth1 lladdr 70:e2:84:13:44:13 STALE
192.168.0.208 dev eth1 lladdr 00:16:3e:51:de:9c REACHABLE
192.168.0.241 dev eth1 lladdr 00:16:3e:0d:48:07 STALE
192.168.1.28 dev eth0 lladdr 00:16:3e:04:12:1c REACHABLE
192.168.1.33 dev eth1 lladdr 00:16:3e:59:96:9e STALE
192.168.0.241 dev eth0 lladdr 00:16:3e:0d:49:06 STALE
192.168.0.209 dev eth1 lladdr 00:16:3e:5c:dd:97 STALE
root@pcfrxen:~#

9. Check out with netstat / ss for any irregularities such as high amount of error of faulty ICMP / TCP / UDP network packs

For example check out the netstat network stack output

# netstat -s

root@pcfrxen:~# netstat -s
Ip:
Forwarding: 2
440044929 total packets received
1032 with invalid addresses
0 forwarded
0 incoming packets discarded
439988902 incoming packets delivered
396161852 requests sent out
3 outgoing packets dropped
100 dropped because of missing route
Icmp:
1025 ICMP messages received
540 input ICMP message failed
ICMP input histogram:
destination unreachable: 1014
timeout in transit: 11
519 ICMP messages sent
0 ICMP messages failed
ICMP output histogram:
destination unreachable: 519
IcmpMsg:
InType3: 1014
InType11: 11
OutType3: 519
Tcp:
1077237 active connection openings
1070510 passive connection openings
1398236 failed connection attempts
111345 connection resets received
83 connections established
438293250 segments received
508143650 segments sent out
42567 segments retransmitted
546 bad segments received
329039 resets sent
Udp:
1661295 packets received
278 packets to unknown port received
0 packet receive errors
1545720 packets sent
0 receive buffer errors
0 send buffer errors
IgnoredMulti: 33046
UdpLite:
TcpExt:
1 invalid SYN cookies received
1398196 resets received for embryonic SYN_RECV sockets
1737473 packets pruned from receive queue because of socket buffer overrun
1118775 TCP sockets finished time wait in fast timer
638 time wait sockets recycled by time stamp
656 packetes rejected in established connections because of timestamp
2218959 delayed acks sent
2330 delayed acks further delayed because of locked socket
Quick ack mode was activated 7172 times
271799723 packet headers predicted
14917420 acknowledgments not containing data payload received
171078735 predicted acknowledgments
52 times recovered from packet loss due to fast retransmit
TCPSackRecovery: 337
Detected reordering 1551 times using SACK
Detected reordering 1501 times using reno fast retransmit
Detected reordering 61 times using time stamp
9 congestion windows fully recovered without slow start
38 congestion windows partially recovered using Hoe heuristic
TCPDSACKUndo: 241
104 congestion windows recovered without slow start after partial ack
TCPLostRetransmit: 11550
1 timeouts after reno fast retransmit
TCPSackFailures: 13
3772 fast retransmits
2 retransmits in slow start
TCPTimeouts: 24104
TCPLossProbes: 101748
TCPLossProbeRecovery: 134
TCPSackRecoveryFail: 3
128989224 packets collapsed in receive queue due to low socket buffer
TCPBacklogCoalesce: 715034
TCPDSACKOldSent: 7168
TCPDSACKOfoSent: 341
TCPDSACKRecv: 16612
150689 connections reset due to unexpected data
27063 connections reset due to early user close
17 connections aborted due to timeout
TCPDSACKIgnoredOld: 158
TCPDSACKIgnoredNoUndo: 13514
TCPSpuriousRTOs: 9
TCPSackMerged: 1191
TCPSackShiftFallback: 1011
TCPDeferAcceptDrop: 699473
TCPRcvCoalesce: 3311764
TCPOFOQueue: 14289375
TCPOFOMerge: 356
TCPChallengeACK: 621
TCPSYNChallenge: 621
TCPSpuriousRtxHostQueues: 4
TCPAutoCorking: 1605205
TCPFromZeroWindowAdv: 132380
TCPToZeroWindowAdv: 132441
TCPWantZeroWindowAdv: 1445495
TCPSynRetrans: 23652
TCPOrigDataSent: 388992604
TCPHystartTrainDetect: 69089
TCPHystartTrainCwnd: 3264904
TCPHystartDelayDetect: 4
TCPHystartDelayCwnd: 128
TCPACKSkippedPAWS: 3
TCPACKSkippedSeq: 2001
TCPACKSkippedChallenge: 2
TCPWinProbe: 123043
TCPKeepAlive: 4389
TCPDelivered: 389507445
TCPAckCompressed: 7343781
TcpTimeoutRehash: 23311
TcpDuplicateDataRehash: 8
TCPDSACKRecvSegs: 17335
IpExt:
InMcastPkts: 145100
OutMcastPkts: 9429
InBcastPkts: 18226
InOctets: 722933727848
OutOctets: 759502627470
InMcastOctets: 58227095
OutMcastOctets: 3284379
InBcastOctets: 1756918
InNoECTPkts: 440286946
InECT0Pkts: 936

List all listening established connections to host

# netstat -ltne

List all UDP / TCP connections

# netstat -ltua

or if you prefer to do it with the newer and more comprehensive tool ss:

List all listening TCP connections

# ss -lt

List all listening UDP connections

# ss -ua

Display statistics about recent connections

root@pcfrxen:~# ss -s
Total: 329
TCP: 896 (estab 70, closed 769, orphaned 0, timewait 767)

Transport Total IP IPv6
RAW   0 0 0
UDP   40 36 4
TCP   127 118 9
INET   167 154 13
FRAG   0 0 0

If you need to debug some specific sport or dport filter out the connection you need by port number

# ss -at '( dport = :22 or sport = :22 )'

Debug for any possible issues with ICMP unreachable but ports reachable with NMAP / telnet / Netcat

# nc 192.168.0.1 -vz

root@pcfrxen:/ # nc 192.168.0.1 80 -vz
pcfreak [192.168.0.1] 80 (http) open

root@pcfrxen:/ # nc 192.168.0.1 5555 -vz
pcfreak [192.168.0.1] 5555 (?) : Connection refused

root@pcfrxen:/# telnet 192.168.0.1 3128
Trying 192.168.0.1…
Connected to 192.168.0.1.
Escape character is '^]'.
^]
telnet> quit
Connection closed.

root@pcfrxen:/# nmap -sS -P0 192.168.0.1 -p 443 -O
Starting Nmap 7.80 ( https://nmap.org ) at 2023-11-27 19:51 EET
Nmap scan report for pcfreak (192.168.0.1)
Host is up (0.00036s latency).

PORT STATE SERVICE
443/tcp open https
MAC Address: 8C:89:A5:F2:E8:D8 (Micro-Star INT'L)
Warning: OSScan results may be unreliable because we could not find at least 1 open and 1 closed port
Aggressive OS guesses: Linux 3.11 (96%), Linux 3.1 (95%), Linux 3.2 (95%), AXIS 210A or 211 Network Camera (Linux 2.6.17) (94%), Linux 2.6.32 (94%), Linux 3.10 (94%), Linux 2.6.18 (93%), Linux 3.2 – 4.9 (93%), ASUS RT-N56U WAP (Linux 3.4) (93%), Linux 3.16 (93%)
No exact OS matches for host (test conditions non-ideal).
Network Distance: 1 hop

OS detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 6.24 seconds
root@pcfrxen:/#

10. Add static MAC address to Ethernet Interface (if you find a MAC address being wrongly assigned to interface)

Sometimes problems with network unrechability between hosts is caused by wrongly defined MAC addresses on a Switch that did not correspond correctly to the ones assigned on the Linux host.
The easiest resolution here if you don't have access to Switch in work environment is to reassign the default MAC addresses of interfaces to proper MAC addresses, expected by remote router.

root@pcfrxen:/#  /sbin/ifconfig eth2 hw ether 8c:89:a5:f2:e8:d6

root@pcfrxen:/#  /sbin/ifconfig eth1 hw ether 8c:89:a5:f2:e8:d5

root@pcfrxen:/#  ifconfig eth0|grep -i ether
ether 8c:89:a5:f2:e8:d6 txqueuelen 1000 (Ethernet)

11. Check for Network Address Translation (NAT) misconfigurations

If you do use some NAT-ing between Linux host and the remote Network Device you cannot reach, make sure IP Forwarding is enabled (i.e. /etc/sysctl.conf was not mistakenly overwritten by a script or admin for whatever reason).

root@server:~# sysctl net.ipv4.ip_forward
net.ipv4.ip_forward = 1
root@server:~# sysctl net.ipv4.conf.all.forwarding
net.ipv4.conf.all.forwarding = 1

root@server:~# sysctl net.ipv6.conf.all.forwarding
net.ipv6.conf.all.forwarding = 0

12. Check for Resolving DNS irregularities with /etc/resolv.conf

If network connectivity is okay on TCP / IP , UDP Level but problems with DNS of course, check what you have configured inside /etc/resolv.conf

And if use newer Linux distributions and have resolving managed by systemd check status of resolvectl

root@server:~# cat /etc/resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
# DO NOT EDIT THIS FILE BY HAND — YOUR CHANGES WILL BE OVERWRITTEN
# 127.0.0.53 is the systemd-resolved stub resolver.
# run "resolvectl status" to see details about the actual nameservers.

nameserver 127.0.0.1
search pc-freak.net
domain pc-freak.net
nameserver 8.8.8.8
nameserver 8.8.4.4
nameserver 109.104.195.2
nameserver 109.104.195.1
nameserver 208.67.222.222
nameserver 208.67.220.220
options timeout:2 rotate
root@pcfreak:~#

root@server:~# resolvectl status
Global
Protocols: -LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
resolv.conf mode: stub

Link 2 (ens3)
Current Scopes: DNS
Protocols: +DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
Current DNS Server: 192.168.5.1
DNS Servers: 192.168.5.1

As seen see, the systemd-resolved service is used to provide domain names resolution and we can modify its configuration file /etc/systemd/resolved.conf to add the DNS server – the following line is set (two DNS servers’ addresses are added):

For example …

DNS=8.8.8.8

13. Fix problems with wrongly configured Network Speed between hosts

It is not uncommon to have a Switch between two Linux hosts that is set to communicate on a certain maximum amount of Speed but a Linux host is set to communicate or lesser or more of Speed, this might create network issues so in such cases make sure either you use the Auto Negitionation network feature
or set both sides to be communicating on the same amount of network speed.

To turn on auto negotiation for ether interface

# ethtool -s eth1 speed 1000 duplex full autoneg on

For example to set a Linux network interface to communicate on 1 Gigabit speed and switch off autonegotiation off.

# ethtool -s eth1 speed 1000 duplex full autoneg off

14. Check arp and icmp traffic with tcpdump

On both sides where the IPs can't see each other we can run a tcpdump to check the ARP and ICMP traffic flowing between the hosts.

# tcpdump -i eth1 arp or icmp

cpdump: verbose output suppressed, use -v[v]… for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), snapshot length 262144 bytes
15:29:07.001841 IP freak-eth1 > pcfr_hware_local_ip: ICMP echo request, id 13348, seq 65, length 64
15:29:07.001887 IP pcfr_hware_local_ip > freak-eth1: ICMP echo reply, id 13348, seq 65, length 64
15:29:07.598413 ARP, Request who-has pcfr_hware_local_ip tell zabbix-server, length 46
15:29:07.598425 ARP, Reply pcfr_hware_local_ip is-at 8c:89:a5:f2:e8:d8 (oui Unknown), length 28
15:29:07.633055 ARP, Request who-has freak_vivacom_auto_assigned_dhcp_ip tell 192.168.1.1, length 46
15:29:08.025824 IP freak-eth1 > pcfr_hware_local_ip: ICMP echo request, id 13348, seq 66, length 64
15:29:08.025864 IP pcfr_hware_local_ip > freak-eth1: ICMP echo reply, id 13348, seq 66, length 64

# tcpdump -i eth1 -vvv
…

If you want to sniff for TCP protocol and specific port and look up for DATA transfered for SMTP you can use something like:

# tcpdump -nNxXi eth0 -s 0 proto TCP and port 25

If you need a bit more thorough explanation on what it would do check out my previous article How to catch / track mail server traffic abusers with tcpdump.

15. Debugging network bridge issues

Having bridge network interface is another brink where things could go totally wrong.
If you have network bridges configured, check out what is the status of the bridge.

root@freak:/etc/network# brctl show
bridge name   bridge id       STP enabled   interfaces
xenbr0       8000.70e284134411   yes       eno1
                           vif1.0
                           vif10.0
                           vif16.0
                           vif16.0-emu
                           vif2.0
                           vif3.0
                           vif4.0
                           vif5.0
                           vif6.0
                           vif9.0
                           vif9.0-emu
xenbr1       8000.70e284134412   yes       eno2
                           vif1.1
                           vif10.1
                           vif16.1
                           vif16.1-emu
                           vif2.1
                           vif3.1
                           vif4.1
                           vif5.1
                           vif6.1
                           vif9.1
                           vif9.1-emu

Check out any configurations such as /etc/sysconfig/network-scripts/ifcfg-* are not misconfigured if on Redhat / CentOS / Fedora.
Or if on Debian / Ubuntu and other deb based Linuxes look up for /etc/network/interfaces config problems that might be causing the bridge to misbehave.

For example one bridge network issue, I've experienced recently is related to bridge_ports variable configured as bridge_ports all.
This was causing the second bridge xenbr1 to be unable to see another local network that was directly connected with a cable to it.

The fix was bridge_ports none. Finding out this trivial issue caused by a restored network config from old backup took me days to debug.
As everything seemed on a network level to be perfect just like in Physical layer, same way and on Software level, routings were okay.
Checked everything multiple times and did not see anything irregular. ping was missing and hosts cannot see each other even though having the right netmask and
network configuration in place.

Below is my /etc/network/interfaces configuration with the correct bridge_ports none changed.

root@freak:/etc/network# cat /etc/network/interfaces
auto lo
iface lo inet loopback

auto eno1
allow-hotplug eno1
iface eno1 inet manual
dns-nameservers 127.0.0.1 8.8.8.8 8.8.4.4 207.67.222.222 208.67.220.220
auto eno2
allow-hotplug eno2
iface eno2 inet manual
dns-nameservers 127.0.0.1 8.8.8.8 8.8.4.4 207.67.222.222 208.67.220.220

auto xenbr0
allow-hotplug xenbr0
# Bridge setup
# fetching dhcp ip from 192.168.1.20 (vivacom fiber optics router) routing traffic via 1Gigabit network
iface xenbr0 inet dhcp
hwaddress ether 70:e2:84:13:44:11
# address 192.168.1.5/22
address 192.168.1.5
netmask 255.255.252.0
# address 192.168.1.8 if dhcp takes from vivacom dhcpd
bridge_ports eno1
gateway 192.168.1.20
bridge_stp on
bridge_waitport 0
bridge_fd 0
bridge_ports none
dns-nameservers 8.8.8.8 8.8.4.4

auto xenbr1
# fetching dhcp ip from pc-freak.net (192.168.0.1) bergon.net routing traffic through it
allow-hotplug xenbr1
iface xenbr1 inet dhcp
hwaddress ether 70:e2:84:13:44:11
## address 192.168.0.3/22
address 192.168.0.8
netmask 255.255.252.0
# address 192.168.0.8 if dhcp takes from vivacom dhcpd (currently mac deleted from vivacom router)
# address 192.168.0.9 if dhcp takes from pc-freak.net hware host
# hwaddress ether 70:e2:84:13:44:13
gateway 192.168.0.1
bridge_ports eno2
bridge_stp on
bridge_waitport 0
bridge_fd 0
bridge_ports none
dns-nameservers 8.8.8.8 8.8.4.4
root@freak:/etc/network#

root@freak:/etc/network# brctl showstp xenbr0
xenbr0
bridge id       8000.70e284134411
designated root   8000.70e284134411
root port       0           path cost       0
max age       20.00           bridge max age       20.00
hello time       2.00           bridge hello time   2.00
forward delay       15.00           bridge forward delay   15.00
ageing time       0.00
hello timer       1.31           tcn timer       0.00
topology change timer   0.00           gc timer       0.00
flags

eno1 (1)
port id       8001           state       forwarding
designated root   8000.70e284134411   path cost       19
designated bridge   8000.70e284134411   message age timer   0.00
designated port   8001           forward delay timer   0.00
designated cost   0           hold timer       0.31
flags

vif1.0 (2)
port id       8002           state       forwarding
designated root   8000.70e284134411   path cost       100
designated bridge   8000.70e284134411   message age timer   0.00
designated port   8002           forward delay timer   0.00
designated cost   0           hold timer       0.31
flags

vif10.0 (12)
port id       800c           state       forwarding
designated root   8000.70e284134411   path cost       100
designated bridge   8000.70e284134411   message age timer   0.00
designated port   800c           forward delay timer   0.00
designated cost   0           hold timer       0.31
flags

vif16.0 (13)
port id       800d           state       disabled
designated root   8000.70e284134411   path cost       100
designated bridge   8000.70e284134411   message age timer   0.00
designated port   800d           forward delay timer   0.00
designated cost   0           hold timer       0.00
flags

vif16.0-emu (14)
port id       800e           state       forwarding
designated root   8000.70e284134411   path cost       100
designated bridge   8000.70e284134411   message age timer   0.00
designated port   800e           forward delay timer   0.00
designated cost   0           hold timer       0.31
flags

vif2.0 (4)
port id       8004           state       forwarding
designated root   8000.70e284134411   path cost       100
designated bridge   8000.70e284134411   message age timer   0.00
designated port   8004           forward delay timer   0.00
designated cost   0           hold timer       0.31
flags

vif3.0 (5)
port id       8005           state       forwarding
designated root   8000.70e284134411   path cost       100
designated bridge   8000.70e284134411   message age timer   0.00
designated port   8005           forward delay timer   0.00
designated cost   0           hold timer       0.31
flags

vif4.0 (3)
port id       8003           state       forwarding
designated root   8000.70e284134411   path cost       100
designated bridge   8000.70e284134411   message age timer   0.00
designated port   8003           forward delay timer   0.00
designated cost   0           hold timer       0.31
flags

vif5.0 (6)
port id       8006           state       forwarding
designated root   8000.70e284134411   path cost       100
designated bridge   8000.70e284134411   message age timer   0.00
designated port   8006           forward delay timer   0.00
designated cost   0           hold timer       0.31
flags

vif6.0 (7)
port id       8007           state       forwarding
designated root   8000.70e284134411   path cost       100
designated bridge   8000.70e284134411   message age timer   0.00
designated port   8007           forward delay timer   0.00
designated cost   0           hold timer       0.31
flags

vif9.0 (10)
port id       800a           state       disabled
designated root   8000.70e284134411   path cost       100
designated bridge   8000.70e284134411   message age timer   0.00
designated port   800a           forward delay timer   0.00
designated cost   0           hold timer       0.00
flags

vif9.0-emu (11)
port id       800b           state       forwarding
designated root   8000.70e284134411   path cost       100
designated bridge   8000.70e284134411   message age timer   0.00
designated port   800b           forward delay timer   0.00
designated cost   0           hold timer       0.31
flags

root@freak:/etc/network#

Sum it up

We have learned how to debug various routing issues, how to add and remote default gateways, check network reachability with ICMP protocol with ping, traceroute as well check for DNS issues and given some hints how to resolve DNS misconfigurations.
We also learned how to check the configured Network interfaces certain settings and resolve issues caused by Network sides max Speed misconfigurations as well how to track and resolve communication issues caused by wrongly configured MAC addresses.
Further more learned on how to do a basic port and protocol debugging of state of Network packets with netstat and nc and check problems related to iptables Firewall and IP Forwarding misconfigurations.
Finally we learned some basic usage of tcpdump on how to track arp and MAC traffic and look up for a specific TCP / UDP protocol and its contained data.
There is certainly things this article is missing as the topic of debugging network connectivity issues on Linux is a whole ocean, especially as the complexity of Linux has grown dramatically these days.
I gues it is worthy to mention that unable to see remote network could be caused by wrong VLAN configurations on Linux or even buggy switches and router devices, due to hardware or software,
but I hope this article at least covers the very basics of network debugging and Linux.

Enjojy 🙂

Tags: article, broadcast, data, debugging, default, Display, fails, guide, histogram, ifconfig eth0, initial, internal, issues, link, linux?, loopback, mtr, MULTICAST, multiple times, network, network speed, Old, operating systems, ping, pipe, Present, running, RX, Settings, system administrators, tcp, time, Tx
Posted in Debian, Educational, Linux, Networking, System Administration | No Comments »

Little known facts about the dedication to Saint Martyr George The Glory-Bringer and his veneration across contries and religions

Saturday, May 8th, 2021

Largest part of body holy relics of the saints are kept in the town of Lod city 15 km (9.3 mi) southeast of Tel Aviv in the Central District of Israel. Back in the history due to the emerging veneration for saint George by both Christians and Muslims the town was temporary holding the name Georgioupolis, while his head relics is kept in Rome. There is a coptic monastery in Cairo pretending to hold personal belongings of saint George. In Saint Catherine Monastery (Mount of Sinai) are kept the three fingers of the arm of st. George. Churches dedicated to the saint started being built across the Roman empire even in the fourth century quite soon after his martyrdom, highest concentration of monasteries in his honor were born in Palestine. Biographics (Living) of saint George are written by the Byzantine authors saint Andreas of Cretes (written in 8-th century), Arcadius of Cyprus, Teodoris Quaestor, saint Gregory of Cyprus, saint Saint Symeon the Metaphrast (written 10th century).

Saint Simeon Metaphrast (μεταφράξειν – translator and a historian of Byzantine empire passed on to Christ, 960 year famous for collecting and systemizing biographies of many saints, his works are important source of history on Byzantine empire)

Hundreds of Apocrypths are dedicated to the glorious living of the saint and his amazing miracles, written mainly in Latin, Greek, Syrac, Arabian, Coptic, Ethiopian and other multitude of other languages. The most famous apocrypha on saint is so called "Greek Vienna's Palimpsest" (5th century). as well as the "The Deeds of Saint George" (from 6-th century., as well as "The Martyrdom of George" etc. The Apocrypha's text are evidently full of hyperbolas and many unhistorical facts different from the true living facts of the saint. The fallacies and apocryphas have been condemned by the Decretum Gelasianum ( thought to be Decretal of the prolific Pope Gelasius I, bishop of Rome 492–496 ) as heretical and blemish for the memory of the saint.

The miracle of Saving the Princess from the Dragon (one of the many apocryphas tradition about st. George) – Depiction Decani Monastery Serbia

Saint great martyr George together with The Holy Theotokos Mother Mary is a protector of Georgia (the country near Russia and not the State of the US :)).

Georgian Metal carved ancient icon of saint George

In Georgia the local verbal tradition assigns a family relation with the first missionary and Baptizer of Georgia saint Nino. The first Church dedicated to saint George in Georgia is built in year 335 ! by King Mirian on the burial place of Saint Nino. In 1098 year saint George has been proclaimed protector of England, after appearing in a vision to the participants to Crusades of that time. One hundred years later during reign of Richard the Lion Heart the status of protector of the Army becomes an official in the West. In year 1222 The Synod of Oxford decides, that saint George is to be venerated throughout the whole kingdom of England on 23 of April (old style calendar) – 6th of May in the current Gregorian public calendar we use – still venerated on the same date in Bulgaria even to this day, while the remembrance day of the saint has been publicly proclaimed as labor free.

In 14th century he is proclaimed a protector of England. In the beginning of 20th century the creator of the Scout Movement Lord Baden-Powell choose saint George for a protector of the Scouts. Saint George is considered protector of Moscow and Catalonia, until 18th century he was officially venerated as a protector of Portugal. In Greece he is venerated as agios Georgios, in Russia he is venerated under alternative names Jurij / Yurij (Юрий) and Egorij (Егорий). In year 1030 Grand prince of Kiev Yaroslav established in Kiev and Novgorod monasteries in honour of saint George (Yuriev Monastery) and gives an order the remembrance of saint George to be considered throughout Russian on 26th of November. The saint has been commonly depicted in Kings coins and seals. In Islam Saint George is famous under the name Djordjis (Djordjic).

His Living is translated in Arabian in the beginning of 8th century and through Arabian-Christians becomes popular among Arabian-muslims. In Arabian apocrypha text his biography is included in "The History of Prophets and Kings" from the 10-th century, where he is presented as a pupil of one of the apostles of Isa Īsā ibn Maryam (Jesus son of Mary). In the Islamic apocrypha st. George is said to have been put to tortures, but even though killed multiple times he always have been resurrected by Allah as a faithful servent. In some Arabian icons on the horse of saint George there is a small human figure with a muslim clothes and a water vessel at hand.

The iconography depicts also the miracle in Ramela that happened during a Church being built in dedication of Saint George, where one of the bought from far a stone pillar for the Church by a poor widow has been transferred by saint George miraculously via the sea by his all powerful prayer and placed to be the second Church holder right sight holder as well as the miracle when a Saracen Muslim soldier shoot towards the icon depiction of st George as an attempt to show that the saint icon is nothing more than a painted tree and immediately onwards his hand started unbearable hurting.

Saint George's (Aravijska)'s Miracle Making icon of Holy Mount Athos Zographous Monastery St. Mrtr. George the GloryBringer

The healing of the unberable came only after a Christian priest give the adive to the soldier to light up a sanctuary lamp in front of the same icon of saint George and to annoint himself with the oil from the burning chancel-lamp. After the miraculous healing the soldier confessed to be Christian and has been immediately punished with a maryrdom death. The name of the martyr is not preserved but the miraculous event is depicted on the arabian ancient icons.

Saint George the glorybringer in Church of saint George village Zlatolist (Bulgaria)

The Famous Miracle making icon of saint George from Hadji-Dimovo Monastery Bulgaria

Another interpretation of why there is depiction of a figure on the backside of the horse of saint George is the so called "Miracle of saint George with the Paphlagonian" that is connected with my homeland Bulgaria. The history says a young-man of Paphlagonia, who has been taken as acaptive by the bulgarians and given as a slave to a wealthy bulgarian nobleman from Preslav. Once when the slave was carrying vessel with a hot water to his master towards the second floor of the house, out of nowhere appeared saint George, he put him backwards on his horse and bring him back to Paphlagonia. In Paphlagonia at this time his parents were already serving a Memorial service for the forgiveness of the sins of their boy thinking he has been already killed in captivity. Being reunited with his parents the youngster thought he has been dreaming to see his parents again out of a sudden and what show him that the miracle translation of the boy from one location back to his parents was a reality was the vessel with water which was still held in his hand, thus as a remembrance of the miracle the boy is depicted on the back of saint George's horse.

There is much more to be said about this glorious saint, as there is plenty of miracles and stories monasteries and Churches events and venerations facts over the last 21 centuries in which the East and The World become Christian, but as the Gospel says it looks all the books on the world written won't be able to contain it.

Tags: Arabian, beginning, belongings, Biographics Living, body, coptic, day, Decretum Gelasianum, dedication, due, during, England, first, Georgia, Greek Vienna Palimpsest, hand, history, holy relics, icon, King Mirian, martyrdom, miracles, missionary, monasteries, multiple times, personal, remembrance, Rome, russia, saint, Saint George, Saint Nino, Tel Aviv, venerated, veneration, vessel, Vienna Palimpsest, year, years
Posted in Christianity, Everyday Life | No Comments »

Remove old unused kernels and cleanup orphaned packages on CentOS / RHEL/ Fedora and Debian Linux

Friday, October 23rd, 2020

remove-old-unused-kernel-on-centos-redhat-rhel-fedora-linux-howto-delete-orphaned-packages

If you administer CentOS 7 / CentOS 8 bunch of servers it is very likely after one of the scheduled Patch days every 6 months or so, you end up with a multiple Linux OS kernels installed on the system.
In normal situation on a freshly installed CentOS machine only one rpm package is installed on the system with the kernel release shipped with CentOS / RHEL / Fedora distro:
The reason to remove the old unused kernels is very simple, you don't want to have a messy installation and after some of the updates to boot up in a revert back old kernel or if you're pedantic to simply save few megas of space.
Some people choose to have more than one kernel just to make sure, if the new installed one doesn't boot, after a restart from ILO / IDRAC remote console interface you can select to boot the proper kernel. I agree having the old kernel before the system *kernel* upgrade as backup recovery is a good thing but this is a good thing to the point the system gets booted after reboot (you know we sysadmins usually after each major system package upgrade), we like to reboot the system warmly praying and hoping it will boot up next time 🙂

1. Remove CentOS last XX kernels from the OS

Of course removal of old kernels could be managed by a simple

yum remove kernel

yum-kernel-remove-centos-linux

One more than one kernel is present you can hence leave only lets say the last 2 installed kernel on the CentOS host (some people prefer to have only one) but just for the sake of having a backup kernel I like more to have last two kernels installed present, to do so run package-cleanup which is contained in yum-utils rpm package CentOS – this is CentOS / Redhat ( RHEL) specific command.

[root@centos ~ ]:# package-cleanup –oldkernels –count=2

package-cleanup-centos-linux-screenshot-1

–count=number argument – tells how many from the latest version kernels to get removed.

Note if you don't have the package-cleanup command install yum-utils package:

[root@centos ~ :]# yum install -y yum-utils

cleanup-old-kernels-linux-leave-only-set-of-2-kernels-active-on-centos-rhel-fedora

2. RemoveOld kernels from Fedora Linux – leave only the latest 3 installed

This is done with dnf by setting the –-latest-limit arg to negative value to how many last kernels want to keep

[root@fedora ~ ]:# dnf remove $(dnf repoquery –installonly –latest-limit=-3 -q)

3. Set how many kernels you want to be present on system all the time after package upgrades

It is possible to tell CentOS / RHEL / Fedora's on how many kernels show be kept installed on the system, the default configured on Operating system install time is to keep the last 5 installed kernel on the OS. This is controlled from installonly_limit=5 value that is usually as of year 2020 RPM based distributions found under /etc/yum.conf (on CentOS / RHEL) and in /etc/dnf/dnf.conf (in Fedora) configuration file and sets the desired number of kernels present on system after issuing commands yum upgrade / dnf upgrade –refresh etc.
The minimum number to give to installonly_limit is 2.

4. Remove orphan rpm packages from server

The next thing to do is to check the installed orphan packages to see if we can safely remove them; by orphaned packages we mean all packages which no longer serve a purpose of package dependencies.
Orphan packages are packages who left over from some old dependencies that are no longer needed on the system but just take up space and impose a possible security risk as some of them might end up with time with a public well known and hacked CVE vulnearbility.

Let me try to explain this concept with a quick example: package A is depended on package B, thus, in order to install package A the package B must also be installed. Once the package A is removed the package B might still be installed, hence the package B is now orphaned package.
Here’s how we can safely see the orphan packages we do have on our system:

[root@centos ~ :]# package-cleanup –quiet –leaves –exclude-bin

And here’s how we can delete them:

[root@centos ~ :]# package-cleanup –quiet –leaves –exclude-bin | xargs yum remove -y

The above commands should be launched multiple times, because the packages deleted with the first batch could create additional orphan packages, and so on: be sure to perform these tasks until no orphan packages appear anymore after the first package-cleanup command.

5. Delete Old Kernels and keep only last three ones on Debian / Ubuntu Linux

To do the same on a debian based distribution there is a command is provided by a deb package byobu, if you want to clean up old kernels on Debians :

$ sudo purge-old-kernels –keep 3

That's all folks enjoy ! 🙂

Tags: boot, installed, kernels, multiple times, packages, remove, servers, setting, system, yum
Posted in CentOS, Linux, Linux Package Management, System Administration | No Comments »

Saint Petka Paraskeva of Bulgaria of Epivates Thracia (famous as St. Petka of Tarnovo) feast day 14 October

Friday, October 16th, 2020

Sveta-Petka-Paraskeva-Bylgarska-Balkanska-Epivatska

The inhabitants of Thracia are of a great and royal origin and due to recent historical studies, Thracians have been one of the most developed nations for its time they're developments and achievements especially in crafts such as vessel creation even up to day are perhaps the most unique.
It is still unknown of the exact technology used to create such a elegant and precise vessels. A little is known of the Thracians society as they have reached their bloom in a high speed and the place of the later Roman Empire province Thracia has been in a place where it was destroyed to the ground and robbed at multiple times eradicating unique piece of one of the best created ever forms of art.
Territories of Thrakia has been geographically today located in Southeast Europe, now split among Bulgaria, Greece, and Turkey, which is bounded by the Balkan Mountains to the north, the Aegean Sea to the south, and the Black Sea to the east.

Thrace_and_Thracians-present-day_state_borderlines-picture

Territy of Thracia shown on a contemporary European (Balkans Maps)

World-famous-Thracian-Treasury-picture-1

One of the most famous piece of such art is the World Famous Thracian's Treasuary.

World-famous-Thracian-Treasury-picture

The thrakians Empire and civillization has its bloom from 5th – 4th century before Christ era (B.C.).
Saint Petka of Epivates region Thrakia was of a Bulgarian origin and lived much later in Xth – XI-th century A.D in Thracia. It is known she was of Bulgarian origin (her mother and father was of Bulgarian origin.) of the first generations who has received in 9-th century Baptism, in the times of the Baptism of Bulgaria conducted by the Apostle equal Saint King Boris I the Baptizer of Bulgaria in year 864 AD. Thracians as an ancient and a blessed nation in craftship and arts was among the nations who received baptism on a good 'soil', as the seed of beauty and goodness has already been in their civillization.

The short Living of Saint Petka of Bulgaria (of Epivates)

Out of this Christian atmosphere has rised Saint Petka also known as (Parashkeva). Saint Petka name means literally translated Friday and due to being born in Thracia on today territory of Balkans she is been venerated highly not only in Bulgaria but across all Orthodox Christians nations on the Balkans – Bulgarians, Romanians, Serbs, Greeks, Macedonians. Due to that Saint Petka is also famous as "Saint Petka of The Bulkans".
Saint Petka could be therefore heard to be called often Petka of Serbs (of Belgrade), Saint Petka of Moldova (of Iași), Mother Paraskeva / Parashkeva, Parascheva the New, Parascheva the Young, Ancient Greek: Ὁσία Παρασκευὴ ἡ Ἐπιβατινή, Greek: Οσία Παρασκευή η Επιβατινή ή Νέα, Romanian: Cuvioasa Parascheva, Bulgarian / Serbian : Света Петка / Sveta Petka or Петка Параскева / Petka Paraskeva, Paraskeva Pyatnitsa, Parascheva of Tirnovo).

The first information about her living is found in a local educated person (writter) which as of the time were too little and writter in Greek in short. It did not follow the Church cannons and due to that by an order of Patriarch of Constantinople Nikolas IV Musalon of Constantinople deacon Vaslik has described in a more well systemized way her living, the Greek original unfortunately is now lost. At the time of writting her biography, she has been mostly popular in the realms of Byzantine Empire Thracia.

Bulgarian-Empire-under-King-Ivan-Asen-II-map-1917

The Bulgarian Empire during the reign of Ivan Asen II. Atlas of Dimitar Rizov year 1917

Since the time of King Ivan Asen II a new biogprahy of saint has been written in Bulgarian which included narration of the transfer of her holy relics to Medieval Capital of Bulgaria Tarnovo. However peak and the key towards the immerse veneration to St. Petka that is evident to this very date has played the biography written by last Bulgarian Patriarch also a saint – st. Euthymius of Tarnovo. in year 1385 AD short before the fall under Turkish Slavery of Bulgaria in y. 1393.

Saint Patriarch Eutymious was the last person who in 1393 has actively parcipated in the protection of the fortified Tarnovo and see with his eyes the fall down of the city (by treachery).

When asked by the terrified people 'To whom do you leave us holy father, when the Turkish were taking him away?' He replied heart tearingly 'To the Holy Trinity The Father, The Son and The Holy Trinity our God I leave you and to the most Blessed Mother of God Theotokos now and For Eternity !!!'

Saint-Patriarch-Eutymious-the-last-Blessing-picture-sveti_Evtimij_seten_blagoslov

Saint Patriarch Eutymius (Evtimij) blessing the people in Medieval Bulgarian city Tarnovo for a last time before the Turkish took him away for imprisonment
Picture source Pravoslavieto.com

St Euthymius of Tarnovo work is one of the most unique bibliographies and a precious piece of medieval literature it is innovative for its time and spectacular, emotion rich creation, who become famous far before the borders of Bulgaria in the whole Slavonic world of that time, especially in todays territory of ex soviet countries Romania, Moldova, Ukraine and even far Russia.

Saint_Patriarch-Eutymius-last-bulgarian-patriarch-before-Turkish-Slavery

Saint Patriarch Eutymious of Bulgaria
Picture source Pravoslavieto.com

The veneration of Saint Petka of Bulgaria as a protector of family and a warm prayerer for all those who venerate her in this country has slowly spread in the coming centuries by pupils of St. Euthymius of Tarnovo who according to some historians whose works came to us in the form of the a bit more standardized Church Slavonic used in the Eastern Orthodox Churches as a fruit of the works of St. Euthymus.

The Living of Saint Petka Parashkeva

Saint Petka Parashkeva Picture source Pravoslavieto.com

Tropion 4-th voice

Desertous and silent living you loved after Christ your groom, diligently you ran to and his good yoke you took in your younghood,
with the Sign of the Cross against the thought enemies you have manly armed, with fasting feats and prayer and with tear drops the coals of passions extinguished oh highly famed Paraskevo. And now in the Heavenly halls with the wise virgins you stay in front of Christ, pray for us who venerate your holy remembrance.

Kontakion, voice 6

Let us piusly sung our reverend mother Saint Petka, because by living the corruptable in live, received the imperishable in eternity, becoming holy intercessor for all in trouble and exhausting from the evils of life. For the reason she received from God imperishable fame, glory and grace to be a wonder worker.

NB ! St. Petka of Epivates has not to be confused with Saint Petka (from Inokia who lived in 303 AD venerated on 28 of October) or St Petka the Roman (feast day 26 July).

St. Petka's has been born in city of Epivates in Byzantium (today city called Selim Pasha nearby Odrin's Aegian City) in 10-th Century from a famous and respectful family, her father Nikita has been very rich landowner.

She lived in the second part of X-th century. According to hear living by Patriarch Eutymious, her smaller brother Eutymious who become a monk has been a Metropolitan of Maditos for 40 years and in year (989 – 996) died aged 81 and is shortly after canonized as saint, his younger sister St. Paraskeva passed away after him in the new 11-th century and is aged at least 85 in the time of passing in the city of Kallikrateia.

The living continues that near the age of 10 year old she heard in a Christian temple a voice by Jesus Christ himself in resemblance to Saint Apostle Paul and said the Evangelical New Testamental words:
"Whoever wants to walk after me, let him deny himself, to take his cross and follow me !".

The unexpected vision convinced the young Paraskeva to immediately exchange her new clothes to a beggers to leave all her belongings to the poor and live a silent living similarto begger for a time in work and prayer, though she did not leave her parents home. On a few occasions all she had worked for has been distributed to the poor.

Sveta-Petka-Bylgarska-Balkanska

Greek typical depiction of Saint Petka of Epivates

When her parents died, her brother as already a monk and Bishop. St. Petka leave her house and travelled to Constanople and received a nun tonsure and as a nun she lived for 5 years near the deserted Church of the "Protection of the Virgin Mary" in the capital suburb of Heraklia. She travelled to the Holy lands visiting Jerusalem and Church of Holy Sepulchre.
Following the example of the blessed famed Saint Mary of Egypt, she lived in Jordan's desert many years till eldership.

Feeling and foreseeing her death, she travelled back through Constantinople to city of Epivates. Settle near the Church "Holy Apostles", where after 2 years of living in deep prayer and fasting labours living in solitary in holiness passed away silently to Christ in heavenly life. Compassionate Christians immediately buried her body of the nun outside of the city walls as a foreigner. A shortly after numerous miracles started happening on her grave.

St_Petka-Parashkeva-Epivatska-Klisura_Monastery_Holy_Icon

Saint Petka Parashkeva Bulgarian Icon from Klisura Monastery located nearby Sofia Bulgaria

In 1230 King Ivan Asen II the most powerful South-eastern European ruler demanded from the the Knights of the Crusaders to submit him her holy relics who are found still in Tracian city Kaliakratea ruled at that time by the Holy Latin Empire. King Ivan Asen II together with the patriach Joachim the first receives her holy relics with honor and settles her incorruptabilities into the newly creates Church in honour of herself St. Petka behind Tsarevets Fortress. Saint Petka became from that point considered as a protectress of the city, the throne and the country.
Her holy relics arrived from Kallikrateia in Tarnovo, the Capital of Second Bulgarian Empire in year 1230 AD, she has been thus called Paraskeva of Tarnovo and has been venerated as a protectress of the Tarnovo city the Bulgarian nation and the country. The attitude towards Saint Petka Tarnovska as a protectress of Bulgarian nation and contry is been clearly seen by the mention in the Bulgarian and International acts (documents) and manuscripts of that XII – XII century.

Saint_Petka-Epivatska-Bylgarska-Romanian-in-Iashi-Romania-veneration-of-romanian-monks

Romanian Monks and Priests venerate the holy relics of Saint Petka of Epivates in Iashi Romania

In subsequent years, St. Petka Paraskevi’s holy relics were transferred to various churches in the region.

In 1393 due to the fall of Bulgarian capital to save them her holy relics were transferred to fortress of Bdin today city of Vidin Bulgaria, but 3 years later 1396 Vidin's fortress also fall under the ungodly yatagan of the muslim enslaver and to protect the relics they were again transferred to Belgrade, specifically the Ružica Church. When Belgrade fell to Ottoman forces in 1521, the relics were transferred to Constantinople. In 1641, the relics were transferred to Trei Ierarhi Monastery, in Iaşi, Moldavia (nowadays, eastern part of Romania). In 1888, they were transferred to the Metropolitan Cathedral of Iaşi.

Since 1888 they are kept in Romanian city of Iaşi and are a target of pilgrims from all around Romania, Bulgaria and other Orthodox Christian countries of the Balkans. For the day her memory is remembered in the Romanian Church usually about 200 000 people mostly from Romania and others travel to Iaşi's Cathedral in the Trei Ierarhi Monastery (Three Hierarchs – saint John Crysostom, St. Basilius the Great and St. Gregory the Great) of the for a blessing and to beg the saint for her families, personal issues, curings especially of eye diseases

A severe drought in 1946-47 affected Moldavia, adding to the misery left by the war. Metropolitan Justinian Marina permitted the first procession featuring the coffin containing the relics of Saint Paraskevi, kept at Iaşi since then. The relics wended their way through the drought-deserted villages of Iaşi, Vaslui, Roman, Bacău, Putna, Neamţ, Baia and Botoşani Counties. The offerings collected on this occasion were distributed, based on Metropolitan Justinian's decisions, to orphans, widows, invalids, school cafeterias, churches under construction, and to monasteries in order to feed the sick, and old or feeble monks.

In the historical document with Venezia as of (year 1347), King Ivan Alexander of Bulgaria swears in the name of most holy considered matters, the document says – quote "in the name of God, The Most Holy Theotokos, The Holy Cross and The Most Holy Paraskeva of Tarnovo".

Since Second Bulgarian Kingdom, St. Petka has been venerated as a main patroness and protector of Bulgarian nation and country, protectress of countries of Moldova, Romania and Bulgarian cities of Veliko Tarnovo, Gabrovo and Troyan.

In Bulgaria it is an old tradition to name our childs in favour of Saint Petka, my grand-grand mother God Forgive us has also been called Parashkeva in favor of Saint Petka.

Holy Mother Petka Paraskeva (Parashkevo) Pray the Lord Jesus Christ to have mercy on All us the sinners !

Tags: Aegian City, Ancient Greek, art, Bulgarian, centuries, Christ, create, created, day, due, goodness, ground, high speed, living, multiple times, North Trakia, October, origin, orthodox christians, Petka Praskeva Parashkeva, piece, place, received, roman empire, Romanian Cuvioasa Parascheva, Second Bulgarian Kingdom, sofia bulgaria, studies, time, today, Turkey Thrace, unknown, venerated, year, years
Posted in Christianity, Various | No Comments »

Find when cron.daily cron.weekly and cron.monthly run on Redhat / CentOS / Debian Linux and systemd-timers

Wednesday, March 25th, 2020

The problem – Apache restart at random times

I've noticed today something that is occuring for quite some time but was out of my scope for quite long as I'm not directly involved in our Alert monitoring at my daily job as sys admin. Interestingly an Apache HTTPD webserver is triggering alarm twice a day for a short downtime that lasts for 9 seconds.

I've decided to investigate what is triggering WebServer restart in such random time and investigated on the system for any background running scripts as well as reviewed the system logs. As I couldn't find nothing there the only logical place to check was cron jobs.
The usual

crontab -u root -l

Had no configured cron jobbed scripts so I digged further to check whether there isn't cron jobs records for a script that is triggering the reload of Apache in /etc/crontab /var/spool/cron/root and /var/spool/cron/httpd.
Nothing was found there and hence as there was no anacron service running but /usr/sbin/crond the other expected place to look up for a trigger even was /etc/cron*

1. Configured default cron execution times, every day, every hour every month

# ls -ld /etc/cron.*
drwxr-xr-x 2 root root 4096 feb 27 10:54 /etc/cron.d/
drwxr-xr-x 2 root root 4096 dec 27 10:55 /etc/cron.daily/
drwxr-xr-x 2 root root 4096 dec 7 23:04 /etc/cron.hourly/
drwxr-xr-x 2 root root 4096 dec 7 23:04 /etc/cron.monthly/
drwxr-xr-x 2 root root 4096 dec 7 23:04 /etc/cron.weekly/

After a look up to each of above directories, finally I found the very expected logrorate shell script set to execute from /etc/cron.daily/logrotate and inside it I've found after the log files were set to be gzipped and moved to execute WebServer restart with:

systemctl reload httpd

My first reaction was to ponder seriously why the script is invoking systemctl reload httpd instead of the good oldschool

apachectl -k graceful

But it seems on Redhat and CentOS since RHEL / CentOS version 6.X onwards systemctl reload httpd is supposed to be identical and a substitute for apachectl -k graceful.
Okay the craziness of innovation continued as obviously the reload was causing a Downtime to be visible in the Zabbix HTTPD port Monitoring graph …
Now as the problem was identified the other logical question poped up how to find out what is the exact timing scheduled to run the script in that unusual random times each time ??

2. Find out cron scripts timing Redhat / CentOS / Fedora / SLES

/etc/cron.{daily,monthly,weekly} placed scripts's execution method has changed over the years, causing a chaos just like many Linux standard things we know due to the inclusion of systemd and some other additional weird OS design changes. The result is the result explained above scripts are running at a strange unexpeted times … one thing that was intruduced was anacron – which is also executing commands periodically with a different preset frequency. However it is considered more thrustworhty by crond daemon, because anacron does not assume the machine is continuosly running and if the machine is down due to a shutdown or a failure (if it is a Virtual Machine) or simply a crond dies out, some cronjob necessery for overall set environment or application might not run, what anacron guarantees is even though that and even if crond is in unworking defunct state, the preset scheduled scripts will still be served.
anacron's default file location is in /etc/anacrontab.

A standard /etc/anacrontab looks like so:

[root@centos ~]:# cat /etc/anacrontab
# /etc/anacrontab: configuration file for anacron

# See anacron(8) and anacrontab(5) for details.

SHELL=/bin/sh
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root
# the maximal random delay added to the base delay of the jobs
RANDOM_DELAY=45
# the jobs will be started during the following hours only
START_HOURS_RANGE=3-22

#period in days delay in minutes job-identifier command
1   5   cron.daily       nice run-parts /etc/cron.daily
7   25   cron.weekly       nice run-parts /etc/cron.weekly
@monthly 45   cron.monthly       nice run-parts /etc/cron.monthly

START_HOURS_RANGE : The START_HOURS_RANGE variable sets the time frame, when the job could started.
The jobs will start during the 3-22 (3AM-10PM) hours only.

cron.daily will run at 3:05 (After Midnight) A.M. i.e. run once a day at 3:05AM.
cron.weekly will run at 3:25 AM i.e. run once a week at 3:25AM.
cron.monthly will run at 3:45 AM i.e. run once a month at 3:45AM.

If the RANDOM_DELAY env var. is set, a random value between 0 and RANDOM_DELAY minutes will be added to the start up delay of anacron served jobs.
For instance RANDOM_DELAY equels 45 would therefore add, randomly, between 0 and 45 minutes to the user defined delay.

Delay will be 5 minutes + RANDOM_DELAY for cron.daily for above cron.daily, cron.weekly, cron.monthly config records, i.e. 05:01 + 0-45 minutes

A full detailed explanation on automating system tasks on Redhat Enterprise Linux is worthy reading here.

!!! Note !!! that listed jobs will be running in queue. After one finish, then next will start.

3. SuSE Enterprise Linux cron jobs not running at desired times why?

in SuSE it is much more complicated to have a right timing for standard default cron jobs that comes preinstalled with a service

In older SLES release /etc/crontab looked like so:

SHELL=/bin/bash
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root
HOME=/

# run-parts
01 * * * * root run-parts /etc/cron.hourly
02 4 * * * root run-parts /etc/cron.daily
22 4 * * 0 root run-parts /etc/cron.weekly
42 4 1 * * root run-parts /etc/cron.monthly

As time of writting article it looks like:

SHELL=/bin/sh
PATH=/usr/bin:/usr/sbin:/sbin:/bin:/usr/lib/news/bin
MAILTO=root
#
# check scripts in cron.hourly, cron.daily, cron.weekly, and cron.monthly
#
-*/15 * * * * root test -x /usr/lib/cron/run-crons && /usr/lib/cron/run-crons >/dev/null 2>&1

This runs any scripts placed in /etc/cron.{hourly, daily, weekly, monthly} but it may not run them when you expect them to run.
/usr/lib/cron/run-crons compares the current time to the /var/spool/cron/lastrun/cron.{time} file to determine if those jobs need to be run.

For hourly, it checks if the current time is greater than (or exactly) 60 minutes past the timestamp of the /var/spool/cron/lastrun/cron.hourly file.

For weekly, it checks if the current time is greater than (or exactly) 10080 minutes past the timestamp of the /var/spool/cron/lastrun/cron.weekly file.

Monthly uses a caclucation to check the time difference, but is the same type of check to see if it has been one month after the last run.

Daily has a couple variations available – By default it checks if it is more than or exactly 1440 minutes since lastrun.
If DAILY_TIME is set in the /etc/sysconfig/cron file (again a suse specific innovation), then that is the time (within 15minutes) when daily will run.

For systems that are powered off at DAILY_TIME, daily tasks will run at the DAILY_TIME, unless it has been more than x days, if it is, they run at the next running of run-crons. (default 7days, can set shorter time in /etc/sysconfig/cron.)
Because of these changes, the first time you place a job in one of the /etc/cron.{time} directories, it will run the next time run-crons runs, which is at every 15mins (xx:00, xx:15, xx:30, xx:45) and that time will be the lastrun, and become the normal schedule for future runs. Note that there is the potential that your schedules will begin drift by 15minute increments.

As you see this is very complicated stuff and since God is in the simplicity it is much better to just not use /etc/cron.* for whatever scripts and manually schedule each of the system cron jobs and custom scripts with cron at specific times.

4. Debian Linux time start schedule for cron.daily / cron.monthly / cron.weekly timing

As the last many years many of the servers I've managed were running Debian GNU / Linux, my first place to check was /etc/crontab which is the standard cronjobs file that is setting the { daily , monthly , weekly crons }

debian:~# ls -ld /etc/cron.*
drwxr-xr-x 2 root root 4096 фев 27 10:54 /etc/cron.d/
drwxr-xr-x 2 root root 4096 фев 27 10:55 /etc/cron.daily/
drwxr-xr-x 2 root root 4096 дек 7 23:04 /etc/cron.hourly/
drwxr-xr-x 2 root root 4096 дек 7 23:04 /etc/cron.monthly/
drwxr-xr-x 2 root root 4096 дек 7 23:04 /etc/cron.weekly/

debian:~# cat /etc/crontab
# /etc/crontab: system-wide crontab
# Unlike any other crontab you don't have to run the `crontab'
# command to install the new version when you edit this file
# and files in /etc/cron.d. These files also have username fields,
# that none of the other crontabs do.

SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin# Example of job definition:
# .—————- minute (0 – 59)
# | .————- hour (0 – 23)
# | | .———- day of month (1 – 31)
# | | | .——- month (1 – 12) OR jan,feb,mar,apr …
# | | | | .—- day of week (0 – 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat
# | | | | |
# * * * * * user-name command to be executed
17 *   * * *   root cd / && run-parts –report /etc/cron.hourly
25 6   * * *   root   test -x /usr/sbin/anacron || ( cd / && run-parts –report /etc/cron.daily )
47 6   * * 7   root   test -x /usr/sbin/anacron || ( cd / && run-parts –report /etc/cron.weekly )
52 6   1 * *   root   test -x /usr/sbin/anacron || ( cd / && run-parts –report /etc/cron.monthly )

What above does is:

– Run cron.hourly once at every hour at 1:17 am
– Run cron.daily once at every day at 6:25 am.
– Run cron.weekly once at every day at 6:47 am.
– Run cron.monthly once at every day at 6:42 am.

As you can see if anacron is present on the system it is run via it otherwise it is run via run-parts binary command which is reading and executing one by one all scripts insude /etc/cron.hourly, /etc/cron.weekly , /etc/cron.mothly .

anacron – few more words

Anacron is the canonical way to run at least the jobs from /etc/cron.{daily,weekly,monthly) after startup, even when their execution was missed because the system was not running at the given time. Anacron does not handle any cron jobs from /etc/cron.d, so any package that wants its /etc/cron.d cronjob being executed by anacron needs to take special measures.

If anacron is installed, regular processing of the /etc/cron.d{daily,weekly,monthly} is omitted by code in /etc/crontab but handled by anacron via /etc/anacrontab. Anacron's execution of these job lists has changed multiple times in the past:

debian:~# cat /etc/anacrontab
# /etc/anacrontab: configuration file for anacron

# See anacron(8) and anacrontab(5) for details.

SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
HOME=/root
LOGNAME=root

# These replace cron's entries
1   5   cron.daily   run-parts –report /etc/cron.daily
7   10   cron.weekly   run-parts –report /etc/cron.weekly
@monthly   15   cron.monthly   run-parts –report /etc/cron.monthly

In wheezy and earlier, anacron is executed via init script on startup and via /etc/cron.d at 07:30. This causes the jobs to be run in order, if scheduled, beginning at 07:35. If the system is rebooted between midnight and 07:35, the jobs run after five minutes of uptime.
In stretch, anacron is executed via a systemd timer every hour, including the night hours. This causes the jobs to be run in order, if scheduled, beween midnight and 01:00, which is a significant change to the previous behavior.
In buster, anacron is executed via a systemd timer every hour with the exception of midnight to 07:00 where anacron is not invoked. This brings back a bit of the old timing, with the jobs to be run in order, if scheduled, beween 07:00 and 08:00. Since anacron is also invoked once at system startup, a reboot between midnight and 08:00 also causes the jobs to be scheduled after five minutes of uptime.
anacron also didn't have an upstream release in nearly two decades and is also currently orphaned in Debian.

As of 2019-07 (right after buster's release) it is planned to have cron and anacron replaced by cronie.

cronie – Cronie was forked by Red Hat from ISC Cron 4.1 in 2007, is the default cron implementation in Fedora and Red Hat Enterprise Linux at least since Version 6. cronie seems to have an acive upstream, but is currently missing some of the things that Debian has added to vixie cron over the years. With the finishing of cron's conversion to quilt (3.0), effort can begin to add the Debian extensions to Vixie cron to cronie.

Because cronie doesn't have all the Debian extensions yet, it is not yet suitable as a cron replacement, so it is not in Debian.

5. systemd-timers – The new crazy systemd stuff for script system job scheduling

Timers are systemd unit files with a suffix of .timer. systemd-timers was introduced with systemd so older Linux OS-es does not have it.
Timers are like other unit configuration files and are loaded from the same paths but include a [Timer] section which defines when and how the timer activates. Timers are defined as one of two types:

Realtime timers (a.k.a. wallclock timers) activate on a calendar event, the same way that cronjobs do. The option OnCalendar= is used to define them.
Monotonic timers activate after a time span relative to a varying starting point. They stop if the computer is temporarily suspended or shut down. There are number of different monotonic timers but all have the form: OnTypeSec=. Common monotonic timers include OnBootSec and OnActiveSec.

For each .timer file, a matching .service file exists (e.g. foo.timer and foo.service). The .timer file activates and controls the .service file. The .service does not require an [Install] section as it is the timer units that are enabled. If necessary, it is possible to control a differently-named unit using the Unit= option in the timer’s [Timer] section.

systemd-timers is a complex stuff and I'll not get into much details but the idea was to give awareness of its existence for more info check its manual man systemd.timer

Its most basic use is to list all configured systemd.timers, below is from my home Debian laptop

debian:~# systemctl list-timers –all
NEXT LEFT LAST PASSED UNIT ACTIVATES
Tue 2020-03-24 23:33:58 EET 18s left Tue 2020-03-24 23:31:28 EET 2min 11s ago laptop-mode.timer lmt-poll.service
Tue 2020-03-24 23:39:00 EET 5min left Tue 2020-03-24 23:09:01 EET 24min ago phpsessionclean.timer phpsessionclean.service
Wed 2020-03-25 00:00:00 EET 26min left Tue 2020-03-24 00:00:01 EET 23h ago logrotate.timer logrotate.service
Wed 2020-03-25 00:00:00 EET 26min left Tue 2020-03-24 00:00:01 EET 23h ago man-db.timer man-db.service
Wed 2020-03-25 02:38:42 EET 3h 5min left Tue 2020-03-24 13:02:01 EET 10h ago apt-daily.timer apt-daily.service
Wed 2020-03-25 06:13:02 EET 6h left Tue 2020-03-24 08:48:20 EET 14h ago apt-daily-upgrade.timer apt-daily-upgrade.service
Wed 2020-03-25 07:31:57 EET 7h left Tue 2020-03-24 23:30:28 EET 3min 11s ago anacron.timer anacron.service
Wed 2020-03-25 17:56:01 EET 18h left Tue 2020-03-24 17:56:01 EET 5h 37min ago systemd-tmpfiles-clean.timer systemd-tmpfiles-clean.service

8 timers listed.

N ! B! If a timer gets out of sync, it may help to delete its stamp-* file in /var/lib/systemd/timers (or ~/.local/share/systemd/ in case of user timers). These are zero length files which mark the last time each timer was run. If deleted, they will be reconstructed on the next start of their timer.

Summary

In this article, I've shortly explain logic behind debugging weird restart events etc. of Linux configured services such as Apache due to configured scripts set to run with a predefined scheduled job timing. I shortly explained on how to figure out why the preset default install configured cron jobs such as logrorate – the service that is doing system logs archiving and nulling run at a certain time. I shortly explained the mechanism behind cron.{daily, monthy, weekly} and its execution via anacron – runner program similar to crond that never misses to run a scheduled job even if a system downtime occurs due to a crashed Docker container etc. run-parts command's use was shortly explained. A short look at systemd.timers was made which is now essential part of almost every new Linux release and often used by system scripts for scheduling time based maintainance tasks.

Tags: checks, crontab, daily, EET, execution, form, last time, list, multiple times, Nothing, root root, running, sbin, scripts, shutdown, system startup, timer, today
Posted in Bash Scripting, Curious Facts, Linux, System Administration, Various | No Comments »

The beheading of Saint John the Baptist feats in Eastern Orthodox Church – A short history of saint John Forerunners Holy Relics

Saturday, September 14th, 2019

Beheading_of_St_John_the_Baptist_Icon_IX-century

Saint John the Baptist (The Forerunner of Christ) is all known for being the baptizer of the Lord Jesus Christ in Jordan's river in Israel.

However as nowadays most people are away from the Church and from traditions, that many generations of our ancestors used to follow, little know the details of his beheading and the meaning of why he is venerated so much by so many generations in the last 2000 years.

Thus In this small article, I'll try to shed some light on the Saint John Beheading feast known in Church Slavonic world as Oseknovenie (Осекновение) = beheading and is considered a day of sorrow for the Church for the reason the biggest Old Testamental prophet, a hermit and a man of Gigantic spiritual significance Saint John the Baptist has been beheaded unfairly for having no fault at all but this happened so his righteousness raise up even more and be clear for the generations to come.

The feast of Saint John the Baptist is celebrated on 29 of August in Eastern Orthodox Church, where old Calendar Churches celebrate the feast (13 days) later on 11 of September – I'll not get into details about calendars as this is a long discussion for a separate article.

It should be said in the Church saint John the Baptist is considered the highest saint among all –“the first among martyrs in grace”, venerated next in glory to Virgin Mary.

The Martyrdom of Saint John happened in the 32 years after the Nativity (The Birth) of Christ as this is said in the Gospel of Mathew 14:1-12 and Gospel of Mark (6:14-29) in New Testament.

Many of the small details, we know about saint John and his earthly living are not given in the Gospel however, but are instead given in the Chuch Tradition (that is kept in the main books and the Living of the Saints, as well as all the books written by the officially canonized saints over the years that used for Eastern Orthodox Church services Singing for many centuries).

From there we know the beheading of Saint John happened short time before the Crucifixion of Christ. After the death of Herode the Great, Romans divided the territory of Province of Palestine in 4 parts and on top of each placed his governor.

Herodos Antipa received by Emperor Augustus Galilea as a territory of Governance. He had a law binded marriage, who was a daughter of king Arepha. Herod left her and cohabited (unlawfully) with Herodias who was his mistress and brother's wife.

As Herodos was a governer and recpetively example for all his subordinate in his Kingdom and was living unlawfully with that woman saint John who knew him personally rebuked him multiple times publicly advising him to leave that woman and live with his lawfully marriaged one as it was written in God's law – that such people are worthy for death just like moreover this was the unwritten law followed by all kind of peoples of his time from noble to smallest and poorest.

Herodoes did not listened and wanted to get rid of saint Johns somehow but he was scared to accuse him for some kind of kingdom lawlessness as the knew saint John was a true prophet of God and feared the people who recognized him as a true prophet as well as feared he might be put off throne for his evil deed if he finds an excuse to kill the prophet of God.

As the critics on Herodoes living with a concubine while being in marriage, eventually not finding any other way to shut Saint John's mouth, king Herodoes put saint John in Prison with the excuse Saint John was a rebel and preaching things against established authority (About this event is said in Bible Gospel of Luke 3, 19-20).

For his birthday Herodoes prepared enormous banquet in which in front of the many invited guests danced (Salome / Salomia) – the daughter of Herod II and Herodias and her dancing was so much pleasing for the already drunk Herod and in his drunkenness he promised to give her anything she desired up to half of his kingdom.
Salome was still young woman and as it was the tradition then not knowing what would be the best to ask for, she asked her Mother and the Mother being in unlawful relations with Herod in her hatred for the rebuking prophet saint John asked, the head of Saint John the Baptist on a platter.

The dance of Salome with Saint John's head on a Platter Orthodox icon

Even though Herod was appaled by this strange request, he had to reluctanly agree to keep his word as he was a ruler of a great power and for that time, not keeping a word publicly given would make him though weak, a fraudulant and eventually this will be reason for a rumors for his unseriousness to circulate the kingdom, thus unwillingly he agreed sent soldier to the prison to behead Saint John and the Head of the saint was brought to the perverted Solome and the harlot mother of hers Herodias.

Due to Church tradition when the Head of the 'Biggest in Spiritual Power' of Man born after Christ, as the Gospel speaks was brought to the lecherous feast, the Head even in the platter continued to rebuke, the unwalfullness of Herod.

The Jewish famous Historian Flavius Josiphus in his historical book Antiques of the Jews wrote, the reason for beheading of Saint John was:

"lest the great influence John had over the people might put it into his [John’s] power and inclination to raise a rebellion, (for they seemed ready to do any thing he should advise), [so Herod] thought it best [to put] him to death."

Flavius also states that many of the Jews believed that the military disaster that fell upon Herod as his throne fall a by the hands of Aretas his (father-in-law) was God's immediate punishment for unrighteous behaviour.
There is no exact date when behading of Saint John occured but the historians place it somewhere in year 28 or 29 A. D. (Anno Dommini).

The body of saint John was buried immediately (separately from the body) as Herodias for her hatred for the prophet ordered the body to be buried separately from the head, it was buried in the small Palestinian Village (Sebaste), while Herodias took his holy head and buried it in a dung heep.
Later Joanna (canonized later by saint known as Saint Joanna) – a wife of Herods steward, secretly went to place took the head and buried in the Mount of Olives, where it remained hidden for many centuries.

But the wrath of God is never late soon after Salome was passing a frozen river and while walking on it the ice collapsed and her body up to the head fall hanging in the water, while her head was sitting still over the water.
Just as she used to kick her feet on the ground, she was now, like dancing, making helpless movements in icy water.

So Salome hung until the sharp ice cut through her neck. Her head, cut off with a sharp ice, then her head was brought to Herod and Herodias, as John the Baptist's head had once been brought to them, and her body had never been found. The king of Arif of Arabia, in revenge for the dishonor of his daughter – the wife of Herod the four-owner – moved his troops against the wicked king and defeated him. The Roman emperor Guy Julius Caesar Caligula (37-41) in anger sent Herod, together with Herodias, into captivity to Gaul, and then to . There they were consumed by the sprawling earth.

By a divine revelation the head of Saint John has been found in the 4th century (Celebrated in the Church with a special feast known as The First Finding of Saint John's the Baptist head by a governing official of Eastern Roman empire district who eventually choose to become a monk (monk Inokentij / Innocent). The head of saint John has been found by both divine revelation and the testimony of an Old Jew who confirmed the Jewish oral tradition for the burial of John the Baptist head on that exact place .
Innocent decided to build a Church and a monastic Cell in glory of Saint John the Baptist as the place was holy and sanctified by the graceous head of St. John.
Fearful that holy relics of such a high importance, might be soon stolen and sold, mocked over by unbelievers or destroyed, he immediately hide (burid) the St. John on the very same place, where he found it in the same vessel it was orginally.
Unfortunately on monk Innocent dead the Church fell into ruin was abandoned due to its desert location and eventually as it always happened in that times with old buildings, people used its construction stones for fortifying their own near village houses.

The Second Finding of the Head of Saint John the Baptist, happened some years later in 452 A.D. , during the days of Constantine the Great by two Christian monks who went for a Jerusalem for pilgrimage.who had God given revelation (Saint John himself appeared in a kind of a Vision to the two) and hsa indicated for the same hidden location where Innocent found it (laying under the Church ruins altar).
After digging on the place, the holy relic was found placed in a sack and brought with them to heir home land. On the way back they've met a potter not telling him what was inside the bag and asked him the bag to carry being lazy to do. Saint John the Baptist appeared the potter and told him to take his head and bring it away from this careless lazy monks immediately. The potter took Saint John's head home, and kept it there praying fervently to saint John the Baptist daily, soon before his death he put the head in a container and gave it to his sister.
The 1st and 2nd finding of saint Johns head is established as a feast celebrated yearly in Eastern Orthodox Church on 24 of February.

The feast of Beheading of Saint John in the Church is always observed in the Eastern Orthodox Churches Bulgarian, Russian, Serbian, Greek, Romanian, Georgian etc. with a strict fasting as a sign for the great sorrow we Christians have for the beheading of the Greatest of Prophets and Highest in sight of God born of man.

In some cultures, the pious will not eat food from a flat plate, use a knife, or eat round or red food (such as tomatoes, watermelon, red peppers etc.) on this day.

A short time after a Hieromonk Eustathius (considered by Church historians) to be part of the Arian heretical division happen to have th chance to possess the holy head and he used it frequently to attract followers to the Heretical teachings of Arius (a Lybian heretic presbyter who was condemned in 325 A.D. on the First Council of Nicea convened by Saint Emperor Constantine The Great. Being in a hardships Eustathius buried the head in a cave near Emesa (circa 810 – 820) and soon after a monastery was built on that place by God's providence.

In the year 452, St. John the Baptist appeared to Archimandrite Marcellus of this monastery and indicated where his head was hidden in a water jar buried in the earth. The relic was brought into the city of Emesa and was later transferred to Constantinople.

Saint_John_Head_Holy-relics-Caput-Sti-Joannis-Baptistae-Praecurssoris-Domini-1

The current pressumable relics of head of Saint John the Baptist kept in San Silvestro in Capite Rome

The head of John Baptist disappeared once again after it was transferred from Comana of Cappadocia during a period of Muslim raids (about 820) and was again hidden in the ground during a period of iconoclastic persecusion.
After the veneration of icons was restored in year 850, A vision was revelead by God to patriarch Ignatius of Constantinople (ruling on patriarchial throne in 847 – 857) saw a vision revealing the place where the head of saint John was hidden around y. 850. The patriarch as the order was then communicated about his vision to emperor Michael III, who sent a delegation to Comana, where the head was found. Soon after the head was transferred to city of Nyc and here on 25 of May it was placed in a church in emperor court in Constantinople. The Church feast of the Third Finding of Saint John Baptist head is established for celebration in the Eastern Orthodox Church on 25 of May.

Currently many small particles of Saint John Head are available for generation among many Eastern Orthodox and Roman Catholic Churches.

The-Face-of-saint-John-the-Baptist-in-Cathedral-of-Our_Lady-in-Amiens-Cathedral

The head is claimed to be in San Silvestro (Saint Silvester) in Capite in Rome or in Amiens Cathedral, said to have been brought from Constantinople by Wallon de Sarton as he was returning from the Fourth Crusade.
There are also some sources claiming that the real head of John the Baptist is buried in Turkish Antioch or Southern France.

Amiens_-_Cathedrale_Notre-Dame_France

Amiens Cathedrale Notre Dame France – one of most magnificent Gothic edifice in Europe.

During the French Revolution the kept Head in Amiens has been secretly hidden by the Amiens city Mayor in his own home to protect this sacred relic from the destruction (as many holy relics saints disappeared or have been destroyed) by the rebellious enraged crowds fighting for the rights of "Liberty, Equality, Fraternity" being the goals of the Masonic bortherhoods and many secret societies in France in that time.

Also a reliquary at the Residenz in Munich, Germany, is labeled as containing the skull of John the Baptist by Catholics. In history some sources claim the St. John used to be owned by Knight Templars
А piece of Saint John Baptist skull is held at the Eastern Orthodox Romanian skete Prodromos on Mount Athos.

Further on according to Church tradition saint Luke the Evangelist went to the city of Sebaste bringing with him the right hand of Saint John the Forerunner which was conducting numerous of miracles.

Some of the Relics of John the Baptist are said to be in the possession of the Coptic Orthodox Monastery of Saint Macarius the Great in Scetes, Egypt.
It is said John the Baptist's arm and a piece of his skull can be found at the Topkapı Palace in Istanbul, Turkey.

the_Holy-head-of-saint-John-the-Baptist-relics

It is said John the Baptist's arm and a piece of his skull can be found at the Topkapı Palace in Istanbul, Turkey.
At the time of Mehmed the Conqueror, the skull was held in Topkapı, while after his death, his stepmother Mara Branković, a Serbian princess, brought it to Serbia. It was then kept a while at the Dionisios monastery at Mount Athos, then the skull fragment was sent to a nearby island in order to prevent the outbreak of a plague; however, the Ottoman fleet seized it and delivered it to Hasan Pasha of Algeria, who held it in his home until his death. It was then returned to Topkapı. The skull is kept on a golden plate decorated with gold bands with gems and Old Serbian inscriptions. The plate itself is stored in a 16th-century rock crystal box.

The-Face-of-saint-John-the-Baptist-in-Cathedral-of-Our_Lady-in-Amiens-Holy-Relics

The face of St. John the Baptist, in the Cathedral of Our Lady in Amiens.

St. John's arm was brought from Antioch to Constantinople at the time of Constantine VII. It was kept in the Emperor's chapel in the 12th century, then in the Church of the Virgin of the Pharos, then in the Church of Peribleptos in the first half of the 15th century. Spanish envoy Clavijo reported that he saw two different arms in two different monasteries while on a visit to Constantinople in 1404. With the Fall of Constantinople, the Ottomans seized possession of it. In 1484, Bayezid II sent it the knights of Rhodes, while they held his brother Cem captive in return. In 1585, Murad III had the arms brought from Lefkosia castle to Constantinople (henceforth known as Istanbul). The arm is kept in a gold-embellished silver reliquary. There are several inscriptions on the arm: "The beloved of God" on the forefinger, "This is the hand of the Baptist" on the wrist, and "belongs to (monk) Dolin Monahu" on the band above the elbow.

In July 2010, a small reliquary was discovered under the ruins of a 5th-century monastery on St. Ivan Island, Bulgaria. Local archaeologists opened the reliquary in August and found bone fragments of a skull, a hand and a tooth, which they believe belong to st. John the Baptist, based on their interpretation of a Greek inscription on the reliquary.The remains have been carbon-dated to the 1st century. Currently The found relics are being placed for veneration in the sea resort town of Sozopol, Bulgaria in the Church of saint saint Cyril and Methodius.

Saint John the Baptist Holy Relics in Sozopol Bulgaria

Sozopol-Lithia-Lity-with-the-holy-relics-of-saint-John-the-Baptist
A Lity (Orthodox Vespers) in front of Saint Cyril and Methodius Church in Sozopol resort Bulgaria

There is much to be said in Saint Johns beheading and many great Theology books have been written on the topic however I hope the goal of this article to give a very brief overview for the ordinary people to know our human history over the last 2000 which is highly entangled with Christian faith succeeded to give a very brief overview on the history of the beheading of saint John the Baptist and the deep history across his holy relics veneration over the centuries.

As a closure for the Article I find worthly to share the sung troparion in the Church services glorifying of saint John the Forerunner in Old Bulgarian / Church Slavonic, Greek and English

TROPARION IN CHURCH SLAVONIC

Па́мять пра́веднаго с похвала́ми, тебе́ же довле́ет свиде́тельство Госпо́дне, Предте́че: показа́л бо ся еси́ вои́стинну и проро́ков честне́йший, я́ко и в струя́х крести́ти сподо́бился еси́ Пропове́даннаго. Те́мже за и́стину пострада́в, ра́дуяся, благовести́л еси́ и су́щим во а́де Бо́га, я́вльшагося пло́тию, взе́млющаго грех ми́ра и подаю́щаго нам ве́лию ми́лость.

TROPARION IN GREEK

Μνήμη Δικαίου μέτ' ἐγκωμίων, σοὶ δὲ ἀρκέσει ἡ μαρτυρία τοῦ Κυρίου Πρόδρομε· ἀνεδείχθης γὰρ ὄντως καὶ Προφητῶν σεβασμιώτερος, ὅτι καὶ ἐν ῥείθροις βαπτίσαι κατηξιώθης τὸν κηρυττόμενον. Ὅθεν τῆς ἀληθείας ὑπεραθλήσας, χαίρων εὐηγγελίσω καὶ τοῖς ἐν ᾍδῃ, Θεὸν φανερωθέντα ἐν σαρκί, τὸν αἴροντα τὴν ἁμαρτίαν τοῦ κόσμου, καὶ παρέχοντα ἡμῖν τὸ μέγα ἔλεος.

TROPARION (TONE 4)
O Prophet and Forerunner of the coming of Christ, in spite of our eagerness to render you due honor, we fall short when singing your praise. Your glorious birth saved your mother from the shame of barrenness, returned to your father the power of speech, and proclaimed to the world the Incarnation of the Son of God.

KONTAKION (TONE 3)
The woman who had been barren becomes fertile and gives birth today to the Forerunner of Christ. He is the greatest and last of the prophets, for standing in the waters of the Jordon River, he placed his hands on Christ whom all the prophets had announced, and in so doing he became a prophet himself, a preacher and a forerunner of the Word of God.

Tags: Amiens Cathedrale Notre Dame France, Below, Christ, death, feast, found, Herodos Antipa, hidden, holy relics, multiple times, place, small article, years
Posted in Christianity | 1 Comment »

Howto install GeoTrust RapidSSL certificate on Debian Lenny / Squeeze / Wheezy Linux

Thursday, March 25th, 2010

I faced the task of generating official Validated SSL Certificates by in mydaily duties as a System Administrator at cadiaholding.com . Though generating self-signedSSL certificate is comparatively easy task. It was a pain in the ass setting Apache version 2.2.9-10+lenny6to correctly serve pages through https:// protocol over openssl version 0.9.8g-15+lenny6.I’ll try to go through the whole process of Generating the certificate in order to help some other Debianusers out there to face less setbacks in such a simple task as installing a Trusted SSL Certificate issued(bought) by RapidSSL. Even though this article will mostly deal with SSL certificate issued by RapidSSL,it should be not a problem to apply this methodogy with Verisign or some of the other Geotrust issuedSecure Socket Layer certificates.

In generating the Validated certficate I used enom which is a domain name,ssl certificates, email and hosting company whole-saler.
Fron emon’s website after logging in and using the web interface, there are two major things required to fill inin order to issue your Trusted SSL certificate.

1. Fill in in a form a CSR file, this is usually generated on the Linux server using the openssl.
To issue the CSR file required by Enom use the following commands:

a. First we generate an DES3 RSA encrypted key which we will use next to generate the opeensl CSR file required by ENOM.
debian:~# /usr/bin/openssl genrsa -des3 -out www.domain.com.key 2048 Enter pass phrase for www.domain.com.key:

You’ll be required to fill in a pass-phrase that will be later be required to fill in before Apache servers starts or restarts,so make sure you fill something you either remember or you keep the password stored in a file.
You have to change also the www.domain.com.key in accordance with your domain name.
Now as we already have a proper generated DES3 RSA key afterwards it’s necessery to generate the CSR file with the openssl command line frontend.
So here is how:

debian:~# /usr/bin/openssl req -new -key /home/hipo/www.domain.com.key -out /home/hipo/www.domain.com.csr

Again in the above example change all the paths and file names as you wish.
It’s necessery that the end user fill in a number of questions related to the Certificate Signing Request.
Herein I’ll list what kind of prompts will emerge after executing the above command:

Enter pass phrase for /home/hipo/www.domain.com.key: You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Country Name (2 letter code) [AU]: State or Province Name (full name) [Some-State]: Locality Name (eg, city) []: Organization Name (eg, company) [Internet Widgits Pty Ltd]: Organizational Unit Name (eg, section) []: Common Name (eg, YOUR name) []: Email Address []: Please enter the following 'extra' attributes to be sent with your certificate request A challenge password []: An optional company name []:

Note that you’ll hav eto fill in the pass phrase previously entered during the generation of the www.domain.com.key file.
In case if you’d like to read more thoroughly on the subject of howto create a Certificate Signing Request or (CSR) as we called it on multiple times, you can read About Certificate Signing Request (CSR) Generation Instructions – Apache SSL more in depth here

2. Hopefully following the above instructions you’ll now have a file named www.domain.csrJust open the www.domain.scr and copy paste it’s content to the ENOM website CSR * webform.

3. Further on select your Webserver type on Enom’s website:In our case we have to select Apache + ApacheSSL

4. What follows next is filling in your company contact information This is also required for proper certificate generation, you have to think twice before you fill in this data, take a note this can’t be changed later on without issuing a brand SSL new certificate.

Apart from the 3 major above requirements to fill in Enom there are some few more radio buttons to use to make some selections according to your personal preferences, however I won’t take time to dig in that and I’ll leave this to you.
After all the above is fulfilled you’ll have to submit your certificate details and choose an email address to which you will receive in a minute a RapidSSL Certificate Request Confirmation

Following a link from the email, will show you some basic information about the certificate about to be generated. That’s your final chance to cancel the issued Trusted Certificated.
If you’re absolutely sure the information about to enter the certificate is correct then you’ll have to follow a link and approve the certificate.

You’ll be informed that you’ll receive your certificate either through Certifier website (e.g. Enom’s website) or via another email.
I thought it’s more probable I receive it via email but anyways I was wrong. More thank 4 hours has passed since the certificate was issued and is available via Enom’s interface but I haven’t received nothing on my mail.
Therefore my friendly advice is to check about your brand new shiny Trusted Certificate on Emom’s website. I had mine ready in about 10 minutes after the CSR was issued.

Assuming that you’ve succesfully obtained the SSL Trusted certificate from RapidSSL what follows is setting up the certificate.
Initially I tried using documentation from RapidSSL website called Installing your SSL Certificate / Web Server Certificate / Secure Server Certificate from RapidSSL.com
I tried to configure one of my Virtualhost as shown in their example inserting in my /etc/apache/sites-available/www.domain.com file, few directives within the VirtualHost something like the shown below

SSLEngine on # domain.com.crt cointains the Trusted SSL certificate generated and obtained by you from RapidSSL SSLCertificateFile /etc/apache2/ssl/www.domain.com.crt # www.domain.com.key contains the file used to generate the CSR file as described earlier in this post SSLCertificateKeyFile /etc/apache2/ssl/www.domain.com.key

It is also possible insetad of using the SSLCertificateFile and SSLCertificateKeyFile directvies directives in Apache config to use:

Another alternative is to use

SSLCertificateFile /etc/ssl/certs/your-domain-name.crt
SSLCertificateKeyFile /etc/ssl/certs/your-domain-name.key
SSLCACertificateFile /etc/ssl/certs/gd_bundle.crt

The key file is the one generated earlier on the server and handed to the SSL regisrar, the files:

your-domain-name.crt and gd_bundle.crt files are provided by RapidSSL or from whatever SSL registrater the SSL was purchased.

After trying the above configuration and restarting apache with:

/etc/init.d/apache2 restart

Apache failed to start, it might be helpful to somebody out there the error I had in my apache error.log:
The error.log red the following:

[warn] RSA server certificate is a CA certificate (BasicConstraints: CA == TRUE !?)

After some 30 minutes or an hour of Googling on the error I came to the conclusion that the error is caused, becauseApache is supposed to work with .PEM files instead of the classical .CRT and .KEY files asnormally approached in most of the other Unix operating systems.

It took me a bit more of reading on the internet to find out that actually the .pem files so widely adopted in Debian simply contain both the www.domain.com.key file and the www.domain.com.crt key simply pasted one after another, this I also observed from the default Apache self-signed certificate that I believe comes with debian /etc/apache2/ssl/apache.pem .
So I copied both the content of my www.domain.com.key and www.domain.com.crt and store it in one file:
/etc/apache2/ssl/www.domain.com.pem

Also the following configuration:
SSLEngine on SSLCertificateFile /etc/apache2/ssl/www.domain.com.pem
had to go in your
/etc/apache2/sites-enabled/www.domain.com

Last thing that’s left is to restart your Apache;

/etc/init.d/apache2 restart

Apache will prompt you for your certificate password entered by you during the www.domain.com.key generation. Type your password and with a bit of luck and hopefully with God’s help you’ll be having a Trusted Certificate on your webserver.

Last step is to check if the certificate is okay accessing your domain https://www.domain.com.

Well this is the end of the article, hope you enjoy.If you do please leave your comments, any corrections are also welcomed 🙂

Tags: administrator, apache servers, apache version, apache2, certficate, certificate, Certificates, com, company, contact, csr, DES, des3, domain, domain name, email, emon, enom, file, generation, geotrust, hipo, Hopefully, hosting company, howto, information, key, key file, lenny, linux server, methodogy, multiple times, necessery, official, openssl, pain in the ass, password, phrase, rapidssl, RSA, setbacks, socket layer, ssl certificate, system administrator, usr bin, verisign, web interface, whole saler, www
Posted in Computer Security, System Administration, Web and CMS | 30 Comments »

Restoring sudden disappeared (deleted) Android Phone Contacts and Data on ZTE, Samsung, LG and Huawei mobile phones

Tuesday, May 20th, 2014

dr-fone-recover-android-and-iphone-broken-accidently-lost-disappeared-contacts-and-lost-data.

For 5th time since I both my ZTE – Blade III Smartphone – Android OS powered mobile, my Phone Contacts disappear. This is pretty annoying and I'm more and more convinced Android is not yet really ready for the masses..

I've enabled on Phone Sync my contacts with Google but even though I can see all my mobile phones visible via – http://gmail.com/contacts/, my attempts to use Android's:

Settings -> Account & Sync -> Gmail -> (Sync Contacts and Sync people details

– to synchronize Gmail Phone and E-mail Contacts to Phone from Gmail did not produced a positive result.

To Solve the situation earlier I've used on multiple times Contacts Sync Fix – Tool to help fix some (not all) of the causes of "Sync is currently experiencing problems. It will be back shortly." message from Google contacts sync. However this time Contacts Sync Fix tool did not revert back my missing Phone Contacts.

I googled to see what kind of Phone Contacts recovery software is there and stumbled on Wondershare Dr. Fone – World’s 1st Andorid, iPhone, iPad and iPod touch data recovery software for personal users. Here is what Dr. Fone does:

Recover data from Andorid and iOS devices, iCloud backup and iTunes backup (including encrypted one).
Available to recover messages, contacts, photos, videos, notes, call history, calendars, voicemail, voice memos, reminders, bookmarks and other documents.
Recover App data like WhatsApp, Instagram, Flickr, iPhoto, iMovie, Kindle, etc.
Compatible with the latest iOS 7 and supports iPhone 5S/5C/5/4S/4/3GS, all iPads and iPod touch 5/4.
Regain lost data due to deletion, device crashed or broken, OS bugs, device loss, jailbreak, iOS upgrade and factory.

Dr. Fone is proprietary software and quite expensive costs a lot 50$ bucks! But Cracked versions are circulating the net on many torrents, there is a cracked version of Dr. Fone on http://ThePiratebay.se
Dr. Fone is very easy to use, to recover my lost Android Phones data, I had to:

1. Connect my ZTE through USB cable to a Windows PC (where I have installed Dr. Fone
2. Scan the device for my lost data, through the program GUI interface (This make a lot of data be seen including all my Videos, Pictures, Contacts etc.)
3. Press the recover data button and choose a destination folder where found files to recover will be stored.

A Note to make here is, Dr.Fone requieres that USB Debugging has to be Switched on from the Phone (on Android version 3.0 to 4.1) – "Settings" -> Click "Developer options" -> Check "USB debugging".
On Android 4.2 and newer to activate USB Debugging – "Settings -> About Phone -> Developer Options -> (Check) USB Debugging"

Found Phone Contacts, are exported into .CSV file, HTML and VCard. So to import back my recovered contacts from my Mobile, I transferred VCard file extension (VCard.VCF) into my Mobile Phone via attached USB cable and from Phone's File Manager clicked on VCard.vcf to import my contacts.

So far so good, I had all my Phone Contacts recovered and again accessible from Android's PhoneBook app, however because of import for some reasons I had many of contacts Imported and visible now multiple times, as well as a lot of junk contacts like just Phone Numbers without names. To resolve that I first thought of manually reviewing and deleting only junk and duplicate addressbook contacts. As I didn't know how to select and delete multiple Phone Contacts? – using the phone default Phone Contacts app, I googled and found the Contact Remover app – a tiny program listing all your contacts letting you choose only those who would like to delete.

android-contact-remover-application-to-delete-multpile-contacts

Though this program was working, Contact selection was lagging and was really irritating, so I googled further to see if there is no Application to delete duplicate phone contacts in Android, finding – Duplicate Contacts – Simple tool to see and delete duplicate contacts.

delete-duplicate-contacts-phones-in-android

Once deleting the duplicate Phones (I had most of the phones occuring 5 times and there were number of email addresses wrongly parsed and imported into Contacts. I just had to delete a number of junky names but happily all my Phone Contacts were succesfully recovered! 🙂
Now because I already had a Contacts set to Sync with Google, once the Junky multiple duplicate contacts were stored on the Phone they started synchronizing with Google Contacts.
Google-Contacts-find-and-merge-delete-duplicate-contacts-and-phones

After synchronization was completed to clean my Google Account stored Phone number names and Contacts – since some time Google introduced a special functionality for that – Find and Merge Duplicates.
For future to Keep my contacts in Ordered and backupped to prevent such a complex future exercises, I was recommended MyPhoneExplorer by a colleague. MyPhoneExplorer supports backup (Export and Import) your Phone Contacts via USB, Bluetooth or even WI-FI on multiple devices, starting from Android Tablets, Android Mobile Phones and even old Nokia Symbian or other archaic Alcatel, LG etc. OS-ed mobile phones.

myphone-explorers-keep-regular-backups-of-your-mobile-phone-contacts

Tags: Android Phones, attempts, Contacts Sync Fix, cracked version, CSV, data, destination, Found Phone Contacts, Google Account, HTML, Huawei, import, LG, lot, multiple, multiple times, OS, phone, Phone Contacts, Press, proprietary software, recovery, revert, software, succesfully, tiny program, USB, ZTE
Posted in Everyday Life, Mobile Phone Apps & Games, Various | 5 Comments »

Install Cacti on Debian, CentOS, SuSE and Gentoo Linux – Tracking graphically server performance

Thursday, July 10th, 2014

There are plenty of ways to track graphically server performance in Web (Nagios, Munin, Zabbix, OpenNMS) etc.
Though Munin and Nagios is a great choice for people who prefer simplicity in installation and maintanenance. If you're looking for customizable reports and time windows history on how server (network, cpu, memory, hdd or services) behaved you will probably need to consider using Cacti.

Cacti is a complete network graphing solution designed to harness the power of RRDTool's data storage and graphing functionality. Cacti provides a fast poller, advanced graph templating, multiple data acquisition methods, and user management features out of the box. All of this is wrapped in an intuitive, easy to use interface that makes sense for LAN-sized installations up to complex networks with hundreds of devices.

Cacti is very much like Munin except Cacti gives you the possibility to select and show statistics for periods in time, making it very convenient to check how server/servers services performed historically in time.

Cacti is a bunch of PHP scripts that gives a nice frontend to the famous RRDTool, it stores all gathered service statistics information in a MySQL database. Web frontend queries the SQL to craete graphs and populate them with data . Cacti has SNMP protocol support for those used to creating traffic graphs with MRTG.
Its easily extendable, there is a "tree view", which allows you to put graphs onto a hierarchical tree for organizational purposes, there is a user based management tool built in so you can add users and give them rights to certain areas of cacti. You can create users that can change graph parameters, while others can only view graphs. Each user also maintains their own settings when it comes to viewing graphs.

Cacti has prebuilt packages for major Linux distributions as well as FreeBSD ports.
Native packages are available via distros repositories for:

Fedora, Debian, SuSE and Gentoo Linux

To work properly Cacti depends on following external packages to be installed:

Apache HTTPD
MySQL
PHP
RRDTool
Net-Snmp
PHP-MySQL module
PHP-Snmp module

1. Install Cacti on Debian 7 or Ubuntu Linux

On Debian based distributions Installation of Cacti is super easy. If you're a debian admin it will save you time:

Cacti package is very well crafted on Debian to have it installed all you have to do is apt-get it:

apt-get install --yes cacti snmpd

You will be prompted a couple of screens requiring you to fill in your MySQL root password (necessery for connection and import of cacti.sql database and tables skele and creation of new user with which cacti will query the db-server.)

mv /etc/snmp/snmpd.conf /etc/snmp/snmpd.conf.bak

vi /etc/snmp/snmpd.conf

# this will make snmpd listen on all interfaces
agentAddress udp:161

# a read only community 'myCommunity' and the source network is defined
rocommunity myCommunity 192.168.10.5
rocommunity myCommunity 127.0.0.1

sysLocation Earth
sysContact email@your-domain.com

You will be next prompted to fill in user and password for cacti user, type whatever you like it to be and save the password somewhere for later use.

After installing it Cacti cron will automatically be added to collect cacti statistics on every 5 minutes, e.g.:

cat /etc/cron.d/cacti

MAILTO=root
*/5 * * * * www-data php /usr/share/cacti/site/poller.php >/dev/null 2>/var/log/cacti/poller-error.log

Cacti Cron job script has to run multiple times until Cacti Graphics gets generated so wait for it 5, 10 minutes or run it manually with www-data user credentials

poller.php is making queries to MySQl and connecting to SNMP service on localhost querying information for configured monitoring from Console

rrd data is stored in /var/lib/cacti/rra

You will have to also create manually following files which will be used by cactito store rrd data.

In case of issues with cacti check that permissionsof filesinthere are correct

To do su run:
su www-data -s /bin/sh

touch /var/lib/cacti/rra/localhost_mem_buffers_3.rrd /var/lib/cacti/rra/localhost_load_1min_5.rrd /var/lib/cacti/rra/localhost_users_6.rrd /var/lib/cacti/rra/localhost_proc_7.rrd

2. Install Cacti on Fedora / CentOS Linux

Installation on Fedora is a little pain in the ass as there is plenty of things to do manually to have cacti working
a) Install pre-required RPM packages

yum -y install mysql-server mysql php-mysql php-pear php-common php-gd php-devel php php-mbstring php-cli php-snmp php-pear-Net-SMTP php-mysql httpd net-snmp-utils php-snmp net-snmp-libs

Note! that if you already have a server with Apache + PHP configured many of above packages will report to be installed.

Whether mysql-server was not existing previously, change your MySQL password to a new-one

mysqladmin -u root password NEWPASSWORD

Create database for Cacti to store collected data:

mysql -u root -p -e 'create database cacti'

Connect to mysql server and create cacti user:

mysql> GRANT ALL ON cacti.* TO cacti@localhost IDENTIFIED BY 'very-secret-password'; mysql> FLUSH privileges;

Create SNMP configuration for Cacti and start SNMP

vi /etc/snmp/snmpd.conf

and paste into it:

com2sec local     localhost           public
group MyRWGroup v1         local
group MyRWGroup v2c        local
group MyRWGroup usm        local
view all    included .1                               80
access MyRWGroup ""      any       noauth    exact all    all    none
syslocation Unknown (edit /etc/snmp/snmpd.conf)
syscontact Root (configure /etc/snmp/snmp.local.conf)
pass .1.3.6.1.4.1.4413.4.1 /usr/bin/ucd5820stat

Start SNMPD and make it start automatically on Fedora boot

# /etc/init.d/snmpd start # chkconfig snmpd on

Test whether SNMP server is properly working and returning info:

snmpwalk -v 1 -c public localhost IP-MIB::ipAdEntIfIndex

You should get output like:

IP-MIB::ipAdEntIfIndex.192.168.10.5 = INTEGER: 2
IP-MIB::ipAdEntIfIndex.217.xx.xx.xxx = INTEGER: 3
IP-MIB::ipAdEntIfIndex.127.0.0.1 = INTEGER: 1

Install Cacti package:

yum -y install cacti

Import Cacti initial SQL data:

– Check in RPM package where is default cacti.sql

rpm -ql cacti | grep cacti.sql

/usr/share/doc/cacti-0.8.7i/cacti.sql

mysql -u cacti -p very-secret-password < /usr/share/doc/cacti-0.8.7i/cacti.sql

Configure Cacti:

vi /etc/cacti/db.php

/* make sure these values refect your actual database/host/user/password */
$database_type = "mysql";
$database_default = "cacti";
$database_hostname = "localhost";
$database_username = "cacti";
$database_password = "very-secret-password";
$database_port = "3306";

A small note to make here is Cacti can be configured to also work with MariaDB backend but, I haven't tried it so far …

Set Cacti cronjob to periodically update cacti stats

vi /etc/cron.d/cacti

*/5 * * * * cacti /usr/bin/php /usr/share/cacti/poller.php > /dev/null 2>&1

Make Cacti accessible from Web – add cacti alias in Apache

vi /etc/httpd/conf.d/cacti.conf

#
# Cacti: An rrd based graphing tool
#
Alias /cacti    /usr/share/cacti

<Directory /usr/share/cacti/>
        Order Deny,Allow
        Deny from all
        Allow from 10.0.0.0/8
</Directory>

Restart Apache to load new config

/etc/init.d/httpd restart

3. Install Cacti on Gentoo Linux

Modify your /etc/make.conf so USE="" options looks like this:

USE="symlink mmx sse sse2 bash-completion vhosts xml sockets snmp"

Install Cacti, PHP and Apache and webapp-config with emerge (Gentoo package manager)

emerge apache php cacti webapp-config

Hopefully all should get installed properly, if you get an error like "Could not read settings from webapp-config" run:

cd /etc/ etc-update

Create a vhost if you don’t already have one and then run the following. Then install cacti to the vhost using webapp-config. Remember to change the -h option to reflect the name of your vhost and that you may need to set a different cacti version number if cacti has been updated since I posted this article.

webapp-config -I -h yourdomain.com -d cacti cacti 0.8.7e-r1 mysqladmin -p --user=root create cacti mysql -p --user=root cacti < /var/www/yourdomain.com/htdocs/cacti/cacti.sql mysql -p --user=root mysql GRANT ALL ON cacti.* TO cactiuser@localhost IDENTIFIED BY 'somepassword'; flush privileges;

Add a cron entry to your tab to get Cacti to update. Add the following entry to your crontab updating the path as needed

crontab -u root -e

And paste

*/5 * * * * apache /usr/bin/php /var/www/yourdomain.com/htdocs/cacti/poller.php > /dev/null 2>&1

Now open in a browser:

http://yourdomain.com/cacti

After logging, Click Graphs tab (located on left top) corner and after a while your should see the graphs start to be drawn.

3. Install Cacti on OpenSuSE

Install Cacti from Yast suse package manager. A very good complete instructions on OpenSuSE installation is here

4. Configuring Cacti to draw statistics for host

So far so good now Cacti is installed but to make it generate its configuration its necessery to do some configuration tampering.
Cacti's configuration is pretty obscure and it needs time and a couple of tries until you have learned how to configure statistics generation.
Cacti's web interface is divided by two major parts Console (From which all kind of configuration is done) and Graphs (From here all confugured graphs can be expected, and information for time periods setted services, memory, CPU, hard drive, etc. can be obtained).

To Complete Cacti installation and configuration using a web-browser

Access URL:

http://your-domain.com/cacti/

cacti-user-login-monitoring-your-server-interactively-in-web-initial-login-screen

By default Cacti is configured to have user admin with password admin. Login to Cacti and you will be prompted to change default password – set it to whatever you like.

cacti_firstscreen_screenshot_linux_debian

Configure Cacti to do SNMP data collection

Cacti will use SNMP protocol to obtain server traffic statistics and visualize them.

Click on:

Devices -> Add

Fill in only fields (and let the rest as it is):

Description: Pc-freak.net
hostname: 127.0.0.1
Select for SNMP Version (drop down menu): SNMP Version 2

and then click on

Create button (located right down)

cacti-snmp-linux-good-config-options-screenshot
Here I've used localhost (127.0.0.1) as a hostname, because localhost server is to be monitored for remote monitoring of cacti set the REMOTE IP or hostname instead.

In field Associated Data queries add Data query for:

SNMP – Get Mounted Partitions
SNMP – Get Process Information
SNMP – Interface statistisc

P.S. Use the Add button for all this ..

Create SNMP graphs collection for first time

Click on Create Graphs for this Host see – top right side.

create-graphs-for-this-host-linux-cacti-screenshot

create-graphs-for-this-host-cacti-debian-redhat-rhel-centos-linux

Select on SNMP – Interface Statistics

Choose a graph type (for exmp. In/Out bytes with total bandwidth)

To Finalize click Create button

Create Graphs for this host

Creating Graph Trees

Use Console -> Graph Trees -> Add -> Sample Tree

Choose:

Setting type: Manual Ordering (No Sorting)

Tree Item Type: Graph

Graph: Server Name – Traffic eth0

Then press Create (button)

To verify the graphs are drawed corretly in a while go to:

Graphs -> Sample Tree (or whatever you named the tree during creation)

Interface Graphs and 64-bit Counters

By default, Cacti uses 32-bit counters in SNMP queries. 32-bit counters are sufficient for most bandwidth graphs, but they do not work correctly for graphs greater than 100 Mbps. If it is known that the bandwidth will exceed more than 100 Mbps, it is always advisable to use 64-bit counters. Using 64-bit counters is not hard at all.

Note! It takes usually around 15 minutes for Cacti to populate new graphs. There is no option to run cacti on cron manually but you have to wait

After some time of Cacti running, you will end up with nice graphics like this:

cacti-example-output-statistics-debian-centos-linux

If you get some issues and even after 15 / 20 minutes you still don't have the graphics drawed to debug the issues check for errors in:

/var/log/apache2/error.log – If there are missing rra files this should be logged here
/var/log/cacti/cacti.log
/var/log/cacti/poller-error.log – If the reason for not drawing the graphics is permissions errors will be here
/var/log/rrd.log – should contain rrd related errors

Important thing to mention is Cacti is requiring php exec() option to be functional. On hosts which has disabled exec(); function for security reasons cacti will fail to produce graphs.

References and thanks to:

http://xmodulo.com/2013/11/monitor-linux-servers-snmp-cacti.html http://openmaniak.com/cacti_tutorial.php http://www.cacti.net/ http://www.debianhelp.co.uk/cacti.htm

Tags: apache php, CentOS, com, community, crontab, Install Cacti, login, multiple times, php, Select, server performance, SNMP, var, www
Posted in Monitoring, Networking, System Administration, Web and CMS | No Comments »

ClamTK Linux Desktop Anti-Virus program – Checking Windows mapped drives with ClamTK

Thursday, June 20th, 2013

Linux desktop graphical program to scan for-viruses ClamTK clamav frontend application

In general Linux has fame for being Virus Free Operating System. During the 13 last years as dedicated GNU / Linux user, I've seen Linux servers with binaries infected with Viruses, however the hosts, were severely messed hosts because noone updated them on time and script kiddy crackers has "hacked" multiple times. In lifetime one of my old testing computers got infected with Virus because of my mistake of running "suspicious" pre-compiled "cracker" software binaries with no MD5 verification and "questionable" websites…
I share this story because, I want to beat-up the Myth that Linux cannot have Viruses. It CAN but not very likely to happen 🙂

As a Desktop user over the last 10 years, even though I installed plenty of packages from third party sources and never happened to infect my computer with Virus – or at least if I infected I never knew it. A lot of popular MS-Windows Anti-Virus programs, has already ports for Linux. Just to mention few non-free Linux AV software providing install binaries;

Avast
BitDefender
AVG
Dr. Web

Though risk of Viruses on Linux is so tiny, it is useful to have ANTI-Virus Software to check files received from Skype, E-mails and onse downloaded with Browser. I always prefer so until now I used Clamav Antivirus to keep an eye periodically on my Desktop Linux host and servers running mail servers (those who run Mail Servers know how useful is Clamav in stopping tons of E-mail attached Malware Viruses and Trojans).

I use mostly Debian Linux, so on every new server or Desktop one of first things I did was to install it, i.e.:

# apt-get --yes install clamav ...

Before I knew Clamav AV for Windows has GUI, anyways till recently I didn't know if there is some kind of free software AV Graphical frontend for Linux. I just found out about ClamTK

Linux Free Antivirus ClamTk clamav Virus Scanner graphical frontend

ClamTK is available in most Linux distributions from default package repositories

On Debian and Ubuntu to install it run common:

debian:~# apt-get --yes clamtk ...
On Fedora and CentOS Linux to install:

[root@fedora ~]# yum -y install clamtk ...
Its best to run it as root superuser (or via sudo) to make ClamTK able read all files or mounts on system:

hipo@debian:~$ sudo clamtk
ClamTK is very simple to use and there are only few configuration options;
clamtk desktop linux free antivirus startup preferences

clamtk scan for viruses linux gui proxy

linux Anti-Virus Desktop graphics easy to use AntiVirus ClamTK preferences screenshot

ClamTK is very useful when used with mounted Samba Shared (Mapped) Windows drives to scan for Viruses and malware, i.e, after mounting share using cmd like:

# smbmount //192.168.2.28/projects /mnt/projects -o user=USERNAME

Tags: anti virus, binaries, crackers, fame, hosts, Linux, linux user, mistake, multiple times, script kiddy, servers, testing computers, virus program, Viruses
Posted in Linux and FreeBSD Desktop, System Administration, Various | 4 Comments »

☩ Walking in Light with Christ – Faith, Computing, Diary

Posts Tagged ‘multiple times’

Little known facts about the dedication to Saint Martyr George The Glory-Bringer and his veneration across contries and religions

Saint Petka Paraskeva of Bulgaria of Epivates Thracia (famous as St. Petka of Tarnovo) feast day 14 October

The short Living of Saint Petka of Bulgaria (of Epivates)

Tropion 4-th voice

The beheading of Saint John the Baptist feats in Eastern Orthodox Church – A short history of saint John Forerunners Holy Relics

Howto install GeoTrust RapidSSL certificate on Debian Lenny / Squeeze / Wheezy Linux

Restoring sudden disappeared (deleted) Android Phone Contacts and Data on ZTE, Samsung, LG and Huawei mobile phones

Daily Bible quote

GET ARTICLE UPDATES

Useful blog? Help it:

Links to Other Places

Recent Posts

Ads

Categories

About Myself

Recent Comments

Top Post Views

blogtopsites