Saint martyr Trendafil of Starazagora (Starozagorski), little known Bulgarian saint martyred in XVI century


August 8th, 2022

In memory of the holy martyr Triandafil of Stara Zagora ( Starozagorski )

saint-Nikodimos-agiogarithes-sveti-Nikodim-Svetogorec

According to Venerable Nicodemus Agiorite, "Synaxaristis", Constantinople, 1845, and "Neon Martyrologion", Athens, 1961.

Holy Martyr Triandafil was very young Bulgarian, about 18 years old, a native of Stara Zagora, and he was a sailor.

svetogorskata-ikona-na-sv.Triandafil-ot-hrama-sv.Georgi-v-Zagora-saint-Trendafil

Suffered as a martyr for his Christian faith in Constantinople on August 8.

For the year of his death, Venerable Nicodemus Hagioritis in his work "Synaxarium" published in (Tsarigrad / Constantinople 1845) indicates year 1570 as a year of his martyrdom, and in his other work "Neon Martyrologion" (newly published in Athens in 1961) as a year of martyrdom 1680.

His memory was celebrated on his day (August 8) every year and the tradition continues today in the Bulgarian Orthodox Church as saint Trendafil is one of the constellation of Bulgarian saints.

Nicodemus Hagiorite reports that his life was written by the biographer Ioannis Cariophilis, but it seems that he did not have the original of the living at hand to include in his works.

Ikona-Sveti-Trendafil-ot-hrama-na-sveteca-v-Zagora.

As Bulgaria has been under a Spiritual and Physical slavery both the country and the Church has been under yoke, the country under the yoke of ottoman turks and the Church under the yoke of Greek slavery, only two verses are preserved in honor of the martyr (as the Greeks custom do), those two verses are given by Venerable Nicodemus in his "Synaxarion":

The name Trendafil literally translated is the flower Rosa multiflora.

Thus the glorification verses in the sinaxarion says:

Triandafil  appeared as the new trendafil (Rosa multiflora),
reddened by the flow of his blood.

So far, nothing else is known about this martyr of ours.

© Living of of the saints, translated into Bulgarian from the Church Slavonic text of Cheti-minei ("Chety-Minei") of St. Demetrius of Rostov.

How to update expiring OpenSSL certificates without downtime on haproxy Pacemaker / Corosync PCS Cluster


July 19th, 2022

pcm-active-passive-scheme-corosync-pacemaker-openssl-renew-fix-certificate

Lets say you have a running PCS Haproxy cluster with 2 nodes and you have already a configuration in haproxy with a running VIP IP and this proxies
are tunneling traffic to a webserver such as Apache or directly to an Application and you end up in the situation where the configured certificates,
are about to expire soon. As you can guess having the cluster online makes replacing the old expiring SSL certificate with a new one relatively easy
task. But still there are a couple of steps to follow which seems easy but systemizing them and typing them down takes some time and effort.
In short you need to check the current certificates installed on the haproxy inside the Haproxy configuration files,
in my case the haproxy cluster was running 2 haproxy configs haproxyprod.cfg and haproxyqa.cfg and the certificates configured are places inside this
configuration.

Hence to do the certificate update, I had to follow few steps:

A. Find the old certificate key or generate a new one that will be used later together with the CSR (Certificate Request File) to generate the new Secure Socket Layer
certificate pair.
B. Either use the old .CSR (this is usually placed inside the old .CRT certificate file) or generate a new one
C. Copy those .CSR file to the Copy / Paste buffer and place it in the Website field on the step to fill in a CSR for the new certificate on the Domain registrer
such as NameCheap / GoDaddy / BlueHost / Entrust etc.
D. Registrar should then be able to generate files like the the new ServerCertificate.crt, Public Key Root Certificate Authority etc.
E. You should copy and store these files in some database for future perhaps inside some database such as .xdb
for example you can se the X – Certificate and Key management xca (google for xca download).
F. Copy this certificate and place it on the top of the old .crt file that is configured on the haproxies for each domain for which you have configured it on node2
G. standby node1 so the cluster sends the haproxy traffic to node2 (where you should already have the new configured certificate)
H. Prepare the .crt file used by haproxy by including the new ServerCertificate.crt content on top of the file on node1 as well
I. unstandby node1
J. Check in browser by accessing the URL the certificate is the new one based on the new expiry date that should be extended in future
K. Check the status of haproxy
L. If necessery check /var/log/haproxy.log on both clusters to check all works as expected

haserver_cluster_sample

Below are the overall commands to use to complete below jobs

Old extracted keys and crt files are located under /home/username/new-certs

1. Check certificate expiry start / end dates


[root@haproxy-serv01 certs]# openssl s_client -connect 10.40.18.88:443 2>/dev/null| openssl x509 -noout -enddate
notAfter=Aug 12 12:00:00 2022 GMT

2. Find Certificate location taken from /etc/haproxy/haproxyprod.cfg / /etc/haproxy/haproxyqa.cfg

# from Prod .cfg
   bind 10.40.18.88:443 ssl crt /etc/haproxy/certs/www.your-domain.com.crt ca-file /etc/haproxy/certs/ccnr-ca-prod.crt 
 

# from QA .cfg

    bind 10.50.18.87:443 ssl crt /etc/haproxy/certs/test.your-domain.com.crt ca-file /etc/haproxy/certs

3. Check  CRT cert expiry


# for haproxy-serv02 qa :443 listeners

[root@haproxy-serv01 certs]# openssl s_client -connect 10.50.18.87:443 2>/dev/null| openssl x509 -noout -enddate 
notAfter=Dec  9 13:24:00 2029 GMT

 

[root@haproxy-serv01 certs]# openssl x509 -enddate -noout -in /etc/haproxy/certs/www.your-domain.com.crt
notAfter=Aug 12 12:00:00 2022 GMT

[root@haproxy-serv01 certs]# openssl x509 -noout -dates -in /etc/haproxy/certs/www.your-domain.com.crt 
notBefore=May 13 00:00:00 2020 GMT
notAfter=Aug 12 12:00:00 2022 GMT


[root@haproxy-serv01 certs]# openssl x509 -noout -dates -in /etc/haproxy/certs/other-domain.your-domain.com.crt 
notBefore=Dec  6 13:52:00 2019 GMT
notAfter=Dec  9 13:52:00 2022 GMT

4. Check public website cert expiry in a Chrome / Firefox or Opera browser

In a Chrome browser go to updated URLs:

https://www.your-domain/login

https://test.your-domain/login

https://other-domain.your-domain/login

and check the certs

5. Login to one of haproxy nodes haproxy-serv02 or haproxy-serv01

Check what crm_mon (the cluster resource manager) reports of the consistancy of cluster and the belonging members
you should get some output similar to below:

[root@haproxy-serv01 certs]# crm_mon
Stack: corosync
Current DC: haproxy-serv01 (version 1.1.23-1.el7_9.1-9acf116022) – partition with quorum
Last updated: Fri Jul 15 16:39:17 2022
Last change: Thu Jul 14 17:36:17 2022 by root via cibadmin on haproxy-serv01

2 nodes configured
6 resource instances configured

Online: [ haproxy-serv01 haproxy-serv02 ]

Active resources:

 ccnrprodlbvip  (ocf::heartbeat:IPaddr2):       Started haproxy-serv01
 ccnrqalbvip    (ocf::heartbeat:IPaddr2):       Started haproxy-serv01
 Clone Set: haproxyqa-clone [haproxyqa]
     Started: [ haproxy-serv01 haproxy-serv02 ]
 Clone Set: haproxyprod-clone [haproxyprod]
     Started: [ haproxy-serv01 haproxy-serv02 ]


6. Create backup of existing certificates before proceeding to regenerate expiring
On both haproxy-serv01 / haproxy-serv02 run:

 

# cp -vrpf /etc/haproxy/certs/ /home/username/etc-haproxy-certs_bak_$(date +%d_%y_%m)/


7. Find the .key file etract it from latest version of file CCNR-Certificates-DB.xdb

Extract passes from XCA cert manager (if you're already using XCA if not take the certificate from keypass or wherever you have stored it.

+ For XCA cert manager ccnrlb pass
Find the location of the certificate inside the .xdb place etc.

+++++ www.your-domain.com.key file +++++

—–BEGIN PUBLIC KEY—–

—–END PUBLIC KEY—–


# Extracted from old file /etc/haproxy/certs/www.your-domain.com.crt
 

—–BEGIN RSA PRIVATE KEY—–

—–END RSA PRIVATE KEY—–


+++++

8. Renew Generate CSR out of RSA PRIV KEY and .CRT

[root@haproxy-serv01 certs]# openssl x509 -noout -fingerprint -sha256 -inform pem -in www.your-domain.com.crt
SHA256 Fingerprint=24:F2:04:F0:3D:00:17:84:BE:EC:BB:54:85:52:B7:AC:63:FD:E4:1E:17:6B:43:DF:19:EA:F4:99:L3:18:A6:CD

# for haproxy-serv01 prod :443 listeners

[root@haproxy-serv02 certs]# openssl x509 -x509toreq -in www.your-domain.com.crt -out www.your-domain.com.csr -signkey www.your-domain.com.key


9. Move (Standby) traffic from haproxy-serv01 to ccnrl0b2 to test cert works fine

[root@haproxy-serv01 certs]# pcs cluster standby haproxy-serv01


10. Proceed the same steps on haproxy-serv01 and if ok unstandby

[root@haproxy-serv01 certs]# pcs cluster unstandby haproxy-serv01


11. Check all is fine with openssl client with new certificate


Check Root-Chain certificates:

# openssl verify -verbose -x509_strict -CAfile /etc/haproxy/certs/ccnr-ca-prod.crt -CApath  /etc/haproxy/certs/other-domain.your-domain.com.crt{.pem?)
/etc/haproxy/certs/other-domain.your-domain.com.crt: OK

# openssl verify -verbose -x509_strict -CAfile /etc/haproxy/certs/thawte-ca.crt -CApath  /etc/haproxy/certs/www.your-domain.com.crt
/etc/haproxy/certs/www.your-domain.com.crt: OK

################# For other-domain.your-domain.com.crt ##############
Do the same

12. Check cert expiry on /etc/haproxy/certs/other-domain.your-domain.com.crt

# for haproxy-serv02 qa :15443 listeners
[root@haproxy-serv01 certs]# openssl s_client -connect 10.40.18.88:15443 2>/dev/null| openssl x509 -noout -enddate
notAfter=Dec  9 13:52:00 2022 GMT

[root@haproxy-serv01 certs]#  openssl x509 -enddate -noout -in /etc/haproxy/certs/other-domain.your-domain.com.crt 
notAfter=Dec  9 13:52:00 2022 GMT


Check also for 
+++++ other-domain.your-domain.com..key file +++++
 

—–BEGIN PUBLIC KEY—–

—–END PUBLIC KEY—–

 


# Extracted from /etc/haproxy/certs/other-domain.your-domain.com.crt
 

—–BEGIN RSA PRIVATE KEY—–

—–END RSA PRIVATE KEY—–


+++++

13. Standby haproxy-serv01 node 1

[root@haproxy-serv01 certs]# pcs cluster standby haproxy-serv01

14. Renew Generate CSR out of RSA PRIV KEY and .CRT for second domain other-domain.your-domain.com

# for haproxy-serv01 prod :443 renew listeners
[root@haproxy-serv02 certs]# openssl x509 -x509toreq -in other-domain.your-domain.com.crt  -out domain-certificate.com.csr -signkey domain-certificate.com.key


And repeat the same steps e.g. fill the CSR inside the domain registrer and get the certificate and move to the proxy, check the fingerprint if necessery
 

[root@haproxy-serv01 certs]# openssl x509 -noout -fingerprint -sha256 -inform pem -in other-domain.your-domain.com.crt
SHA256 Fingerprint=60:B5:F0:14:38:F0:1C:51:7D:FD:4D:C1:72:EA:ED:E7:74:CA:53:A9:00:C6:F1:EB:B9:5A:A6:86:73:0A:32:8D


15. Check private key's SHA256 checksum

# openssl pkey -in terminals-priv.KEY -pubout -outform pem | sha256sum
# openssl x509 -in other-domain.your-domain.com.crt -pubkey -noout -outform pem | sha256sum

# openssl pkey -in  www.your-domain.com.crt-priv-KEY -pubout -outform pem | sha256sum

# openssl x509 -in  www.your-domain.com.crt -pubkey -noout -outform pem | sha256sum


16. Check haproxy config is okay before reload cert


# haproxy -c -V -f /etc/haproxy/haproxyprod.cfg
Configuration file is valid


# haproxy -c -V -f /etc/haproxy/haproxyqa.cfg
Configuration file is valid

Good so next we can the output of status of certificate

17.Check old certificates are reachable via VIP IP address

Considering that the cluster VIP Address is lets say 10.40.18.88 and running one of the both nodes cluster to check it do something like:
 

# curl -vvI https://10.40.18.88:443|grep -Ei 'start date|expire date'


As output you should get the old certificate


18. Reload Haproxies for Prod and QA on node1 and node2

You can reload the haproxy clusters processes gracefully something similar to kill -HUP but without loosing most of the current established connections with below cmds:

Login on node1 (haproxy-serv01) do:

# /usr/sbin/haproxy -f /etc/haproxy/haproxyprod.cfg -D -p /var/run/haproxyprod.pid  -sf $(cat /var/run/haproxyprod.pid)
# /usr/sbin/haproxy -f /etc/haproxy/haproxyqa.cfg -D -p /var/run/haproxyqa.pid  -sf $(cat /var/run/haproxyqa.pid)

repeat the same commands on haproxy-serv02 host

19.Check new certificates online and the the haproxy logs

# curl -vvI https://10.50.18.88:443|grep -Ei 'start date|expire date'

*       start date: Jul 15 08:19:46 2022 GMT
*       expire date: Jul 15 08:19:46 2025 GMT


You should get the new certificates Issueing start date and expiry date.

On both nodes (if necessery) do:

# tail -f /var/log/haproxy.log

Saint Sergius of Radonezh The WonderWorker (Sergij Radonezhki) a quick helper saint in life problems, pupils, students in educational hardships, exams and Orphans. A short living of father of Russian monasticism


July 6th, 2022

Saint-Sergius-of-Radonezh-icon-Russian-saint-hermit

Saint Sergius of Radonezh the Wonderworker (born in 1314, passed to Christ in 1392) is an ancient saint with enormous importance for the whole Christiandome and the Eastern Orthodox World.
He is perhaps the greatest ascet of the Russian land and a spiritual star who shone over the whole world sharing the and increasing the faith of Christ to shine upon many by his holy prayers. Saint Sergij Radonezhki is sometimes called in Russia the Abbot of the Russian land, for the reason he become the initiator (creator) and first abbot of the most notable and biggest monastery of Russia the Monastery of Saint Sergij of Radonezh in the XIV century.

In the Church service sung books, he is for that reason glorified calling him "a bright beacon of the Russian land, shining through its miracles like a second sun".  Saint Sergius Radonezhki's monastery was established in Glory of the Most Holy Trinity and the great ascet following the earlier example and spiritual tradition bequeted of the Great ascet Saint John of Rila (Ioan Rilski)  who lived in IX century in (876 – c. 946)  who established in the Rila mountain, the most famous Rila Monastey in Bulgaria, saint Sergius started his spiritual endurance in Russia as an ascet and did not initially planned to create a monastery, but God who sees everything seing his great ascetism sent him monks, willing to learn true spiritual life and that is how the Radonezh Monastery was born. Monastery quickly become aprototype of a new, amazingly pure and strict monastic life in Holy Russia, centered near in Moscow and the monastery which become a Laurel (the slavonic word translated as Lavra), (meaning the inhibitants of the monastery were exceeding 1000 monks) since y. 1744, become known in worldwide as Holy Trinity-Sergius Lavra.

Saint-Sergius-of-Radonezh-Lavra-Sveti-Sergieva-Lara-Holy-Trinity-monastery

Just as the spiritual heart of Bulgaria is located in the Rila Holy Monastery (Rilski Manastir), where are the the holy incorruptable relics of most glorified saint Reverend John of Rila the Wonderworker, the Sergius Trinity lavra has emerged as most important spiritual center of Russia Kingdom and later Russian Empire. What was the spiritual significance for preventing the Orthodox Christian faith and shedding hope via the dark ages of Ottoman Turks slavery of Bulgaria,  Reverend  Sergiy Radonezhki was for Russia especially in the hard times when Russia was a small country and fought for their freedom and independence from the Tatars and other surrounding nations, who were constantly destroying parts of the then small kingdom of Rus. Thousands of pilgrims with reverence and gratitude have come and continue to come to the Trinity-Sergius Lavra for worship, near the monastery now is established the city of Sergiyev Posad (inhabited today with about 103 000 people).

Saint_Sergius-of-Radonezh-the-Wonderworker-Sveti-Sergij_Radonezhki-Chudotvoretz-aliased-abbot-of-Russia-land

The shroud of the holy relics of Saint Sergius of Radonezh XV century

Prayers to St. Sergius of Radonezh are famous to protect you from any life problems.  People pray to the saint to protect children from bad influences and from failures at school. The prayer to the saint who was a model of humility helps in achieving humility and subduing the pride of ourselves and others.

Icons-and-frescoes-of-the-Holy-Trinity-Church-Radonezh-monastery-author-famous-iconograph-st-Andrey-Rublev

The very famous Holy Trinity unique icon held in main Church of Radonezh monastery painted by most famous Russian iconographer Saint Andrey Rublev

The icons and frescoes in the Trinity Cathedral in St Radonezh Lavra are unique piece of medieval art and were the work of Reverend Andrei Rublev and Daniil Chernyov in 1425. The main church icon that was painted in "praise of Rev. Sergius" is the old testamental visit the Holy Trinity to Abraham and Sarra icon – which is among the most famous and unique work of Russian icon painting. 

The Church feast of the Saint Sergij

The memory of St. Sergius of Radonezh is honored 4 times a year:

1. October 8 – on the day of his presentation to God
2. June 5 – together with the feast of the Rostov-Yaroslavl saints
3. October 18 – on the day of the discovery of the relics of St. Sergius of Radonezh
(interestingly just 1 day after in the Church calendar is 19 of October is the main feast of Saint John of Rila)
4. and on July 19 – together with the Radonezh Saints Church.

Saint_Sergius-of-Radonezh-holy-relics-raque

Reliquary with the incorruptable Holy Relics of Saint Sergij of Radonezh, kept in the Saint-Sergieva-Lavra monastery

Saint-Sergij-Radonezhki-holy-relics-opened-relics-raque

Opened reliquarium coffin with the incorruptable relics of saint Serigius of Radonezh (the feasts are opened for veneration by pilgrims 4 times a year during the saint feast veneration)

saint-Sergij-Sergius-Radonezhki-ikona-the-appearance-of-the-mother-of-God-to-saint-Sergius

The Apparition of the Mother of God to St. Sergius 16th century icon

 

Short Living (Biography) of Saint Sergius Radonezhki

Sergius of Radonezh was one of the most famous political and historical figures of the 14th century.
He is the founder of the Trinity-Sergius Lavra, teacher and mentor of many Russian saints canonized by the Church.
According to ancient tradition, mainly from his student Epiphanes, Rev. Sergius was born in 1314 in the village of Varnitsa, Rostov in the family of the famous Rostov boyars Kiril and Maria (who are also canonized as saints) and in whose memory there are Churches consecrated in Russia and Belarus.

His parents named him Bartholomew at birth.
Although the family was noble, they lived very modestly and were very religious.
At the age of 7, Sergius and his brothers started going to school, and while his two brothers' education came relatiely easily, it did not for Bartholomew as for bartholomew learning was hard.
Suffering for his hardships to study even though he put great efforts to do so, as he did not have any other means to become proficient pupil like his brothers, wishing to change this, the boy fervently prayed to God to give him reason and strength to advance in learning.

According to the chronicles, the boy once he was looking for the lost horses of his father met an old schimonk elder who was praying to God. The boy asked the old man to pray for his success in school. The the unusual bright monk prayed and blessed him. Since then, the boy began to progress in his studies quickly.

In 1330, Bartholomew's parents moved to the village of Radonezh, near Moscow. Soon his parents died, and he, together with his older brother Stefan, left the inheritance to the youngest brother and took up a hermit life in the nearby forest, where they built a cell, and then built a small church, which was consecrated with the name of the Holy Trinity.

Soon Stefan decided to abandon the hermit life and went to the Moscow Epiphany Monastery, where he became abbot.
Bartholomew stayed and in 1337, at the age of 23, he was ordained a monk with the name Sergius.

The young ascetic spent more than a year in complete solitude. His life consisted of prayers, fasting and hard work.
Sergius became famous in the surrounding area and soon other monks began to come to him for advice and soon the brotherhood numbered 12 people. Each monk lived in a separate cell, and together they gathered for worship.
This is how the famous Sergius-Troitskaya Lavra was founded. In 1354, Sergius was ordained abbot.

As the biography testifies, Reverend Sergius of Radonezh performed miracles even in his earthly life:

"…
Pilgrims came, they saw the poverty of the desert, but they also saw the peace and grace among the brothers, and they brought this leaven into their native families like light, like salt. And the name of the God-pleaser became glorious throughout Russia and many came to him with faith. Once, by praying, the old man healed a seriously ill person, and another time he healed a mad nobleman who was not in his right mind, ranting and fighting, so that ten they couldn't hold the man back."

St. Sergius of Radonezh was honored with a vision of the Most Holy Mother of God, who appeared to him during a night service and said:
"Do not be afraid, My chosen one. I have come to visit you. Do not grieve, because your prayer for the students and the monastery has been heard; and your abode shall abound in all things; not only in your life, but also in your presentation before God. I will not leave this place, and will irrevocably supply all that is necessary, preserve and protect it with my covering."


Saint Sergius of Radonezh knew how to act with "quiet and meek words" even on the most hardened and cruel hearts and in this way reconciled even ruling country region principles and kings at war.
Thanks to him, all the princes united before the Kulikovo battle, thanks to whom Russia managed to establish itself as a leading world country in the mid-centuries recognizing the main role of the Moscow prince Dimitriy.
The Russian army received a blessing from the venerable Sergius of Radonezh before the upcoming battle with the Tatars.

He predicted (prophecised) the victory over the Tatars, and on September 8, 1380, on the feast of the Nativity of the Virgin, the Russian troops defeated the Tatars at Kulikovo field, marking the beginning of liberation from the Tatar yoke. In other words at the time when Bulgarian Empire had crashed and has been enslaved, the Trnov kingdom fall down in 1393 and Vidin's kingdom of Bulgaria in 1396, just few years before the opposite hapened in Russia. Already baptized by many Bulgarian clergymen and given the Old Bulgarian Church language so called in newer times the Church Slavonic Books, Russia together with its spiritual flourishment managed to liberate from the Tatars and increasing gruadally in influence, countryland and power.
Prince Dimitrii deeply respected his spiritual father – the Reverend Sergius of Radonezh, who was also the godfather of his children.

Sergius of Radonezh died on September 25, 1392, reaching a very old age.
He predicted his death 2 years before and appointed his successor – his student Reverend Nikon.
St. Sergius of Radonezh was buried in a monastery founded by him, and 30 years later his body and clothes were found incorruptible and odoring with heavenly odor.

This happened in 1422 during the construction of the new and enlargened "Life-giving Trinity" church.

Shortly before the construction began, Sergius of Radonezh appeared in a dream to a pious man, telling him to convey the following words to the brothers:
"Why do you leave me so long in the grave, buried in the ground with all this water, it is narrow here for my body. "

When they dug for the foundations, the imperishable relics and clothes of St. Sergius of Radonezh were found unharmed, and the grave was full of water.

During the consecration of the Trinity Church, the relics of the saint were transferred to it, where they lay even to this day.

Let by the Holy Prayers of Saint Sergius of Radonezh the world, find more peace, love, faith and brotherhood, which we desperately need in these day of the absurd brother shedful war in Ukraine. 

Holy Reverend Father Sergij of Radonezh, pray the Lord Jesus Christ to save our souls and grant repentance to us sinners and peace to rule again in our hearts !

 

Living of New Martyr Saint Onuphrius of Gabrovo, a Bulgarian saint martyred in year 1818


June 17th, 2022

sveti-Onufrij-Gabrovski-saint-Onufrius-Gabrovski

The New Martyr saint Onufrij ( Onuphrius ) (1786 – 1818) was born in Gabrovo, Veliko Tarnovo Diocese, to pious and noble parents.
(His father Decho later became a monk under the name of Daniel in the same Hilendar monastery on Mount Athos, where his son was then active).
The child Onuphrius was given the name Matthew in Holy Baptism.
When he grew up, he was sent to a one of scarce Bulgarian schools, where he studied well.
When he was 17 years old, his parents once punished him for some childish thing unrest, and out of frivolous childishness, he declared in the presence of Turks that he would accept the Muslim faith.

In such cases, the Turks immediately seized the person who gave the promise to convert to islam and performed the rite of Mohammedan circumcision on him.

To prevent this, his parents hid him and perhaps sent him to the "fotress" of Christian Orthodox FaIth and keeper of Bulgarian spirit, the Troyan Monastery "Holy Mother of God".
In Troyan Monastery, to this day there is a the mouth to mouth legend that the Venerable Martyr Onuphrius began his monastic feat and received his first monastic haircut here with the name Manasseah (Manasij).

sveti-Onufrij-Gabrovski-Bylgarski-svetec

He ascended diligently in spiritual life, but the voice of his conscience began to rebuke him more and more for his public denial of Christianity, even if only in words.
Probably because of this he went to Holy Mount Athos, hoping that there, under the guidance of more experienced elders, he would repent enough and calm his conscience.
Manasseah spent some time in the Hilendar Monastery (a monastery that at this time has been inhibited with many Bulgarian monks), where he was ordained a deacon.

But, as the holy fathers of the Church say, the more a Christian grows in virtue, the deeper he humbles himself and his small sins seem great., same happened with Hierodeacon Manasseah.

He was always impressed by the words of the Savior Christ:

"Whosoever shall confess me before men, him will I confess also before my Father which is in heaven; but whosoever denieth me before men, him will I also deny before my Father which is in heaven." (Matt. 10: 32-33).

And from the lives of the saints he was especially deeply moved by the example of the holy martyr Barlaam, who held his hand without trembling over the burning pagan altar until his hand burned completely, but did not drop incense on the altar, to protect the occusation that he has offered incense to the idols.
His heart was inflamed with jealousy when the Venerable Euthymius, Ignatius and Acacius (Agathius), performed their martyrdom.
Then Manasseah secretly left Hilendar and went to the Forerunner's Hermitage to the local clergyman (elder) Nicephorus with a request to prepare him for such a martyrdom.

For four months he worked hard on enormous spiritual and bodly feats under the guidance of this elder.
Every day Manasseh made four thousand bows; his prayer was unceasing; his remorseful mood brought tears to his eyes.

During these four months of preparation he ate two and a half kilograms of dried grapes, and in the strictest forty-day fast he ate 30 grams of bread every two or three days and drank water in moderation.
After Elder Nicephorus thus prepared him for the impending martyrdom, he cut his hair in a great scheme receiving the great-schema name of Onufrij ( Onuphrius ) and sent him to the island of Chios with the same companion, Elder Gregory, whom he sent with the other martyrs анд вхере тхеир feat would take place there.

On Island Chios Saint Onuphrius lived one Sunday in fasting and prayer, while on Friday, the day of Christ's suffering, he appeared in Turkish robes before the local turkish judge, openly blasphemed Muhammad and threw the green turban on his head.

holy-new-martyr-Onufrius-Onufrij-of-Gabrovo

He was exhorted, thrown into prison, tortured, and sentenced to death the same day.
On January 4, 1818, his head was cut off on the seashore and along with his blood his body was thrown into the sea, so that Christians could not take for granted his holiness any particle veneration of the Venerable Martyr.

Before his death, some asked him about his name and homeland.
The Venerable Martyr replied that his name was Matthew and that he was from Veliko Tarnovo.
In this way he wanted to save the Holy Mount Athos and his monastery from troubles by the Turks.
Soon after his martyrdom, the Greeks from Mount Athos, soon canonized him and compiled a (living) biography and a service in his honor.

Text Translated from:

© Lives of the Saints. Synodal Publishing House of Bulgarian Orthodox Church, Sofia, 1991, edited by Parthenius, Bishop of Lefkada and Archimandrite Dr. Athanasius (Bonchev).

Zabbix: Monitor Linux rsyslog configured central log server is rechable with check_log_server_status.sh userparameter script


June 8th, 2022

zabbix-monitor-central-log-server-is-reachable-from-host-with-a-userparamater-script-zabbix-logo

On modern Linux OS servers on Redhat / CentOS / Fedora and Debian based distros log server service is usually running on the system  such as rsyslog (rsyslogd) to make sure the logging from services is properly logged in separate logs under /var/log.

A very common practice on critical server machines in terms of data security, where logs produced by rsyslog daermon needs to be copied over network via TCP or UDP protocol immediately is to copy over the /var/log produced logs to another configured central logging server. Then later every piece of bit generated by rsyslogd could be  overseen by a third party auditor person and useful for any investigation in case of logs integrity is required or at worse case if there is a suspicion that system in question is hacked by a malicious hax0r and logs have been "cleaned" up from any traces leading to the intruder (things usually done locally by hackers) or by any automated script exploit tools since yesr.

This doubled logging of system events to external log server  ipmentioned is very common practice by companies to protect their log data and quite useful for logs to be recovered easily later on from the central logging server machine that could be also setup for example to use rsyslogd to receive logs from other Linux machines in circumstances where some log disappears just like that (things i've seen happen) for any strange reason or gets destroyed by the admins mistake locally on machine / or by any other mean such as filesystem gets damaged. a very common practice by companies to protect their log data.  

Monitor remote logging server is reachable with userparameter script

Assuming that you already have setup a logging from the server hostname A towards the Central logging server log storepool and everything works as expected the next logical step is to have at least some basic way to monitor remote logging server configured is still reachable all the time and respectively rsyslog /var/log/*.* logs gets properly produced on remote side for example with something like a simple TCP remote server port check and reported in case of troubles in zabbix.

To solve that simple task for company where I'm employed, I've developed below check_log_server_status.sh:
 

#!/bin/bash
# @@ for TCP @ for UDP
# check_log_server_status.sh Script to check if configured TCP / UDP logging server in /etc/rsyslog.conf is rechable
# report to zabbix
DELIMITER='@@';
GREP_PORT='5145';
CONNECT_TIMEOUT=5;

PORT=$(grep -Ei "*.* $DELIMITER.*:$GREP_PORT" /etc/rsyslog.conf|awk -F : '{ print $2 }'|sort -rn |uniq);

#for i in $(grep -Ei "*.* $DELIMITER.*:$GREP_PORT" /etc/rsyslog.conf |grep -v '\#'|awk -F"$DELIMITER" '{ print $2 }' | awk -F ':' '{ print $1 }'|sort -rn); do
HOST=$(grep -Ei "*.* $DELIMITER.*:$GREP_PORT" /etc/rsyslog.conf |grep -v '\#'|awk -F"$DELIMITER" '{ print $2 }' | awk -F ':' '{ print $1 }'|sort -rn)

# echo $PORT

if [[ ! -z $PORT ]] && [[ ! -z $HOST ]]; then
SSH_RETURN=$(/bin/ssh $HOST -p $PORT -o ConnectTimeout=$CONNECT_TIMEOUT 2>&1);
else
echo "PROBLEM Port $GREP_PORT not defined in /etc/rsyslog.conf";
fi

##echo SSH_RETURN $SSH_RETURN;
#exit 1;
if [[ $(echo $SSH_RETURN |grep -i ‘Connection timed out during banner exchange’ | wc -l) -eq ‘1’ ]]; then
echo "rsyslogd $HOST:$PORT OK";
fi

if [[ $(echo $SSH_RETURN |grep -i ‘Connection refused’ | wc -l) -eq ‘1’ ]]; then
echo "rsyslogd $HOST:$PORT PROBLEM";
fi

#sleep 2;
#done


You can download a copy of the script check_log_server_status.sh here

Depending on the port the remote rsyslogd central logging server is using configure it in the script with respective port through the DELIMITER='@@', GREP_PORT='5145', CONNECT_TIMEOUT=5 values.

The delimiter is setup as usually in /etc/rsyslog.conf this the remote logging server for TCP IP is configured with @@ prefix to indicated TCP mode should be used.

Below is example from /etc/rsyslog.conf of how the rsyslogd server is configured:

[root@Server-hostA /root]# grep -i @@ /etc/rsyslogd.conf
# central remote Log server IP / port
*.* @@10.10.10.1:5145

To use the script on a machine, where you have a properly configured zabbix-agentd service host connected and reporting data to a zabbix-server monitoring server.

1. Set up the script under /usr/local/bin/check_log_server_status.sh

[root@Server-hostA /root ]# vim /usr/local/bin/check_log_server_status.sh

[root@Server-hostA /root ]# chmod +x /usr/local/bin/check_log_server_status.sh

2. Prepare userparameter_check_log_server.conf with log_server.check Item key

[root@Server-hostA zabbix_agentd.d]# cat userparameter_check_log_server.conf 
UserParameter=log_server.check, /usr/local/bin/check_log_server_status.sh

3. Set in Zabbix some Item such as on below screenshot

 

check-log-server-status-screenshot-linux-item-zabbix.png4. Create a Zabbix trigger 

check-log-server-status-trigger-logserver-is-unreachable-zabbix


The redded hided field in Expression field should be substituted with your actual hostname on which the monitor script will run.

Living of unknown saint Saint Sophronius of Sofia, known also as Saint Soprhonius of the Bulgaria / Sofronij of the Balkans


May 28th, 2022

Sveti-Sofronij-Bylgarski-SofijskiSaint-Sophronius-of-Sofia-Bulgaria-of-the-Balkans
 

Biography of Staint Sofronij / Sophronius of Bulgaria Sofia, Known also as Saint Soprhonius of the Balkans

The parish priest of the village of Penkyovtsi (Sofia, Bulgaria region) Stefan (Te fled to Sofia with his wife due to Turkish violence, from hence he later fled to Wallachia region to the great Wallachian voivode (Duke) Radul. His wife died there and he became a monk named Sophronius (not to be mistaken with the very famous Bulgarian saint Saint Sofronius of Vratza / Sofronij Vrachanski).

After the death of Duke Radul, he returned from the Danube river to his homeland and settled in a monastery near city of Ruse (probably he lived in the cave monastery of Saint Joachim I Patriarch of Tarnovo and the Venerable Demetrius Besarabovsky, that even today is the biggest monastery nearby the city of Ruse).

There he struggled with common sanctification practices as fasting, prayer, work and alms for the poor. The devil did not tolerate his monastic great achievements and set a monastic servant against him, who struck him on the head with an ax and killed him.

Three years later, Sophronius appeared to the people living in the monastery, who obeyed his suggestion (obviously they got inspired to  dug up his grave which is a common ancient Christian practice for notable christians who might have been saints) and look up if his relics are incorruptable and found his relics incorruptible and fragrant (emitting a heavenly odor), as they have been inspired by God's Spirit to do.
People with great joy placed the holy relics in a coffin for common (universal) worship of all the Christians.

We do not know the exact dates he was  born or has been killed, because of the devilish envy, as at that time Bulgaria has been under the Ottoman turks and the founding of his holy relics has happened years after his martyrdom. But most probably the Venerable Sofronij  lived in the second half of the fifteenth century and the beginning of the sixteenth century, by the second decade of which he must have suffered. This information about him is told to us by the Bulgarian writer and priest Father (the bulgarian word for which is Pop – stems from the greek word Papas (Father) Peyu. The same Pop Peyu has been also the author of the life of Saint George of Sofia the New not to be mistaken with Saint George the Newest from Sofia  who maryrdom suffered in year 1515 because of his unwillingless to accept the false Islamic faith.

© Lives of the Saints. Synodal Publishing House, Sofia, 1991, edited by Parthenius, Bishop of Lefkada and Archimandrite Dr. Athanasius (Bonchev) with minor inclusions of the article author Georgi Georgiev

Let by his holy prayers the Bulgarian homeland and especially the suffering in Ukraine and all people everywhere, finds more Peace, Love, Hope, Faith and Goodness 

Saint Hieromartyr Therapont of Serdika ( Sofia ) martyred year † 1555 for Christ


May 26th, 2022

Saint Therapontius of Serdika is celebrated every year on on May 27 in the Bulgarian Orthodox Church, together with St. Holy Martyrs Therapont of Sardis († 259). He is born and lived in the ancient city of Serdika (today Sofia).

He is part of the nine saints of Sofia, that are celebrated in the Church throughout the Liturgical year.

A little is known of him and just like the martyrologies of much of the ancient saints, we have only few sentences left mentioning his great martyrdom for Christ, along with other local Bulgarian saints. He has a written  celebration service in the Minelogion Church book for  27-th of May.
Minelogion for those unaware is one of the service books used in the Night and Morning services songs and containing services details about the glorified saints for each day of the year.

Saint-Therapontius-Theraponti-of-Serdika-Sofia-Sveti-Terapont-Sofiiski

Troparion of the Holy Martyr, voice 4
He became a partaker of morals and viceroy of the apostles
in the way of your contemplation, inspired by God, you have done deeds,
therefore you have faithfully taught the word of truth,
for his faith he suffered even to the point of blood,
Holy Martyr Therapont, beg the Christ God to save our souls.

 

Saint Hieromartyr Therapont of Roman Fortress Serdika ( Sofia )
The famous Bulgarian medieval historian Matei Gramatik, who was a contemporary of Saint Nicholas of Sofia (a 15h century famous Bulgarian martyr saint) and a witness to his martyrdom in 1555, who wrote his biography with great skill, writes about this holy martyr. There he describes the situation in which St. Nicholas of Sofia lived – both geographically and spiritually.
In his description of the spiritual situation, he gives brief information about the saints of Sofia, including St. Terapont of Serdica (Sofia). He writes:

Saint-hieromartyr-Theraponti-Therapontius-of-Serdika-Sofia-icon

"When you listen for the inhabitants of Sofia, don't think about the current contemporary ones,
but for the heavenly ones, who were once co-inhabitants of us and now live with the angels.
So forth it is beneficial, to tell about 2 , 3 of them. The holy hieromartyr Therapontius, who
being a citizen of this place and a presbyter of the holy God's Church in Serdika (Sofia), lived filled
with lot of virtues and at the end, during the persecutions of Christians, has been detained by guards
by the lawless for his Christ confession. After a lot of martyrs and being enchained with a heavy
iron chains, he has been put out of the city and on a distance of one day walking, on
this place he was beheaded and henceforth he received a martyrs death for Christ.
They say on the place where his blood was shed, in that time a large oak tree has grown  and it is seen until today and there a lot of miraculous healings occur,
whenever one comes with faith."

Today a part of this trunk of the oak is kept as a sacred relic in the ancient Sofia's capital ancient church "St. Petka", where the memory of the holy martyr is celebrated every year on May 27.

Translated from: © Lives of the Saints Book. Synodal Publishing House, Sofia, 1991, edited by Parthenius, Bishop of Lefkada and Archimandrite Dr. Athanasius (Bonchev).


Another Bulgarian saint Paisios of Hilendar / Paisij Hilendarski (1722 – 1773) also mentions in his history book History of the Slavo-Bulgarians states: "There are three holy martyrs in the city of Sofia:
1) St. George;
2) St. Nicholas;
3) St. Terapontius.

This saint was a priest in town of Tran, where many people now go for healing. Where in the Turks has slayered the saint a an oak greaw and with his prayers a lot of healing is given on the place of this oak. Same manner Saints George and Nicholas suffered from the godless Turks during Selim's reign; and their holy relics give healing in this city of Sofia. "

Icon Images of St. Therapontius are known to exist today from the XIX century. There are icons icons in the Sofia church "St. Paraskeva ”, in Pernik, in the church of " St. Petka ” in Sofia, fresco in the church“ St. Dimitar ” in the village of Yarlovtsi, Transko, wherein used to an icon also whose location now is unknown.

In Tran and Godech respectively there were folk customs associated with the saint and therein and in the region he is revered as a healer and protector of the harvests.
There is also a cave in Trun, which is indicated as a refuge for the saint.
A chapel in his name was in the city, burned by the Turks in the 30s of the XIX century.

In Glory of St. Terapontius of Sofia during the Second World War and until 1957 was dedicated the Revival church "Holy Trinity ", today – Saint Great-martyr (Mina)  Menas, in the Slatina district of Sofia. The church was built on the remains of the monastery “Holy  Trinity", according to mouth to mouth tradition kept for ages. 

According to the legend, the saint was slaughtered here and this gives some reason to presume that St. Terapontius might have been one of the spiritual fathers who were serving in the monastery at that time.

Let by the holy prayers of Saint Terapont God gives forgives to our multitude of transgressions and grants more Peace, Love, Hope, Faith and goodness to everyone !

How to RPM update Hypervisors and Virtual Machines running Haproxy High Availability cluster on KVM, Virtuozzo without a downtime on RHEL / CentOS Linux


May 20th, 2022

virtuozzo-kvm-virtual-machines-and-hypervisor-update-manual-haproxy-logo


Here is the scenario, lets say you have on your daily task list two Hypervisor (HV) hosts running CentOS or RHEL Linux with KVM or Virutozzo technology and inside the HV hosts you have configured at least 2 pairs of virtual machines one residing on HV Host 1 and one residing on HV Host 2 and you need to constantly keep the hosts to the latest distribution major release security patchset.

The Virtual Machines has been running another set of Redhat Linux or CentOS configured to work in a High Availability Cluster running Haproxy / Apache / Postfix or any other kind of HA solution on top of corosync / keepalived or whatever application cluster scripts Free or Open Source technology that supports a switch between clustered Application nodes.

The logical question comes how to keep up the CentOS / RHEL Machines uptodate without interfering with the operations of the Applications running on the cluster?

Assuming that the 2 or more machines are configured to run in Active / Passive App member mode, e.g. one machine is Active at any time and the other is always Passive, a switch is possible between the Active and Passive node.

HAProxy--Load-Balancer-cluster-2-nodes-your-Servers

In this article I'll give a simple step by step tested example on how you I succeeded to update (for security reasons) up to the latest available Distribution major release patchset on one by one first the Clustered App on Virtual Machines 1 and VM2 on Linux Hypervisor Host 1. Then the App cluster VM 1 / VM 2 on Hypervisor Host 2.
And finally update the Hypervisor1 (after moving the Active resources from it to Hypervisor2) and updating the Hypervisor2 after moving the App running resources back on HV1.
I know the procedure is a bit monotonic but it tries to go through everything step by step to try to mitigate any possible problems. In case of failure of some rpm dependencies during yum / dnf tool updates you can always revert to backups so in anyways don't forget to have a fully functional backup of each of the HV hosts and the VMs somewhere on a separate machine before proceeding further, any possible failures due to following my aritcle literally is your responsibility 🙂

 

0. Check situation before the update on HVs / get VM IDs etc.

Check the virsion of each of the machines to be updated both Hypervisor and Hosted VMs, on each machine run:
 

# cat /etc/redhat-release
CentOS Linux release 7.9.2009 (Core)


The machine setup I'll be dealing with is as follows:
 

hypervisor-host1 -> hypervisor-host1.fqdn.com 
•    virt-mach-centos1
•    virt-machine-zabbix-proxy-centos (zabbix proxy)

hypervisor-host2 -> hypervisor-host2.fqdn.com
•    virt-mach-centos2
•    virt-machine-zabbix2-proxy-centos (zabbix proxy)

To check what is yours check out with virsh cmd –if on KVM or with prlctl if using Virutozzo, you should get something like:

[root@hypervisor-host2 ~]# virsh list
 Id Name State
—————————————————-
 1 vm-host1 running
 2 virt-mach-centos2 running

 # virsh list –all

[root@hypervisor-host1 ~]# virsh list
 Id Name State
—————————————————-
 1 vm-host2 running
 3 virt-mach-centos1 running

[root@hypervisor-host1 ~]# prlctl list
UUID                                    STATUS       IP_ADDR         T  NAME
{dc37c201-08c9-589d-aa20-9386d63ce3f3}  running      –               VM virt-mach-centos1
{76e8a5f8-caa8-5442-830e-aa4bfe8d42d9}  running      –               VM vm-host2
[root@hypervisor-host1 ~]#

If you have stopped VMs with Virtuozzo to list the stopped ones as well.
 

# prlctl list -a

[root@hypervisor-host2 74a7bbe8-9245-5385-ac0d-d10299100789]# vzlist -a
                                CTID      NPROC STATUS    IP_ADDR         HOSTNAME
[root@hypervisor-host2 74a7bbe8-9245-5385-ac0d-d10299100789]# prlctl list
UUID                                    STATUS       IP_ADDR         T  NAME
{92075803-a4ce-5ec0-a3d8-9ee83d85fc76}  running      –               VM virt-mach-centos2
{74a7bbe8-9245-5385-ac0d-d10299100789}  running      –               VM vm-host1

# prlctl list -a


If due to Virtuozzo version above command does not return you can manually check in the VM located folder, VM ID etc.
 

[root@hypervisor-host2 vmprivate]# ls
74a7bbe8-9245-4385-ac0d-d10299100789  92075803-a4ce-4ec0-a3d8-9ee83d85fc76
[root@hypervisor-host2 vmprivate]# pwd
/vz/vmprivate
[root@hypervisor-host2 vmprivate]#


[root@hypervisor-host1 ~]# ls -al /vz/vmprivate/
total 20
drwxr-x—. 5 root root 4096 Feb 14  2019 .
drwxr-xr-x. 7 root root 4096 Feb 13  2019 ..
drwxr-x–x. 4 root root 4096 Feb 18  2019 1c863dfc-1deb-493c-820f-3005a0457627
drwxr-x–x. 4 root root 4096 Feb 14  2019 76e8a5f8-caa8-4442-830e-aa4bfe8d42d9
drwxr-x–x. 4 root root 4096 Feb 14  2019 dc37c201-08c9-489d-aa20-9386d63ce3f3
[root@hypervisor-host1 ~]#


Before doing anything with the VMs, also don't forget to check the Hypervisor hosts has enough space, otherwise you'll get in big troubles !
 

[root@hypervisor-host2 vmprivate]# df -h
Filesystem                       Size  Used Avail Use% Mounted on
/dev/mapper/centos_hypervisor-host2-root   20G  1.8G   17G  10% /
devtmpfs                          20G     0   20G   0% /dev
tmpfs                             20G     0   20G   0% /dev/shm
tmpfs                             20G  2.0G   18G  11% /run
tmpfs                             20G     0   20G   0% /sys/fs/cgroup
/dev/sda1                        992M  159M  766M  18% /boot
/dev/mapper/centos_hypervisor-host2-home  9.8G   37M  9.2G   1% /home
/dev/mapper/centos_hypervisor-host2-var   9.8G  355M  8.9G   4% /var
/dev/mapper/centos_hypervisor-host2-vz    755G   25G  692G   4% /vz

 

[root@hypervisor-host1 ~]# df -h
Filesystem               Size  Used Avail Use% Mounted on
/dev/mapper/centos-root   50G  1.8G   45G   4% /
devtmpfs                  20G     0   20G   0% /dev
tmpfs                     20G     0   20G   0% /dev/shm
tmpfs                     20G  2.1G   18G  11% /run
tmpfs                     20G     0   20G   0% /sys/fs/cgroup
/dev/sda2                992M  153M  772M  17% /boot
/dev/mapper/centos-home  9.8G   37M  9.2G   1% /home
/dev/mapper/centos-var   9.8G  406M  8.9G   5% /var
/dev/mapper/centos-vz    689G   12G  643G   2% /vz

Another thing to do before proceeding with update is to check and tune if needed the amount of CentOS repositories used, before doing anything with yum.
 

[root@hypervisor-host2 yum.repos.d]# ls -al
total 68
drwxr-xr-x.   2 root root  4096 Oct  6 13:13 .
drwxr-xr-x. 110 root root 12288 Oct  7 11:13 ..
-rw-r–r–.   1 root root  4382 Mar 14  2019 CentOS7.repo
-rw-r–r–.   1 root root  1664 Sep  5  2019 CentOS-Base.repo
-rw-r–r–.   1 root root  1309 Sep  5  2019 CentOS-CR.repo
-rw-r–r–.   1 root root   649 Sep  5  2019 CentOS-Debuginfo.repo
-rw-r–r–.   1 root root   314 Sep  5  2019 CentOS-fasttrack.repo
-rw-r–r–.   1 root root   630 Sep  5  2019 CentOS-Media.repo
-rw-r–r–.   1 root root  1331 Sep  5  2019 CentOS-Sources.repo
-rw-r–r–.   1 root root  6639 Sep  5  2019 CentOS-Vault.repo
-rw-r–r–.   1 root root  1303 Mar 14  2019 factory.repo
-rw-r–r–.   1 root root   666 Sep  8 10:13 openvz.repo
[root@hypervisor-host2 yum.repos.d]#

 

[root@hypervisor-host1 yum.repos.d]# ls -al
total 68
drwxr-xr-x.   2 root root  4096 Oct  6 13:13 .
drwxr-xr-x. 112 root root 12288 Oct  7 11:09 ..
-rw-r–r–.   1 root root  1664 Sep  5  2019 CentOS-Base.repo
-rw-r–r–.   1 root root  1309 Sep  5  2019 CentOS-CR.repo
-rw-r–r–.   1 root root   649 Sep  5  2019 CentOS-Debuginfo.repo
-rw-r–r–.   1 root root   314 Sep  5  2019 CentOS-fasttrack.repo
-rw-r–r–.   1 root root   630 Sep  5  2019 CentOS-Media.repo
-rw-r–r–.   1 root root  1331 Sep  5  2019 CentOS-Sources.repo
-rw-r–r–.   1 root root  6639 Sep  5  2019 CentOS-Vault.repo
-rw-r–r–.   1 root root  1303 Mar 14  2019 factory.repo
-rw-r–r–.   1 root root   300 Mar 14  2019 obsoleted_tmpls.repo
-rw-r–r–.   1 root root   666 Sep  8 10:13 openvz.repo


1. Dump VM definition XMs (to have it in case if it gets wiped during update)

There is always a possibility that something will fail during the update and you might be unable to restore back to the old version of the Virtual Machine due to some config misconfigurations or whatever thus a very good idea, before proceeding to modify the working VMs is to use KVM's virsh and dump the exact set of XML configuration that makes the VM roll properly.

To do so:
Check a little bit up in the article how we have listed the IDs that are part of the directory containing the VM.
 

[root@hypervisor-host1 ]# virsh dumpxml (Id of VM virt-mach-centos1 ) > /root/virt-mach-centos1_config_bak.xml
[root@hypervisor-host2 ]# virsh dumpxml (Id of VM virt-mach-centos2) > /root/virt-mach-centos2_config_bak.xml

 


2. Set on standby virt-mach-centos1 (virt-mach-centos1)

As I'm upgrading two machines that are configured to run an haproxy corosync cluster, before proceeding to update the active host, we have to switch off
the proxied traffic from node1 to node2, – e.g. standby the active node, so the cluster can move up the traffic to other available node.
 

[root@virt-mach-centos1 ~]# pcs cluster standby virt-mach-centos1


3. Stop VM virt-mach-centos1 & backup on Hypervisor host (hypervisor-host1) for VM1

Another prevention step to make sure you don't get into damaged VM or broken haproxy cluster after the upgrade is to of course backup 

 

[root@hypervisor-host1 ]# prlctl backup virt-mach-centos1

or
 

[root@hypervisor-host1 ]# prlctl stop virt-mach-centos1
[root@hypervisor-host1 ]# cp -rpf /vz/vmprivate/dc37c201-08c9-489d-aa20-9386d63ce3f3 /vz/vmprivate/dc37c201-08c9-489d-aa20-9386d63ce3f3-bak
[root@hypervisor-host1 ]# tar -czvf virt-mach-centos1_vm_virt-mach-centos1.tar.gz /vz/vmprivate/dc37c201-08c9-489d-aa20-9386d63ce3f3

[root@hypervisor-host1 ]# prlctl start virt-mach-centos1


4. Remove package version locks on all hosts

If you're using package locking to prevent some other colleague to not accidently upgrade the machine (if multiple sysadmins are managing the host), you might use the RPM package locking meachanism, if that is used check RPM packs that are locked and release the locking.

+ List actual list of locked packages

[root@hypervisor-host1 ]# yum versionlock list  

…..
0:libtalloc-2.1.16-1.el7.*
0:libedit-3.0-12.20121213cvs.el7.*
0:p11-kit-trust-0.23.5-3.el7.*
1:quota-nls-4.01-19.el7.*
0:perl-Exporter-5.68-3.el7.*
0:sudo-1.8.23-9.el7.*
0:libxslt-1.1.28-5.el7.*
versionlock list done
                          

+ Clear the locking            

# yum versionlock clear                               


+ List actual list / == clear all entries
 

[root@virt-mach-centos2 ]# yum versionlock list; yum versionlock clear
[root@virt-mach-centos1 ]# yum versionlock list; yum versionlock clear
[root@hypervisor-host1 ~]# yum versionlock list; yum versionlock clear
[root@hypervisor-host2 ~]# yum versionlock list; yum versionlock clear


5. Do yum update virt-mach-centos1


For some clarity if something goes wrong, it is really a good idea to make a dump of the basic packages installed before the RPM package update is initiated,
The exact versoin of RHEL or CentOS as well as the list of locked packages, if locking is used.

Enter virt-mach-centos1 (ssh virt-mach-centos1) and run following cmds:
 

# cat /etc/redhat-release  > /root/logs/redhat-release-$(hostname)-$(date '+%Y-%m-%d_%H-%M-%S').out
# cat /etc/grub.d/30_os-prober > /root/logs/grub2-efi-$(hostname)-$(date '+%Y-%m-%d_%H-%M-%S').out


+ Only if needed!!
 

# yum versionlock clear
# yum versionlock list


Clear any previous RPM packages – careful with that as you might want to keep the old RPMs, if unsure comment out below line
 

# yum clean all |tee /root/logs/yumcleanall-$(hostname)-$(date '+%Y-%m-%d_%H-%M-%S').out

 

Proceed with the update and monitor closely the output of commands and log out everything inside files using a small script that you should place under /root/status the script is given at the end of the aritcle.:
 

yum check-update |tee /root/logs/yumcheckupdate-$(hostname)-$(date '+%Y-%m-%d_%H-%M-%S').out
yum check-update | wc -l
yum update |tee /root/logs/yumupdate-$(hostname)-$(date '+%Y-%m-%d_%H-%M-%S').out
sh /root/status |tee /root/logs/status-before-$(hostname)-$(date '+%Y-%m-%d_%H-%M-%S').out

 

6. Check if everything is running fine after upgrade

Reboot VM
 

# shutdown -r now


7. Stop VM virt-mach-centos2 & backup  on Hypervisor host (hypervisor-host2)

Same backup step as prior 

# prlctl backup virt-mach-centos2


or
 

# prlctl stop virt-mach-centos2
# cp -rpf /vz/vmprivate/92075803-a4ce-4ec0-a3d8-9ee83d85fc76 /vz/vmprivate/92075803-a4ce-4ec0-a3d8-9ee83d85fc76-bak
## tar -czvf virt-mach-centos2_vm_virt-mach-centos2.tar.gz /vz/vmprivate/92075803-a4ce-4ec0-a3d8-9ee83d85fc76

# prctl start virt-mach-centos2


8. Do yum update on virt-mach-centos2

Log system state, before the update
 

# cat /etc/redhat-release  > /root/logs/redhat-release-vorher-$(hostname)-$(date '+%Y-%m-%d_%H-%M-%S').out
# cat /etc/grub.d/30_os-prober > /root/logs/grub2-efi-vorher-$(hostname)-$(date '+%Y-%m-%d_%H-%M-%S').out
# yum versionlock clear == if needed!!
# yum versionlock list

 

Clean old install update / packages if required
 

# yum clean all |tee /root/logs/yumcleanall-$(hostname)-$(date '+%Y-%m-%d_%H-%M-%S').out


Initiate the update

# yum check-update |tee /root/logs/yumcheckupdate-$(hostname)-$(date '+%Y-%m-%d_%H-%M-%S').out 2>&1
# yum check-update | wc -l 
# yum update |tee /root/logs/yumupdate-$(hostname)-$(date '+%Y-%m-%d_%H-%M-%S').out 2>&1
# sh /root/status |tee /root/logs/status-before-$(hostname)-$(date '+%Y-%m-%d_%H-%M-%S').out


9. Check if everything is running fine after upgrade
 

Reboot VM
 

# shutdown -r now

 

10. Stop VM vm-host2 & backup
 

# prlctl backup vm-host2


or

# prlctl stop vm-host2

Or copy the actual directory containig the Virtozzo VM (use the correct ID)
 

# cp -rpf /vz/vmprivate/76e8a5f8-caa8-5442-830e-aa4bfe8d42d9 /vz/vmprivate/76e8a5f8-caa8-5442-830e-aa4bfe8d42d9-bak
## tar -czvf vm-host2.tar.gz /vz/vmprivate/76e8a5f8-caa8-4442-830e-aa5bfe8d42d9

# prctl start vm-host2


11. Do yum update vm-host2
 

# cat /etc/redhat-release  > /root/logs/redhat-release-vorher-$(hostname)-$(date '+%Y-%m-%d_%H-%M-%S').out
# cat /etc/grub.d/30_os-prober > /root/logs/grub2-efi-vorher-$(hostname)-$(date '+%Y-%m-%d_%H-%M-%S').out


Clear only if needed

# yum versionlock clear
# yum versionlock list
# yum clean all |tee /root/logs/yumcleanall-$(hostname)-$(date '+%Y-%m-%d_%H-%M-%S').out


Do the rpm upgrade

# yum check-update |tee /root/logs/yumcheckupdate-$(hostname)-$(date '+%Y-%m-%d_%H-%M-%S').out
# yum check-update | wc -l
# yum update |tee /root/logs/yumupdate-$(hostname)-$(date '+%Y-%m-%d_%H-%M-%S').out
# sh /root/status |tee /root/logs/status-before-$(hostname)-$(date '+%Y-%m-%d_%H-%M-%S').out


12. Check if everything is running fine after upgrade
 

Reboot VM
 

# shutdown -r now


13. Do yum update hypervisor-host2

 

 

# cat /etc/redhat-release  > /root/logs/redhat-release-vorher-$(hostname)-$(date '+%Y-%m-%d_%H-%M-%S').out
# cat /etc/grub.d/30_os-prober > /root/logs/grub2-efi-vorher-$(hostname)-$(date '+%Y-%m-%d_%H-%M-%S').out

Clear lock   if needed

# yum versionlock clear
# yum versionlock list
# yum clean all |tee /root/logs/yumcleanall-$(hostname)-$(date '+%Y-%m-%d_%H-%M-%S').out


Update rpms
 

# yum check-update |tee /root/logs/yumcheckupdate-$(hostname)-$(date '+%Y-%m-%d_%H-%M-%S').out 2>&1
# yum check-update | wc -l
# yum update |tee /root/logs/yumupdate-$(hostname)-$(date '+%Y-%m-%d_%H-%M-%S').out 2>&1
# sh /root/status |tee /root/logs/status-before-$(hostname)-$(date '+%Y-%m-%d_%H-%M-%S').out


14. Stop VM vm-host1 & backup


Some as ealier
 

# prlctl backup vm-host1

or
 

# prlctl stop vm-host1

# cp -rpf /vz/vmprivate/74a7bbe8-9245-4385-ac0d-d10299100789 /vz/vmprivate/74a7bbe8-9245-4385-ac0d-d10299100789-bak
# tar -czvf vm-host1.tar.gz /vz/vmprivate/74a7bbe8-9245-4385-ac0d-d10299100789

# prctl start vm-host1


15. Do yum update vm-host2
 

# cat /etc/redhat-release  > /root/logs/redhat-release-vorher-$(hostname)-$(date '+%Y-%m-%d_%H-%M-%S').out
# cat /etc/grub.d/30_os-prober > /root/logs/grub2-efi-vorher-$(hostname)-$(date '+%Y-%m-%d_%H-%M-%S').out
# yum versionlock clear == if needed!!
# yum versionlock list
# yum clean all |tee /root/logs/yumcleanall-$(hostname)-$(date '+%Y-%m-%d_%H-%M-%S').out
# yum check-update |tee /root/logs/yumcheckupdate-$(hostname)-$(date '+%Y-%m-%d_%H-%M-%S').out
# yum check-update | wc -l
# yum update |tee /root/logs/yumupdate-$(hostname)-$(date '+%Y-%m-%d_%H-%M-%S').out
# sh /root/status |tee /root/logs/status-before-$(hostname)-$(date '+%Y-%m-%d_%H-%M-%S').out


16. Check if everything is running fine after upgrade

+ Reboot VM

# shutdown -r now


17. Do yum update hypervisor-host1

Same procedure for HV host 1 

# cat /etc/redhat-release  > /root/logs/redhat-release-vorher-$(hostname)-$(date '+%Y-%m-%d_%H-%M-%S').out
# cat /etc/grub.d/30_os-prober > /root/logs/grub2-efi-vorher-$(hostname)-$(date '+%Y-%m-%d_%H-%M-%S').out

Clear lock
 

# yum versionlock clear
# yum versionlock list
# yum clean all |tee /root/logs/yumcleanall-$(hostname)-$(date '+%Y-%m-%d_%H-%M-%S').out

# yum check-update |tee /root/logs/yumcheckupdate-$(hostname)-$(date '+%Y-%m-%d_%H-%M-%S').out
# yum check-update | wc -l
# yum update |tee /root/logs/yumupdate-$(hostname)-$(date '+%Y-%m-%d_%H-%M-%S').out
# sh /root/status |tee /root/logs/status-before-$(hostname)-$(date '+%Y-%m-%d_%H-%M-%S').out


18. Check if everything is running fine after upgrade

Reboot VM
 

# shutdown -r now


Check hypervisor-host1 all VMs run as expected 


19. Check if everything is running fine after upgrade

Reboot VM
 

# shutdown -r now


Check hypervisor-host2 all VMs run as expected afterwards


20. Check once more VMs and haproxy or any other contained services in VMs run as expected

Login to hosts and check processes and logs for errors etc.
 

21. Haproxy Unstandby virt-mach-centos1

Assuming that the virt-mach-centos1 and virt-mach-centos2 are running a Haproxy / corosync cluster you can try to standby node1 and check the result
hopefully all should be fine and traffic should come to host node2.

[root@virt-mach-centos1 ~]# pcs cluster unstandby virt-mach-centos1


Monitor logs and make sure HAproxy works fine on virt-mach-centos1


22. If necessery to redefine VMs (in case they disappear from virsh) or virtuosso is not working

[root@virt-mach-centos1 ]# virsh define /root/virt-mach-centos1_config_bak.xml
[root@virt-mach-centos1 ]# virsh define /root/virt-mach-centos2_config_bak.xml


23. Set versionlock to RPMs to prevent accident updates and check OS version release

[root@virt-mach-centos2 ]# yum versionlock \*
[root@virt-mach-centos1 ]# yum versionlock \*
[root@hypervisor-host1 ~]# yum versionlock \*
[root@hypervisor-host2 ~]# yum versionlock \*

[root@hypervisor-host2 ~]# cat /etc/redhat-release 
CentOS Linux release 7.8.2003 (Core)

Other useful hints

[root@hypervisor-host1 ~]# virsh console dc37c201-08c9-489d-aa20-9386d63ce3f3
Connected to domain virt-mach-centos1
..

! Compare packages count before the upgrade on each of the supposable identical VMs and HVs – if there is difference in package count review what kind of packages are different and try to make the machines to look as identical as possible  !

Packages to update on hypervisor-host1 Count: XXX
Packages to update on hypervisor-host2 Count: XXX
Packages to update virt-mach-centos1 Count: – 254
Packages to update virt-mach-centos2 Count: – 249

The /root/status script

+++

#!/bin/sh
echo  '=======================================================   '
echo  '= Systemctl list-unit-files –type=service | grep enabled '
echo  '=======================================================   '
systemctl list-unit-files –type=service | grep enabled

echo  '=======================================================   '
echo  '= systemctl | grep ".service" | grep "running"            '
echo  '=======================================================   '
systemctl | grep ".service" | grep "running"

echo  '=======================================================   '
echo  '= chkconfig –list                                        '
echo  '=======================================================   '
chkconfig –list

echo  '=======================================================   '
echo  '= netstat -tulpn                                          '
echo  '=======================================================   '
netstat -tulpn

echo  '=======================================================   '
echo  '= netstat -r                                              '
echo  '=======================================================   '
netstat -r


+++

That's all folks, once going through the article, after some 2 hours of efforts or so you should have an up2date machines.
Any problems faced or feedback is mostly welcome as this might help others who have the same setup.

Thanks for reading me 🙂

How to monitor Haproxy Application server backends with Zabbix userparameter autodiscovery scripts


May 13th, 2022

zabbix-backend-monitoring-logo

Haproxy is doing quite a good job in High Availability tasks where traffic towards multiple backend servers has to be redirected based on the available one to sent data from the proxy to. 

Lets say haproxy is configured to proxy traffic for App backend machine1 and App backend machine2.

Usually in companies people configure a monitoring like with Icinga or Zabbix / Grafana to keep track on the Application server is always up and running. Sometimes however due to network problems (like burned Network Switch / router or firewall misconfiguration) or even an IP duplicate it might happen that Application server seems to be reporting reachable from some monotoring tool on it but unreachable from  Haproxy server -> App backend machine2 but reachable from App backend machine1. And even though haproxy will automatically switch on the traffic from backend machine2 to App machine1. It is a good idea to monitor and be aware that one of the backends is offline from the Haproxy host.
In this article I'll show you how this is possible by using 2 shell scripts and userparameter keys config through the autodiscovery zabbix legacy feature.
Assumably for the setup to work you will need to have as a minimum a Zabbix server installation of version 5.0 or higher.

1. Create the required  haproxy_discovery.sh  and haproxy_stats.sh scripts 

You will have to install the two scripts under some location for example we can put it for more clearness under /etc/zabbix/scripts

[root@haproxy-server1 ]# mkdir /etc/zabbix/scripts

[root@haproxy-server1 scripts]# vim haproxy_discovery.sh 
#!/bin/bash
#
# Get list of Frontends and Backends from HAPROXY
# Example: ./haproxy_discovery.sh [/var/lib/haproxy/stats] FRONTEND|BACKEND|SERVERS
# First argument is optional and should be used to set location of your HAPROXY socket
# Second argument is should be either FRONTEND, BACKEND or SERVERS, will default to FRONTEND if not set
#
# !! Make sure the user running this script has Read/Write permissions to that socket !!
#
## haproxy.cfg snippet
#  global
#  stats socket /var/lib/haproxy/stats  mode 666 level admin

HAPROXY_SOCK=""/var/run/haproxy/haproxy.sock
[ -n “$1” ] && echo $1 | grep -q ^/ && HAPROXY_SOCK="$(echo $1 | tr -d '\040\011\012\015')"

if [[ “$1” =~ (25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?):[0-9]{1,5} ]];
then
    HAPROXY_STATS_IP="$1"
    QUERYING_METHOD="TCP"
fi

QUERYING_METHOD="${QUERYING_METHOD:-SOCKET}"

query_stats() {
    if [[ ${QUERYING_METHOD} == “SOCKET” ]]; then
        echo "show stat" | socat ${HAPROXY_SOCK} stdio 2>/dev/null
    elif [[ ${QUERYING_METHOD} == “TCP” ]]; then
        echo "show stat" | nc ${HAPROXY_STATS_IP//:/ } 2>/dev/null
    fi
}

get_stats() {
        echo "$(query_stats)" | grep -v "^#"
}

[ -n “$2” ] && shift 1
case $1 in
        B*) END="BACKEND" ;;
        F*) END="FRONTEND" ;;
        S*)
                for backend in $(get_stats | grep BACKEND | cut -d, -f1 | uniq); do
                        for server in $(get_stats | grep "^${backend}," | grep -v BACKEND | grep -v FRONTEND | cut -d, -f2); do
                                serverlist="$serverlist,\n"'\t\t{\n\t\t\t"{#BACKEND_NAME}":"'$backend'",\n\t\t\t"{#SERVER_NAME}":"'$server'"}'
                        done
                done
                echo -e '{\n\t"data":[\n’${serverlist#,}’]}'
                exit 0
        ;;
        *) END="FRONTEND" ;;
esac

for frontend in $(get_stats | grep "$END" | cut -d, -f1 | uniq); do
    felist="$felist,\n"'\t\t{\n\t\t\t"{#'${END}'_NAME}":"'$frontend'"}'
done
echo -e '{\n\t"data":[\n’${felist#,}’]}'

 

[root@haproxy-server1 scripts]# vim haproxy_stats.sh 
#!/bin/bash
set -o pipefail

if [[ “$1” = /* ]]
then
  HAPROXY_SOCKET="$1"
  shift 0
else
  if [[ “$1” =~ (25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?):[0-9]{1,5} ]];
  then
    HAPROXY_STATS_IP="$1"
    QUERYING_METHOD="TCP"
    shift 1
  fi
fi

pxname="$1"
svname="$2"
stat="$3"

DEBUG=${DEBUG:-0}
HAPROXY_SOCKET="${HAPROXY_SOCKET:-/var/run/haproxy/haproxy.sock}"
QUERYING_METHOD="${QUERYING_METHOD:-SOCKET}"
CACHE_STATS_FILEPATH="${CACHE_STATS_FILEPATH:-/var/tmp/haproxy_stats.cache}"
CACHE_STATS_EXPIRATION="${CACHE_STATS_EXPIRATION:-1}" # in minutes
CACHE_INFO_FILEPATH="${CACHE_INFO_FILEPATH:-/var/tmp/haproxy_info.cache}" ## unused
CACHE_INFO_EXPIRATION="${CACHE_INFO_EXPIRATION:-1}" # in minutes ## unused
GET_STATS=${GET_STATS:-1} # when you update stats cache outsise of the script
SOCAT_BIN="$(which socat)"
NC_BIN="$(which nc)"
FLOCK_BIN="$(which flock)"
FLOCK_WAIT=15 # maximum number of seconds that "flock" waits for acquiring a lock
FLOCK_SUFFIX='.lock'
CUR_TIMESTAMP="$(date '+%s')"

debug() {
  [ “${DEBUG}” -eq 1 ] && echo "DEBUG: $@" >&2 || true
}

debug "SOCAT_BIN        => $SOCAT_BIN"
debug "NC_BIN           => $NC_BIN"
debug "FLOCK_BIN        => $FLOCK_BIN"
debug "FLOCK_WAIT       => $FLOCK_WAIT seconds"
debug "CACHE_FILEPATH   => $CACHE_FILEPATH"
debug "CACHE_EXPIRATION => $CACHE_EXPIRATION minutes"
debug "HAPROXY_SOCKET   => $HAPROXY_SOCKET"
debug "pxname   => $pxname"
debug "svname   => $svname"
debug "stat     => $stat"

# check if socat is available in path
if [ “$GET_STATS” -eq 1 ] && [[ $QUERYING_METHOD == “SOCKET” && -z “$SOCAT_BIN” ]] || [[ $QUERYING_METHOD == “TCP” &&  -z “$NC_BIN” ]]
then
  echo 'ERROR: cannot find socat binary'
  exit 126
fi

# if we are getting stats:
#   check if we can write to stats cache file, if it exists
#     or cache file path, if it does not exist
#   check if HAPROXY socket is writable
# if we are NOT getting stats:
#   check if we can read the stats cache file
if [ “$GET_STATS” -eq 1 ]
then
  if [ -e “$CACHE_FILEPATH” ] && [ ! -w “$CACHE_FILEPATH” ]
  then
    echo 'ERROR: stats cache file exists, but is not writable'
    exit 126
  elif [ ! -w ${CACHE_FILEPATH%/*} ]
  then
    echo 'ERROR: stats cache file path is not writable'
    exit 126
  fi
  if [[ $QUERYING_METHOD == “SOCKET” && ! -w $HAPROXY_SOCKET ]]
  then
    echo "ERROR: haproxy socket is not writable"
    exit 126
  fi
elif [ ! -r “$CACHE_FILEPATH” ]
then
  echo 'ERROR: cannot read stats cache file'
  exit 126
fi

# index:name:default
MAP="
1:pxname:@
2:svname:@
3:qcur:9999999999
4:qmax:0
5:scur:9999999999
6:smax:0
7:slim:0
8:stot:@
9:bin:9999999999
10:bout:9999999999
11:dreq:9999999999
12:dresp:9999999999
13:ereq:9999999999
14:econ:9999999999
15:eresp:9999999999
16:wretr:9999999999
17:wredis:9999999999
18:status:UNK
19:weight:9999999999
20:act:9999999999
21:bck:9999999999
22:chkfail:9999999999
23:chkdown:9999999999
24:lastchg:9999999999
25:downtime:0
26:qlimit:0
27:pid:@
28:iid:@
29:sid:@
30:throttle:9999999999
31:lbtot:9999999999
32:tracked:9999999999
33:type:9999999999
34:rate:9999999999
35:rate_lim:@
36:rate_max:@
37:check_status:@
38:check_code:@
39:check_duration:9999999999
40:hrsp_1xx:@
41:hrsp_2xx:@
42:hrsp_3xx:@
43:hrsp_4xx:@
44:hrsp_5xx:@
45:hrsp_other:@
46:hanafail:@
47:req_rate:9999999999
48:req_rate_max:@
49:req_tot:9999999999
50:cli_abrt:9999999999
51:srv_abrt:9999999999
52:comp_in:0
53:comp_out:0
54:comp_byp:0
55:comp_rsp:0
56:lastsess:9999999999
57:last_chk:@
58:last_agt:@
59:qtime:0
60:ctime:0
61:rtime:0
62:ttime:0
"

_STAT=$(echo -e "$MAP" | grep :${stat}:)
_INDEX=${_STAT%%:*}
_DEFAULT=${_STAT##*:}

debug "_STAT    => $_STAT"
debug "_INDEX   => $_INDEX"
debug "_DEFAULT => $_DEFAULT"

# check if requested stat is supported
if [ -z “${_STAT}” ]
then
  echo "ERROR: $stat is unsupported"
  exit 127
fi

# method to retrieve data from haproxy stats
# usage:
# query_stats "show stat"
query_stats() {
    if [[ ${QUERYING_METHOD} == “SOCKET” ]]; then
        echo $1 | socat ${HAPROXY_SOCKET} stdio 2>/dev/null
    elif [[ ${QUERYING_METHOD} == “TCP” ]]; then
        echo $1 | nc ${HAPROXY_STATS_IP//:/ } 2>/dev/null
    fi
}

# a generic cache management function, that relies on 'flock'
check_cache() {
  local cache_type="${1}"
  local cache_filepath="${2}"
  local cache_expiration="${3}"  
  local cache_filemtime
  cache_filemtime=$(stat -c '%Y' "${cache_filepath}" 2> /dev/null)
  if [ $((cache_filemtime+60*cache_expiration)) -ge ${CUR_TIMESTAMP} ]
  then
    debug "${cache_type} file found, results are at most ${cache_expiration} minutes stale.."
  elif "${FLOCK_BIN}" –exclusive –wait "${FLOCK_WAIT}" 200
  then
    cache_filemtime=$(stat -c '%Y' "${cache_filepath}" 2> /dev/null)
    if [ $((cache_filemtime+60*cache_expiration)) -ge ${CUR_TIMESTAMP} ]
    then
      debug "${cache_type} file found, results have just been updated by another process.."
    else
      debug "no ${cache_type} file found, querying haproxy"
      query_stats "show ${cache_type}" > "${cache_filepath}"
    fi
  fi 200> "${cache_filepath}${FLOCK_SUFFIX}"
}

# generate stats cache file if needed
get_stats() {
  check_cache 'stat' "${CACHE_STATS_FILEPATH}" ${CACHE_STATS_EXPIRATION}
}

# generate info cache file
## unused at the moment
get_info() {
  check_cache 'info' "${CACHE_INFO_FILEPATH}" ${CACHE_INFO_EXPIRATION}
}

# get requested stat from cache file using INDEX offset defined in MAP
# return default value if stat is ""
get() {
  # $1: pxname/svname
  local _res="$("${FLOCK_BIN}" –shared –wait "${FLOCK_WAIT}" "${CACHE_STATS_FILEPATH}${FLOCK_SUFFIX}" grep $1 "${CACHE_STATS_FILEPATH}")"
  if [ -z “${_res}” ]
  then
    echo "ERROR: bad $pxname/$svname"
    exit 127
  fi
  _res="$(echo $_res | cut -d, -f ${_INDEX})"
  if [ -z “${_res}” ] && [[ “${_DEFAULT}” != “@” ]]
  then
    echo "${_DEFAULT}"  
  else
    echo "${_res}"
  fi
}

# not sure why we'd need to split on backslash
# left commented out as an example to override default get() method
# status() {
#   get "^${pxname},${svnamem}," $stat | cut -d\  -f1
# }

# this allows for overriding default method of getting stats
# name a function by stat name for additional processing, custom returns, etc.
if type get_${stat} >/dev/null 2>&1
then
  debug "found custom query function"
  get_stats && get_${stat}
else
  debug "using default get() method"
  get_stats && get "^${pxname},${svname}," ${stat}
fi


! NB ! Substitute in the script /var/run/haproxy/haproxy.sock with your haproxy socket location

You can download the haproxy_stats.sh here and haproxy_discovery.sh here

2. Create the userparameter_haproxy_backend.conf

[root@haproxy-server1 zabbix_agentd.d]# cat userparameter_haproxy_backend.conf 
#
# Discovery Rule
#

# HAProxy Frontend, Backend and Server Discovery rules
UserParameter=haproxy.list.discovery[*],sudo /etc/zabbix/scripts/haproxy_discovery.sh SERVER
UserParameter=haproxy.stats[*],sudo /etc/zabbix/scripts/haproxy_stats.sh  $2 $3 $4

# support legacy way

UserParameter=haproxy.stat.downtime[*],sudo /etc/zabbix/scripts/haproxy_stats.sh  $2 $3 downtime

UserParameter=haproxy.stat.status[*],sudo /etc/zabbix/scripts/haproxy_stats.sh  $2 $3 status

UserParameter=haproxy.stat.last_chk[*],sudo /etc/zabbix/scripts/haproxy_stats.sh  $2 $3 last_chk

 

3. Create new simple template for the Application backend Monitoring and link it to monitored host

create-configuration-template-backend-monitoring

create-template-backend-monitoring-macros

 

Go to Configuration -> Hosts (find the host) and Link the template to it


4. Restart Zabbix-agent, in while check autodiscovery data is in Zabbix Server

[root@haproxy-server1 ]# systemctl restart zabbix-agent


Check in zabbix the userparameter data arrives, it should not be required to add any Items or Triggers as autodiscovery zabbix feature should automatically create in the server what is required for the data regarding backends to be in.

To view data arrives go to Zabbix config menus:

Configuration -> Hosts -> Hosts: (lookup for the haproxy-server1 hostname)

zabbix-discovery_rules-screenshot

The autodiscovery should have automatically created the following prototypes

zabbix-items-monitoring-prototypes
Now if you look inside Latest Data for the Host you should find some information like:

HAProxy Backend [backend1] (3 Items)
        
HAProxy Server [backend-name_APP/server1]: Connection Response
2022-05-13 14:15:04            History
        
HAProxy Server [backend-name/server2]: Downtime (hh:mm:ss)
2022-05-13 14:13:57    20:30:42        History
        
HAProxy Server [bk_name-APP/server1]: Status
2022-05-13 14:14:25    Up (1)        Graph
        ccnrlb01    HAProxy Backend [bk_CCNR_QA_ZVT] (3 Items)
        
HAProxy Server [bk_name-APP3/server1]: Connection Response
2022-05-13 14:15:05            History
        
HAProxy Server [bk_name-APP3/server1]: Downtime (hh:mm:ss)
2022-05-13 14:14:00    20:55:20        History
        
HAProxy Server [bk_name-APP3/server2]: Status
2022-05-13 14:15:08    Up (1)

To make alerting in case if a backend is down which usually you would like only left thing is to configure an Action to deliver alerts to some email address.

Christ is Risen ! Truly He is Risen ! The origin of the tradition Paschal Greeting and Coloring of Eggs on Easter Holidays in the Church


April 27th, 2022

 

Christ-is-Risen-Truly-he-is-risen-and-the-christian-origin-of-red-eggs-worldwide-Christ-triumphant-icon

Christ is Risen ! Truly He is Risen !

Христос воскресе ! Воистину воскресе ! (Khristos voskrese! Voistinu voskrese!) – Church Slavonic Paschal Greeting

Χριστὸς ἀνέστη!  Ἀληθῶς ἀνέστη ! (Khristós anésti! – Alithós anésti!) – Greek Paschal Greeting

Christus Resurrexit ! Resurrexit Vere ! – Latin Paschal Greeting

The Easter Eggs are so famous today for the kids worldwide, even though the world does not put much accent on the feast of Pascha (Easter). All kind of colored eggs are to be find in stores, many christian countries both Western and Eastern all throughout the world have the tradition of coloring eggs for Easter. 
The tradition is also the same here in Orthodox Bulgaria, as we have the tradition to boil and color eggs in various colors. 
Usually the first egg is colored in Dark Red and once sanctified in the Church is put on the iconostasis (the prayer corner in the house) in front of Christ, Virgin Mary and saints icons and kept their until next year.

Miraculously this Egg usually does not start decaying or smelling as an ordinary egg will do if left out of the fridge for a month or so. This first egg in dedication and memory of Christ's resurrection is kept on the iconostasis until the next year's Pascha and then buried somewhere in a green clean place for sanctification of the land.

This is a good and well followed tradition for those strict about religion, but even those who did not strictly follow Christianity or orthodoxy do color eggs for the fun of kids and as an expression for joy of the Paschal feast. Both grown and kids then try out their forces whose egg is more powerful by knocking each other's eggs to test whose egg shell is more solid and can stand up the break. The egg that is "victorious" once people test their power that is stronger and withstand the "egg fight" is kept for another egg duel with another person.

According to old superstitious belief if you win over in an eggs fight this is interpteted as you will have a good health and well being for the upcoming period till next year's Pascha.
 

How and from wherein this Boiled Eggs coloring originated ?


The short answer is it is connected to one of Church's traditions about the poor apostle Equal Saint Mary ( Maria ) Magdalene, who have given as a gift to Roman emperor Tiberius an Egg with the All Famous Greeting dialog in the Orthodox Churches among people with person A saying:   Christ is Risen ! person B responding: Truly he is Risen! (Христос Воскресе Войстину Воскресе !)

From the time of the many appearances of the Savior Christ in flesh after his Glorious Resurrection described by the Holy Evangelists in the Gospels and the fervent sermon of St. Mary Magdalene (one of the of the so called Myrrh- Bearing Woman who were the first who have visited the tomb where Dead body of Christ was led and become witnesses of  the Resurrection). The surviving Bible's New Testament 4 Gospel books do not provide further details about the activities of St. Mary Magdalene and her life. The Church mouth-by -mouth tradition of her later life in several local Christian churches differ slightlyhowever everywhere they essentially report on the zealous co-apostolic activity of St. Mary Magdalene. And the differences between these traditions depend on which of the evangelical women these churches understand by the name of St. Mary Magdalene.

Some Western Christian churches, as well as the Church Fathers and learned theologians, unite in one or two personalities three evangelical women: the sinner who repented in the house of Simon the Pharisee, shed tears at the Savior Christ's feet, wiped them with her hair. and she anointed them with precious ointment, and Mary of Bithynia, sister of Lazarus of Bythynia ( resurrected in the fourth day after death by Chrsit and commemorated one day before Palm Sunday )' and Mary Magdalene, who was delivered from the Savior Christ by seven demons. But the Orthodox Church now, as before, recognizes those mentioned in the Gospels with different signs, three persons as different separate ones, and does not want to base historical information on arbitrary, plausible interpretations. Therefore, the tradition of the Orthodox Church states that after the Gospel appearances of the Risen Christ before His Ascension and after, St. Mary Magdalene resided with the Blessed Virgin and the Apostles and was an active helper of the first successes of spreading the Christian faith first in Jerusalem. But full of zeal, fervent faith, and zealous love for God's gospel, she then preached in other lands, proclaiming everywhere the heavenly grace, joy, and salvation of all who believed in the Savior of the world, the Risen Christ.

Saint-Mary-Magdalene-gifting-red-egg-to-emperor-Tiberius-Orthodox-icon-one-of-Myrrh-Bearing-Woman

While visiting Italy to preach, St. Mary Magdalene found an opportunity to appear before the then-reigning Emperor Tiberius I, and presented him, according to generally accepted Eastern custom, with an egg painted red and greated him with "Christ is risen!"

The modesty of the gift of Mary Magdalene did not surprise the emperor, because he knew the ancient custom of the East, also among the Jews, going for the first time to superiors, or on solemn occasions to acquaintances or patrons, to offer a gift of honor, with some known or special, symbolic meaning. Examples of this can be found in Jewish Old Testament history, as are the gifts presented by the rich Wise men (magis – today their relics are kept for veneration in Cathedral of Cologne Germany) to the born Jesus Christ in Bethlehem of Judea. Even the poor in such cases offered as a gift various fruits from their locality or eggs from birds. Thus, partly following this ancient custom and with the red color of the egg laid and with the hitherto unheard words "Christ is risen!" to arouse the curiosity of the suspicious Emperor Tiberius. The holy co-apostle Mary Magdalene, by explaining the significance of this gift, began her fervent sermon on the Resurrection truth and the teachings of the Savior Christ for salvation. With great inspiration and conviction she told the emperor about the life, miracles, crucifixion and resurrection of Jesus Christ according to his own prophecy. She gave a direct, simple-minded account of the extremely unjust, biased judgment of Jesus Christ by the embittered members of the Jerusalem Sanhedrin. governor of Judea Pilate of Pontus, in condemning Jesus Christ to crucifixion. She explained how all this incurred the wrath of the Roman emperor then and how Tiberius handed them over to a court in which Pilate was deprived of power and exiled to Gaul, in the city of Vienna, where, according to legend, tormented by remorse and despair, he killed himself. According to another legend, Pilate repented, turned to Christ in prayer, as a sign of which his head was accepted by an angel after being cut off.

According to Church tradition, the sisters of Lazarus Martha and Mary went to Italy with St. Mary Magdalene; and Pilate, learning of this and fearing the denunciation of his unlawful actions by the Christians, himself sent a message to the Emperor Tiberius about Jesus Christ, in which he testified to the virtuous life of Christ, the healing of all diseases and infirmities from Him, even for the resurrection of the dead and for His other great miracles. Pilate asserted that in examining the accusations of the Jews, he found no fault in Jesus Christ; he made great efforts to deliver Him from the hands of the troubled Jews, but failed to deliver Him and betrayed Jesus to their will because of the cries of the people and the rebellious accusation of the Jews against Pilate himself …

 

... as a witness, overwhelmed with fear, he told the emperor about everything that had happened to Jesus Christ, who became an object of faith as God …

After such testimonies from the Roman governor of Judea and the worshipers of the Savior Christ, Emperor Tiberius, according to legend, himself believed in the Savior Christ, proposed to include Jesus Christ in the image of the Roman gods, and even when the Roman Senate rejected this proposal, Tiberius by royal decree threatened to punish anyone who dared to grieve believers in Jesus Christ.

In this way, with the zealous, fearless sermon on the Savior Christ, St. Mary Magdalene, along with other devout Christians, persuaded the pagan governor of Judea to testify in writing about the universal event of Christ's Resurrection before the pagan world and persuaded the then Roman emperor of the Savior Christ, thus facilitating the spread of Christianity.

Anastasis-Hristos-Voskrese-beautiful-orthodox-Mosaic
Anastasis (Resurrection) Church Fresco


And the Christians of that time, learning about the significance and strength of the impression caused by the offering of a red egg by Mary Magdalene to Emperor Tiberius with the words: "Christ is risen!" then began to imitate her in this and as a remembrance of Christ's Resurrection they began to give each other red eggs and say: "Christ is risen! … He is risen indeed! …"

Thus, this custom gradually spread everywhere and became universal for Christians around the world. In it, the egg serves as a symbol of Christ's resurrection and the resurrection of the dead, and of our expected new-birth for eternal bliss in the future life, the pledge for which is Christ's Resurrection.

Just as a bird is born from an egg and begins to live an independent life after its release from the shell, and the vast circle of life is revealed to it, so we, at the second coming of Christ to earth , rejected from ourselves together with the earthly body all that is mortal on earth.

By the power of Christ's Resurrection we will be resurrected and resurrected to another, higher, eternal, immortal life.

And the red color of the Easter egg reminds us that the redemption of mankind and our future new life have been acquired through the shedding of the cross on the pure blood of the Savior Christ.

Thus, the red egg serves to remind us of one of the most important dogmas of the Divine revealed Christian faith.

 

After the crucifixion of Jesus by the Jews in terrible miracles took place in nature, many dead righteous people rose, with His resurrection on the third day.Pilate, as a witness overwhelmed with great fear, informed the Caesar of all things that had happened to Jesus Christ.

In Eastern Orthodox Tradition Christ is Risen ! Truly he is Risen Greeting is used to joyfully great each other all around the Orthodox countries in the first 3 days of easter, and can be used instead of normal Hello greeting ! for the upcoming week The Holy Easter Weak which is a week of great joy and even by a hello greating in the Church could be used for 40 days as a normal greeting.

It is worthy to close this article with the praisal words, read on the first day of Pascha  authored by one of the most important Church fathers and
compiler of most served Liturgy service throughout the yearly service calendar:

"Christ is risen, and you are overthrown!
Christ is risen, and the demons are fallen!
Christ is risen, and the angels rejoice!
Christ is risen, and life reigns!
Christ is risen, and not one dead remains in a tomb!
For Christ, being raised from the dead, has become the first-fruits of them that have slept."

Saint John Chrysostom