Archive for the ‘Everyday Life’ Category

Run custom user script after reboot with a cronjob on Linux

Friday, September 21st, 2018

howto-add-custom-script-on-reboot-with-non-administrative-root-user-on-gnu-linux

Perhaps you have a websites on a server on some Linux distro / FreeBSD / AIX / HP-UX / Sun OS that uses Vixie-cron cron jobs to run / respawn dead php / python perl scripts etc.  that do stuff on the server every lets say 30 minutes an hour or even every 12 / 24 hours in the background.
But sometimes due to server or Linux kernel upgrades you need to reboot the server with reboot command or shutdown -r now right in the minutes the scripts were supposed to run and do a database backup / synchronize some data with a remote MySQL with replication configured or do some site maintenance job such as clearing old Messages / Spam / data log file records.

Of course one possible workaround to that is to add the non-root user scripts in question  to /etc/rc.local to run on every server boot, but that fix requires a root access and very often developers did not have that, neither sysadmins are willing to bother  add a user sudo-ed scripts e.g. add  (sudo -u whateveruser "/path/to/script") to /etc/rc.local.

Run custom user  script after reboot the cron way

Happily there is ctually a better cron way to do that by telling crond to execute a cronjob during boot and assuming the non-admin user on the Linux has access to shell and access to using cron jobs by using @reboot cron direcive.

Here is few examples on how to run a re-run cron job on start up:
 

linux:~$ crontab -e


Some editor as nano or vim will open listing all your previous set system jobs to add scripts phpjob.php

@reboot  /user/dir/path/to/phpjob
@reboot  /path/to/shell_script
@reboot  /path/to/linux-command

That's pretty shitty situation but thanksfully remote access of website username with SSH will be enough to set the right cron activity (of course this can't be made for servers that are missing crond service running.

The scripts set in cron job that way will respawn right after the OS system had booted and there will be no need for them to wait the next hour to execute configured data synch.
For more on how to run a tiny script respawn every second using a single cron job check out my previous article How to set a crontab to execute commands on a seconds time interval on GNU / Linux and FreeBSD.

Historically it is interestingly to mention that in times before systemd appeared in modern Linux distributions,
a cool thack to run a script that had to be respawned every second after boot for a privileged user was to use /etc/inittab (no longer available in most all non System V Linux distrubutions in 2018), to do so
if you happen to still administer some old Linux servers CentOS 7 etc. you and you need to add a custom script to run and respawn all the time by including a line in /etc/inittab (again assuming a System V Linux is on remote machine):
 

mysvc:235:respawn:/home/me/bin/my_service_starter_script

Putting a service to respawn in that way via inittab uses init (process) and the kernel and keeps re-running it.

Note: 

If a command fails when it starts, and init is configured to restart it, it will use a lot of system resources: init starts it, it fails, init starts it, it fails, init starts it, it fails, and so on, ad infinitum. To prevent this, init will keep track of how often it restarts a command, and if the frequency grows to high, it will delay for five minutes before restarting again.If the kernel 
Using inittab should always be tested on a testbed before adding to remote server, note that if the script is using a lot of memory and keeps crashing it can easily leave out the kernel without memory and the system is about to get errors like:
 

process respawning too fast 

 

Another useful thing if you have doubts that the script might be crashing is to use something like monit to monitor the script (assuming the script does provide some kind of tcp / udp connection on port) and report you via email / sms about issues with crashing script.
If you hear monit the first time I recommend you read my previous article Monitoring and restart server services (Apache, Mysql, Bind) with Monit to prevent server downtimes.

Install Slack and Mattermost clients for Start Up Business communication on Linux

Wednesday, September 19th, 2018

install-slack-and-mattermost-clients-for-start-up-business-communication-on-Linux
Many businesses nowadays are lookig for alternatives to the Microsoft dominated market of communication – Skype / Skype for Business Chat Audio and Video desktop client.
The two are the defacto standard for most of Corporate Businesses and is heavily used across most largest Corporations (companies) such as IBM / Xeror / DXC / CSC / Oracle / SAP / Microsoft / Amazon / Adobe … the list goes on and on.

However even though Skype is so easy to use across Microsoft Domain connected Computers the many start-up companies of today often try to avoid its use. The reason, well Skype is totally Proprietary non-transparent and by using it you probably get spied by Microsoft the CIA and God knows how many other Country Agencies. Besides that Skype has a bad history often had problems with Audio (Linux microphone and Video settings) in Free Software (Linux, FreeBSD etc.) realm and even though nowdays situation is improving and Skype Video / Audio runs fine on GNU / Linuxes its Skype for Business has no working release by Microsoft and has left-up Free Software users and Staring Business companies platform of the size of 20 to 1000 people  that choose Linux as a main Desktop / Work had to look for other ways to communicate internally within company and with clients.

Jabber XMPP communication protocol has been one alternative for a long time and historically many compainies that were running out of Skype use for their work often were using Small internally company hosted jabber servers, however as Jabber's  communication clients such as Gajim development is lagging behind seriously over the last 7 years and it prooved so buggy many businesses were looking for ways to avoid it.

Slack_Technologies_-corporate-communication-alternative-to-skype-Logo.png


Slack is multi platform just like Skype and has versions for Linux / Windows for macOS but its power comes mostly because most of its users use it via Skype Web Client (while Skype is a Desktop app and heavily used in Web Browser.

slack-web-communication-client-in-webbrowser-screenshot

Slack reminds in a what of things to the good old IRC chats and has channels in a similar fashion, it support Audio conversations but unfortunately at the moment didn't support Video.

The emergency of New Age of Computing and the quick adoption of Clouds as an aim to cut business costs put Jabber totally out of the game and in the niche and in August 2013 on the scene raised Slack (Team Messaging) which is an acronymi of (Searchable Log of All Conversations and Knowledge) – a cloud-based set of proprietary team collaboration tools and services, founded by Stewart Butterfield for the purpose of online Game (now defunct) called Glitch.

asana-for-slack-integration-2018-2-linux
The problem with Slack is that it is a freemium product, whose main paid features are the ability to search more than 10,000 archived messages (the ordinary free version allows the user to make up to 10000 searches in chat history), the paid Slack versions adds also unlimited apps and integrations and theoretically unlimited number of users (though this is seriously doubtful).

slack-10000-messages-per-user-limitation-shot

One very handy feature of Slack is its integration with "The World's Leading Software Development Platform" – GitHub .

To solve the problem with the little amount of Slack Chat history in Users conversations many Start-Up Business Companies do use Slack as a communication media with Clients and does often use as a communication Media another very popular Cloud messaging Open Source Software called Mattermost deploys to Cloud infrastruture but is at IT control of your company or your hired support and not third party vendor supports, making it a great communication tool for small and mid-sized companies who want to save money of purchasing a special server and hiring an admin or paying for one to support it all the time but instead directly use their Cloud account and deploy it there.

Mattermost is capable to Reach anyone, anywhere on any device. From the airport to data center, safely connect teams with EMM apps, hybrid cloud deployment and enterprise-grade flexibility to meet the unique needs of enterprise.

It is capable to Integrate with existing applications and build new workflows and empower your teams especially operations and DevOps – to perform faster and effectively. Mattermost


To install Slack on Linux:

Go and Download slack from Slack Linux download (the DEB / RPM 64 bit package)

As of time of writting this article latest Slack Desktop packages are: slack-desktop-3.3.1-amd64.deb and slack-3.3.1-0.1.fc21.x86_64.rpm

Depending on the type of Linux distribution install it with dpkg or rpm

1. Installing Slack Desktop client on Debian / Ubuntu Linux

On Debian / Ubuntu / Mint install Slack with:

 

root@ubuntu:~# dpkg -i slack-desktop-*.deb


For Ubuntu users there is also unofficial third-party Slack app ScoudCloud

it integrates well with Ubuntu Unity desktop (which I personally dislike 🙂 ) and gives you some extra goodies such as showing in an Unity manner unread message count, notification, bubbles, unity quicklists for fast-switching between Slack channels etc.

2. Installing Slack Desktop client on Redhat / Fedora / CentOS Linux

On Redhat / Fedora / CentOS install it with:

 

[root@fedora ~]:# rpm -ivh slack-*.rpm

 


3. Installing Mattermost Desktop client on Linux

mattermost-open-source-communication-in-the-cloud

Download Mattermost Linux package from download URL here

As of time of writting the DEB versions are mattermost-desktop-4.1.2-linux-amd64.deb mattermost-desktop-4.1.2-linux-i386.deb and there is no official RPM package for Fedora / CentOS users however I guess the .deb package can easily be converted to .rpm with alien tool.

To install Mattermost on Debian (in moment of writting September 2018):

 

root@debian:~# wget https://releases.mattermost.com/desktop/4.1.2/mattermost-desktop-4.1.2-linux-amd64.deb

 

root@debian:~# dpkg -i mattermost-desktop-4.1.2-linux-amd64.deb


mattermost-linux-client-2-screenshot

Mattermost supports file attachment (send) / Video previews (you can play sent Web videos directly within the Mattermost client) and on experimental level even supports Video and Audio Calls.

mattermost-linux-client-screenshot-1

mattermost-markdown-help-linux-screenshot

One nice feature of Mattermost for those who love coding is using tags to format messages

There is plenty of features of Mattermost among the best ones are integrations Private Cloud open source integrations (Jira, Jenkins, Bots, clients), supports Webhooks, Restful APIS, CLI and Public Cloud connections via Zapier – a connect and uatomate workflows e.g. gives you ability to move info between web apps automatically.

Happy New Church Year ! – The Eastern Orthodox Church Year celebrated every 1st of September and some history on the Indiction

Wednesday, September 5th, 2018

Eastern_Orthodox-Church-new-year-indiction

Happy New Year, Happy New Church Year !  🙂

On first of September every year the Holy Eastern Orthodox Church to which I belong is celebrating the beginning of a new Church Calendar feast year.

It is a common practice here in Bulgaria (as we are one of the 14 Eastern Orthodox Churches worldwide) it is a tradition to serve holy liturgies across the country and prayers are lifted to heaven for the well being of the Bulgarian nation and the world.

In the past the beginning of the new year was called also the beginning of the indiction.
In the VI-th century at the time of Emperor Justinian The Great in Byzantine Empire was introduce the Cyclical Calendar based calculation of days (based on indictions ) – this is delivered from latin word indictio (meaning announcement).
Indictions
were celebrated originally on a cyclical periods of 15 years when the а financial review of the properties (possessions) of citizens  that were taxated.

In the Latin Empire the word indictio has been used to mark the exact amount of taxes owed each year and in this is how with the beginning of the new indiction was beginning the new financial year of the empire according to some historians the origin of indictions system has been originating from Egypt.

The official byzantine calculation of indictions begun at the time of Saint Emperor Constantine on 1st of September year 312 A.D.
Historically it is interesting fact both in the Western and in the Eastern Latin empire March was another time marking a New Year …

On 1st of September in the Church it is accepted to read the Bible chapter where Jesus Christ red in the synagogue the prophecy of Isaiah (Is. 61:1-2) for the occurance of the favorable year of God. The Byzantines see in that reading a reference of the Lord for a celebration of a New Year. 

In the Menologion of Basyl II from the 10th century it is said "From that time, he has granted to us the Christians this feast".

Primary_Chronicle_Povesti_Vremeninnih_let-baptizmal-of-Olga

It is interesting fact that in Byzantium this Bible reading was red in the Church by the Patriarch in Hagia Sophia Church Cathedral or another Church where it was common for him to serve Holy Liturgy on that date.

In Russia even though the Baptismal and Christianization of the nation until 15th century the Church New year begun on 1st of March. This is evident from all medieval manuscripts including The most famous and among most ancient Russian manuscript written by the eminent Nestor so called Primary Manuscript (Primary Chronicle) also called Tale of Begone Years.
 

With an Emperor decree of King Peter the I-st (The Great) the new year in Russia is transferred to 1st of January.

Saint_Simeon_Stylites-Stulpnik

Saint Simeon the Stylites

Our Bulgarian Orthodox Church by tradition according to the main Church service instructions book – the Typikon marks on the first day of the new year as a very important "tiny Lord feast" and has tradition to celebrate the memory of saint Simeon the Stylites who spend mostly all his life praying on a Stylite and was a notable ascet of the Orthodox Church.

 

Recovering from Herniated Cervical disc (neck) and spine disease – keeping in good shape your Back and Neck in easy 4 steps Exercises

Friday, August 31st, 2018

recovering-from-herniated-cervical-disc-and-spine-disease-keep-your-back-in-good-shape

If you are a long term computer geek, IT specliast, system administrator a hacker or any type of office worker that spends as a maximum 8 hours / 5 days a week and you do not do some kind of Sports (Running Heavy Lifting, Back or neck Exercises, do Fitness) etc. sooner or later you will start noticing a change in your normal back posture and heavy pain either in back or neck of your body.

The posture you have is in risc and your muscles will slowly start atrophying. It is very likely with years that you will start developing a body muscle strain in the area of neck or back and you will feel tensed / anxious most of the time without a clear reason on what is causing the constant stress you experience.

A heavy and constant anxiety you might develop with the months and years of chair sitting, sedentary lifestyle and staring into the computer screen or spending many hours driving might lead you into sleeping problems a bad blood flow to the brain (if you have neck problems) and other not nice health issues such as ringing in your year / head (tinnitus) and as an impact of all the degeneration of body a change in bone tissues and even brain might happen.
If you're suffering a back spine or neck disease the pain it is about to create daily for you will also impact the level of depression you experience and often it alone could be a cause for depression, because of the violated bodily organs blood flow (especially the brain) …

That's heavy health issues and if you are young you might thing that all this is just an imaginations and it will never happen to you but unfortunately in today's digital age where people tend to spend 10 to 14 hours daily in front of PC notebook / monitor and smartphone or in few hours conversations on the phone, all this with years contributes and sooner or later you will face the terrible reality of suffering from some of the many existing types of Herniated Cervical Disc disease.

So what is Herniated Cervical Disc Disease (shortly explained) ?

Spinal disc herniation, also known as a slipped disc, is a medical condition affecting the spine in which a tear in the outer, fibrous ring of an intervertebral disc allows the soft, central portion to bulge out beyond the damaged outer rings.

The reason to get a herniated cervical disc is usually due to aging a genetical predsposition, constant straining of back due to many hours without physical activity sitting or in bed, traumas, injuries attained during heavy lifing or sports etc.

* What are the symptomps of Herniated Cervical disc ?

The symptoms vary depending of the location of a herniated disc and depends on the soft tissue between the bones that gets involved. Common symptoms however is severe and unrelented neck pain, low back pain, pain in knees or feet, undefined pains in thight etc. … it really depends by the affected region served by affected nerve roots that are irritated or impinged by the herniated material.

Some people suffering from spinal disc herniation is symptoms of numbness, tingling of limbs, overall muscular weakness, paralysis and effection of reflexes, double vision, noise in the head / ears, insomnia and more …

If the herniated disc is in the lumbar region, you will experience scietica (a pain going down the leg from the lower back).

The things becomes even worser and the risk even higher if you're not using a comfortable computer chair at work or home, but you are into the habit to use a laptop or as the girls / women like to do lay on the bed and do stuff on the computer or tablet such as chatting all day long in a social networks such as Facebook or watching movies after a heavy office day.

Just take a look at young people's necks around leaning up and their crooked back posture on the street in bus or metro if you don't believe me and thing and you will be amazed how terrible deformations so many youngsters (teenagers) and even mid age person suffer some early or advanced stage of kyphosis disease (a deformation of the spine / neck) without even realizing it.

Herniated-Lumbar-Disc-pinched-nerve

Suffering Kyphosis (and the usual surrounding Herniated Disc illness) is a terrible thing to experience and this is a silent killer that is plaguing people worse than any other disease, because the disease is developing relatively slow in time it is few that really catch it in early stage and it usually is tracked once you already suffer with some kind of constant pain in neck / back and a Spinal disc herniation or  you had a pinched nerve somewhere in the body because of some kind of constant compression in between spine bones etc.


cervical-herniated-disc-cervical-disease

The majority of spinal disc herniation cases occur in the lumbar region (95% in L4-L5 or L5-S1 – this is a doctors naming for regions where it occurs). The second most common site is the cervical region (C5-C6, C6-C7). The thoracic region accounts for only 0.15% to 4.0% of cases.

For IT specialists Gamers and mostly and people whose profession includes spending their days sitting in front of the computer screen C5-C6 / C6-C7 regions are very common (something that could contribute additionally for the disease is if you suffer some kind of astigmatism eye disease and because of that you keep your neck constantly even more forward to see better what's on the PC screen)

aging-and-body-back-deformation-in-grown-and-aging-people

With age spine and bones are decaying (wearing out) and as a result the spinal cord gets deformated and this is a normal process, we can't change but we can delay it with some daily exerices and work on our bodies so at old age we experience a little better health ..

Well dying is ineveitable anyways 🙂 .. but why not make your quality of life and overall experience in the body a little bit better and do some exercises that will boost your mood to happiness, improve overall blood flow and either help you recover or protect you against a possible herniated disc problems?

I was to a doctor today for a Manual Therapy session, because I happen to suffer from Herniated Cervical Disc ( neck on C6 and C7) which bringing me a lot of unfortable nervous pressure and messing up with my quality of life.
To deal with my problem I went to see one of the best doctors in the field of Physical Medicine and Rehabilitation in Bulgaria and perhaps in the World Assen Aleksiev M. D. Phd in Aleksandrovska University Hospital.

Mr Assen Aleksiev is living legend and one of the bright persons as of time of writting this article he is:

1. Physician – specialist and expert in:

• Physical Medicine and Rehabilitation (with emphasize on orthopaedic and neurological disorders),

• Manual Medicine (Osteopathy, Chiropractics)

• Clinical Electro-diagnostics, Electromyography and Electro-stimulation.

2. Lecturer., 3. Scientist/researcher., 4. Reviewer of medical literature., 5. Author and co-author of medical literature., 6. Manager., 7. National consultant for Physical Medicine and Rehabilitation 2014.
 

After seeing my case and having a number of cases, just like mine and did some Manual therapy fixing my neck back,  limbs etc.

I will share with you the 4 steps, he said I and anyone has to follow in order to prevent himself from Herniated cervical disc degenerative disease to progress  and in worst case to turn me into a mildly disabled.

Even though this exercises were prescribed as a cure and support mechanism for a herniated disc patient that I turned to be, the good doc. shared with me that this set of 4 steps if followed could prevent not only your spine and waist from experiencing the painful discomfort of herniated disc diasese but could generally strengthen the body functioning and protect you from dying from a future strokes (heart attacks) and a number of other cardiological diseases.

So here are the 4 Golden Exercises (STEPS) that, if practiced regularly  will keep you from spinal curvature, will keep your neck and back in a good motion and help you recover from herniated disc diseases.


1. ALWAYS TIGHTEN YOUR STOMACH WHEN YOU SIT DOWN OR STAND UP !!!

– Make the habit to always tighten your stomach when you sit and stand up, this moves the weight of the body on the muscles instead on the bones. If you sit down or stand up without tightened stomach (because of the gravitation the body weight you have increases doubles if you're 70 kilos the body / bones have to bear a weight of 140 kg or so …). Once, again  tightening the stomach makes the weight be received by your muscles and this saves your bones

2. CHANGE THE POSITION OF THE BODY EVERY 15 MINUTES (FOR EXAMPLE YOU'RE SITTING FOR 15 MINUTES ON A CHAIR STAND UP MOVE YOUR SHOULDER FOR 2 / 3 AND SIT BACK) !!!

– Stand up from your chair or whenever you sit and move around your shoulders

3. ON EVERY HOUR PRACTICE SCHOOL (GYMNASTICS) EXERCISES !!!

– 5 Table push ups (if you're in an office for example in a Focus Corner Room) or outside the office building on a bench or something) – compulsory is to make it with a tightened stomach

table-push-ups-Counter-Push-ups-1

Table Push ups (the table angle inclination degree should be at best case about 45 to 60 degree but if it is different angle it is okay as well)
wall-push-up-exercise
if you are in a place where you can't do a table push-up you can do a wall push up

Waist Rotation (Exercise) Put your arms on the waist (in a trinagle form), your legs should be a little bit far from each other in a small triangle like form too and turn your whole body left and then right do it 3 to 5 times each side.

waist-rotation-exercise

Spinal Stretch Exercise (Place fingers at the base of the spine bend forward and backward then left side and right side slowly! do it each direction at least 3 to 5 times (e.g. 5 backwards / forward, 5 left and 5 times right)

spinal-stretch-exercise-forward-and-back

The 8 tact exercise is a complex of 4 exercises each on repeated 2 by the (count number 1, 2, 3, 4, 5, 6, 7, 8)

1, 2, 3, 4 – Put your heads on the level of the chest (bent them so the elbows point right and left then move  them both right and left to the maximum back and stretch cross shaped your both hands

5, 6 – Standing still move Right hand up and move it backwards as much as you can left hand down and move it backwards as much as possible

7,8 – Reverse the hands, Left hand up and move it backwards to maximum and right hand down and move it backwards to maximum

6,8

4. 3 TIMES A DAY (MORNING, MID DAY AND 1 HOUR BEFORE GO TO BED DO A SET OF 15 MINUTES ACTIVE CARDIO TRAINING EXERCISES) !!!

– This could be 15 minutes of running , jumping, push ups, heavy lifting, biking whatever you like to do but let the heart (pulse) rate be heavy loaded …

cardio-Exercises_Guide-you-can-do-at-home


5. Hanging on a lever (recommended not by the good doctor but by Dr. Georgiev e.g. hip0 myself) 🙂

Just find a lever and hang for about 60 seconds every day

Hanging-a-lever-helps-Spine-decompression-relieves-symptoms-of-herniated-disc
 

Following this routine of exercises: will not cost you too much of a time daily

a. It is (3×15 minutes cardio) is 45 minutes of exercise (and maybe another 5 to 15 minutes to go to where you practice it) = 60 minutes (1 hour)

b. Each hour 1 minute of exercises (approximately 14 hours is 14 minutes or at worst case if it takes you two minutes to do the complex of exercises 2×14 = 28 minutes.

c. tightening the stomach once learned is 0 seconds and 0 minutes.

d. hanging (if you choose to do this as well) is 60 seconds (1 minute) – going to the lever is another 3 to 5 minutes at worst case (lets say an overall of 6 minutes)

f. change position of the body every 15 minutes (costs you just up to 5 secs) or (14 hours x (4 x 15) = 60 secs) – that means another 14 minutes.

So the overall time for keeping your spine and neck in a good shape is (28 + 14 + 6 + 0 + 60 minutes) = 108 minutes, that is less than 1 hours and 48 minutes at worst scenario a day.

One helpful program in oder to remind you to not forget your exercises schedule if you choose to give it a try is Workrave see here my previous article on how it can help you Preserve health  from computer Caused Strain Injury (RSI).

But the advantages of following this exercise routine would be your:

1. bettered overall health,
2. less noise in head,

3. better digestion,

4. less strain and pain in neck and back,

5. feeling less anxiety and stress,

6. feeling happier (more Serotonin and Dopamine and other liquids that are boosting mood and helping brian work are pumped up more regularly into the brian),

7. a better concentration / focus on things (something most computer specialists and computer related jobs people miss seriously,

8. a better memory and a better sleep + following the standard school gymnastics exercises is much more safe when compared to Fitness,

9. Less Body Mental Fatigue (as most people with Herniated cervical disc suffer from a body fatigue that often affects the mind and creates a mental fatigue)

10. You don't risk to receive a body or muscle trauma that will last for a life time 🙂

Well, I hope that article helps someone with a herniated disc out or at least serve as a BIG WARNING and push people involved in IT, Digital Marketing, Programming, Logistics and mostly all professionals who require a long hours of hovering in front of the PC to realize the great dangers all this keeps for their health and get more physical activity daily to protect timely from ending up with a spinal digenerative disease and suffer 'till the end of life.

I will be mostly glad to hear if anyone had a personal experience with herniated disc and any spinal / neck degenerative disease and give me more hints on useful things that help in living a life with a herniated disc disease?

Thanks for reading  !

Classical System Administration is dying – you either say hello to DevOps and SRE or move to programming or other business if you can

Wednesday, August 29th, 2018

sysadmin-hell-being-a-sysadmin-is-easy-its-like-riding-a-bike

1. Back in the normal computer old Sys Admin days before the new Age of Computing (the Cloud HELL)

I've been in the system (server) administration business for more than 15 years. We started as kids dreaming about managing big Data Centers having ultimate control over servers data and services and in a sense the beginning of the 2000s looked like the system adminsitration will be among the most promising and profitable professions for the coming 30 years or so.

The amount of servers installed were booming, the Domain Registrantrant Ballon (Dot-Com Bubble) and the appearing need for everyone to have and run a website with the connected hardware and software (OS) needs made the sysadmin of the time like a precious asset for a company and business …

Many companies (small and mid-sized) still did not have a separate role for sys admin, but hired some crazy IT enthusiast that was doing a lot of the sysadm job for them.

It was wild years of freedom for the common IT specialist with a server software install / update / maintenance background.

The complexity level to install configure or tune for performance a (UNIX) like server be it GNU / Linux or FreeBSD or farm of servers was also high and there was little documentation than today and a lot of custom tweaks (scripts) to develop to make things working and system administration job was way more custom than today.
In other words the sys admin was a digital artist just like the UI / Web designer or the common programmer (who was way more advanced and hack, thought oriented) than todays "coders" most of which knows no damn thing but are a great Human Robots serving the functionos of ("Google Search for some ind of Programming language code" then "Copy" and "Paste" into a buggy module / script / application function) and then of course as a result you have a large clumsy (softwares) programs which eat a lot of Server resources (often crash – that's especially true for Java based applications) in the background and get respawned (which does severely load the servers CPUs / Memory) but as the end user is not aware of that it is considered a job finely done.

computers-kills-people-silence-means-security


2. The IT Computing and SysAdmin / Programmer Jobs offered today

In other words nowadays computing is becoming a mess, just like a system is complicating it becomes more prone to failures, the same happens with modern informatics. The chaos of programming languages code and concepts (especially), the abstracts makes a programming code harder to debug than in the past (of course that depends on the programmer too), but as most programmers are totally lame and doesn't understand even basic Hardware / Electronics concepts but are more of a Code Monkeys (yes I can say today's programmers are not really a programmers but a CODE MONKEYS !).

The result for the avarage sys admin is that the developed software are less and less custom but written in a way (to just run it on a server) and usually the sys admin ends up with less and less options for modification or debug problems of the software. As the tendency of installable services / programs (I am talking about the proprietary ones) are becoming more and more monolithic of nature.
As a consequence that starts making the classical system administration as most of todays softwares can be installed even by a highly trained monkeys (no real sysadm needed) and even if you work as a sysadmin it is very likely you are not involved in interesting job but doing more and more routine and burecracy work (which is hell at least for me – as one of my primary motivators to start a career in the IT field and specifically in the field of System Administration that back in the day the system administrator used to be a more important person for a company as a whole company infrastructure depended on the work of that single Super Man that made possible the Internet Accessibility for office users, made possible Linux / Windows servers to operate fine with a bunch of websites and some crazy softwares and platforms, and even took a periodic maintenance of an Office Workers PCs, not to mention the responsibilities to do the frequent data backups, do a support functions (talk heavily on the phone with customers with issues etc.) and help programmers set-up their crazy testing environments (developed project code) on a testing servers etc.

It was the golden age of system administration … and perhaps a golden age for the ones involved in the field of Computing .. really …

3. What if you end up to be a Jobless System Administrator today? What does current sys admin Job Market Place look like?


Have you listened to Venom (black metal band) song – Welcome to Hell?  … its like that ..

Yes, that's the worst nightmare for most of us sys admins , becoming jobless due to company bankruptcy, dismissal or just a desire for a rest for some time from the over active job to talk over the phone with uneasy and angry customers.
Al this put you you in a very harsh situation, because the Classical System Administartor jobs from the past such as building a Strong Company Firewall with IPTABLES or BSD PF is nowdays done by some pre-purchased router such as:

McAffee, Palo Alto, JuniperSRX 2020, Next Generation (firewall as a service such as Cato Networks), Kaspersky, Fortinet, (if you're lucky pfSense), Comodo Internet Security, Zone Alarm (the possible list of sh*t goes on and on …)

In other words businesses nowadays, prefer to buy a ready solution and most of this solutions even though being configurable, often have a weird interfaces and force the user to use a ready set of firewall rules (policies) rather than building ones from scratch … and most of the softwares can be configured by a normal non sysadmin anyways so mostly or soon the sysadm is not needed.

devops-diagram-explained-512px-Devops-toolchain.svg

If in the past you have build things from source or deployed / configured things server by server and each of your servers as a consequence had its kind of own spirit, because of the many custom things placed on it, the current situation with sysadmin job are mass deployments of pre-bundled packages (DevOpsDevelopment Operations – another crazy business non-sense buzzword that describes server scripting automation development) as a DevOps (SysAdmin) which is some kind of Hybdir between a programmer / scripter / db developer / and scripter you have to be eloquent or at least have some basic knowledge in mass deploy tools such as Docker, Ansible, Chef, Puppet, TeamCity, Bamboo, Fabric, Etc.
and to add even more hell to the hell, in most System Administration jobs you perhaps won't manage your own company data even but you will have to deal with third party vendors such as AWS Amazon or store the company important data in some external Cloud Storage service (except if you don't have the option to choose for a custom Own Cloud solution)

But often this is not enough you have to be more or less aware or have some experience with some SRE (Site Reliability Engineering)

But wait, that's not enough you need to be also a good Team Player communicate to a good number of often lame burecrats / lame progammers / a manager over your head that usually does not know shit about technology / a project manager / some Database guys that oten have a very questionable knowledge in Database programming maintenance .. etc. …  and the worst (in my humbe opinion) is that you have to spend 2, 3 as a mimumum daily in a non-sense meetings over proprietary non-free software program such as Skype For Business or Web Room meeting online such as WebEx with people that have little to know idea about technology or are presenting professionals but have a very questionable amount of knowledge in their field …

To summarize modern SysAdmin jobs, just like all other jobs are slavery but with the difference that in most common daily jobs most people have more freedom and are less dependent for their daily work, than you end up as a New Age of Computing Sys Admin.

system-administrator-stress-October-Poll-Sysadmin-Results-stress

Oh yeah and lets not forget the high amounts of STRESS you get daily as a sysadmin that for some is almost 24/7 especially for people who manage a large networks or server infrastructures. Suppose you migrate a Web services, database service, mail server, DNS record etc. and you make a minor mistake so the users can't access the service, guess who will be fired first ?! YOU !!! Even if you don't get hired, you'll be murmored and send for some kind of meaningless training just because you did a mistake (which is very normal, as every human daily days tons of mistakes) …

Another thing is if you're truely dedicated to system administration profession and you spend hours reading and learning new technologies (which in the field of system administration is inevitable) or just doing work from home as a freelancer to get some extra bucks and you don't have to actively sport (Running, Biking, Fitness, Mountaun Riding, whatver …), your Spinal problems and Herniated Discs (Neck or Waist) is to soon knock your door
and stay with you until your death bed.

 

But that's not all of the hurdles, many of the System Administrator like jobs of today require you to have an overview knowledge on Virtualization technologies such as VMWare ESX, VServer … and have a good idea about VPS management and even some employeers require a knowledge in Astrerisk IP PBX (Open Source Communiation Software) or other IP Telelphony software strangie …

Dear sysadmin collegues, my opinion is this kind of requirements are a little bit higher and almost impossible to match (or there are none to any living flesh) that attains all this knowledge or they will ever be.

… But even if you get employeed (and you tricked the HR interviee that you own the SuperMan + Batman + Robocop + You name your favourite movie superhero superpowers and went through the other interview (hell) circles) … finally you get hired and you end up often part of projects that are already seriously messed up from the start or developed in a way that even if succeed in a short term, guarantees a long term failure.

Oh the hirement process is also a lot of enjoyment for the burecracy freak, you have to fill in a number of documents, describing tons of information, provide tons of documents, certifications, talk a number of times on the phones with inadequate Human Resource representative (usually highly brainwashed ppl) "specialist" that knows shit about technology … Then you have to go to a few more selections, interviews further with a technical guy, fill in tests online (maybe not always) and finally talk to a company manager.

All above screening and selection I'm desribing of-course is featuring large corporations (which are among the little) that offers some decent sallaries like 1500 – 1800 EUR (for Eastern Europe) or 3000 – 3500 for rest of Western Europe (if you're a lucky American citizen you might earn up to 10 000 – 11 000 $).
The advantages of the large corporation besides the so-so sallary is the sense of security (that you want be jobless just next year or two from your day 1 in the company).

You can always become a sysadmin in a start-up company but finding such is also nowadays a real pain in the ass and even if you have a 12000+ unique a day visits site such as mine and you offer your sys admin skills for really cheap , you still will have troubles in finding clients / employeer for whom you can practice your skills and make a living as a SysAdmin.

That's pretty weird for me especially with the fact that everyone is tubing that more and more IT specialists are required ..

Anyways assuming you have the "luck" to get hired in a large corporation such as IBM you will have to do a very tedious job, such as either Backup with (IBM Data Protect), Veritas Backup, Barracuda Backup, HP Data Protector or similar software, only do build or deploy new servers, web services, databases or whatever else. E.g. your type of work is likely to be monotonоus and boring and will offer you not much than learning a little bit more about the technology you're already acquainted to ..

Moreover, because in modern IT, human freedom is not really respected … you either comply to the company brainwashing strategies a bulk shit procedures or you get fired, you either become a small wheel in the failing machine (here i mean most large companies you might end up hired nowdays reached its peak state are into a decline) and a logical result is living in constant fear that they might fire you end you might end up jobless or you stand up for what you're in the company and be careless about political correctnes and you quickly get inconvenient, politically incorrect (oh yes I forgot to mention this other craziness if you happen to be employeed you have to be politically correct) and do periodically a stupid exhausting Trainings (I prefer to call them a brainwashing session as most of the trainings are not teaching you anything but just wash your head to comply to shit). But if that Hell is not enough in the large corporation in order to look "normal" you have to partcipate in the Non-Sense Teambuildings, with team mates you have little to know affection (with the very same people you spend 5 days a week, now you have spend 1 /2 more day. every month or so …

long-term-ago-people-who-sacrifice-their-time-sleep-family-food-laughter-were-called-saints-now-they-are-called-it-professionals

So welcome to modern HELL OF system administration, or better to say welcome to the Cult of the large corporation businesses.

4. What are your options if you end up as a poor old school sys admin on the job market?

If you have a long history as a sys admin and computers become too boring for you like my case, you can always think about migrating to a Management position in the field of IT (this in most cases means doing nothing all day long pretending that you understand management and talking shit (laughing in a group), being present in a crazy management meeting whose essense is a shit talk all day long … with a bunch of people who facebook / youtube all day long talk about Latest Cars models and how they wish to have a half million car, watch and talk about fuzzy hand clocks, cheeks, plan their next vacation or where to have the lunch and housing (apartments) all day long (in some more extravagant cases you have some guys being wacky talking about drugs, sex and  rock-and-roll.)
but the unpleasent surprise here is even as a Manager you will probably have to start working for a corporation and have the same depressing atmosphere of people standing in front of their computers (tailor like) all their long with the only difference you will have to speak more with a number of computer addict zoombies (left without much options) that are doing some monkey programming / coding or Services job day after …

Other option you have is to move out of the virtual business at all and get into a real works industry such as getting a Construction job (but believe me such transitions, though I heard of are too painful) and sooner or later you will get back to computing virtual business ..

I have a friend Jose Mathew, whose exit poll from the IT business was to graduate a 2 years post-university course to become a professional Chef (cook) in restaurant but after already few years employeed as a Cook, he is again considering getting back into the IT and paradoxically he wants to enter the niche of Network Administrator (which I forget to mention earlier in that article).

The Network Administrators are among lucky System Administartors job profiles because there job is depending nowadays mostly on their CCNA / CCNP certificate, there experience with network routers such as Juniper, LinkSys, Cisco, Avaya etc.  But the big problem with being one of the guys is that the employment jobs offered are much less than the general Senior or Junior System Administrator (that is more free software Linux based).

The most luckly ones are the Windows System Administrators as the amount of such that are looked up on the market at the moment of writting this article is relatively high. The type of job for Win Sys Admin offered on the market as long as I researched is for Windows Sys Admins that have a good amount of experience / knowledge (with Active Directory) domain controller.

There might be some enjoyment for the Win SysAdmin if you have to develop your own PowerShell scripts or do some kind of automations on a domain controller level and from that perspective this job positions are attractive, but unfortunately that comes at the price for being a totally Microsoft software dependent (junkie).
But in overall it is much easier for the ordinary Win Sys Admin than the Unix one because of the reason Windows Servers and related scripting automation solutions is generally much easier to learn and many of the things you have to make up yourself on a common *NIX OS are already available in Windows in the form of some proprietary extra software you have to buy …
However for people as me who are involved in the UNIX world for the last 15 years, it want be easy to migrate to Windows System Administartor.

In my previous employment Job in Hewlett Packard (and later DXC) I have to do a lot of Windows System Administration jobs and I have to says, that was too easy in general but the downside of deploying some third party software on Windows in case of failure is the debugging on Windows is generally harder task than on Linux / BSD..

Another option if you want to move from the field of System Administarton is to start your own company in either Sys Admin or Programming field or Website building, Website hosting.
That's easy especially if you have a good amount of experience but the problem with this is you need a partner and often finding a partner is a tedious job …
Plus most of the clients you can get for your business are already clients of the Large Sharks corporations and at best you or your company might have to work as a contractor for the uncle SAM corporations ..

Of course as a sysadmin you can always repair computers and could try to start a business of computer (OS) repair niche, but as the competition in the field is enormous and you will have to work like crazy to be able to make a decent living, plus it is very likely that you bankrupt, because of lack of enough clients in need to fix their OS (as most people nowadays have learned on how to install Windows and basic surrounding softwares) …

 



system-administration-is-dying-grave-RIP-sysadmins

 


If you have land like my parents you can try to make a living by growing vegetables like Bio potatoes, cucumbers, tomatoes, cabbage, onions, garlic and other fruits such as Apples, Pears, Walnuts, Peaches etc.
The bio-fruits growing business though profitable in western societies is way from profitable in Eastern world so if you happen to be in some eastern country and you want to make good moving to the fruit growing / selling business might not make you rich but at least you will have benefits for your health because of the village / land work + you will have a little bit more independence and your mind will be much clearer. If you decide to try a physical work like this, your concentration level will improve as most IT industry people because of the long hours of computer madness jobs slowly start totally loose focus and often the stress of the Computer works impare memory ..

 

 

Another option for exit from System Administration industry if you have some little marketing experience or background is to move to become a Marketing or E-Marketing SEO specialist, that's not a bad option but the problem is still you will bundled in a permament marriage with the computer and the sallary you will get would most likely no different from the one you will get as a system administrator.
So just like any other Computer related job in order to keep in shape you either have to go Fitness 2 / 3 times a week or actively sport something, otherwise you might experience a growing decline in health over time (just like you already might have in sys admin field).

To sum up being a sysadmin is very enjoyable fun and bright profession, the only small problem is most true dedicated system administrators are know tend to suffer constant anxiety, hyper activity, have physical health issues, suffer forms of depressions or have mental issues (perhaps because of the inhuman amount of information they have to process daily and the large amounts of hard alcohol vodka, beer etc. 🙂 consumed as a mean of anti-depressant) …
But it seems other IT specialists I know such as programmers tend to often suffer similar problems. Besides that many of the people involved in sysadmin business or IT have troubles finding decent woman to marry, as they tend to become more or less anti-social (or gradually loose their ability for proper interactivion with human) because of the fact most of their life is being led in the virtual reality online.

But lets be optimistic, perhaps there are many sysadmins who have the luck to have started a normal life in a normal company and managed their life well with family and kids it is just I haven't met them yet 🙂

I know this post was quite a lot of rant and I would like to excuse anyone who was bored to read all this mess, but I felt obliged to share about this problem as the things are rushing through my mind for over a two years now and we had quite a discussions with friends / collegues on the realization that the system administration job is loosing its attractivity and that the new age of (cloud) computing is pushing computer science to move towards a bad and dark path which makes the individual both employee and user more dependant and less free  …

MySQL crashes after upgrade from MySQL to MariaDB and how to fix it

Tuesday, August 21st, 2018

how-to-fix-crashing-mysql-after-upgrade-to-mariadb-database-mariadb-logo.png

If you have recently upgraded your Debian / Ubuntu / CentOS Linux Server to the latest RPM / DEB packages as part of the upgrade you might have noticed the upgrade of MySQL Community Server  (which was bought by Oracle Corporation few years ago) is automatically upgraded to MariaDB (which is a MySQL fork made by the original developers of MySQL and guaranteed to stay open source. Just to name some of the Notable users include Wikipedia, WordPress.com and Google.).

You might have noticed MariaDB's restart script which is still under /etc/init.d/mysql  won't start and a quick check in /var/log/mysql.err | /var/log/mysql.log
shows errors of /usr/bin/mysqld crashing with errors like:

140502 14:13:05 [Note] Plugin 'FEDERATED' is disabled.
InnoDB: Log scan progressed past the checkpoint lsn 108 1057948207
140502 14:13:06  InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files…
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer…
InnoDB: Doing recovery: scanned up to log sequence number 108 1058059648
InnoDB: 1 transaction(s) which must be rolled back or cleaned up
InnoDB: in total 15 row operations to undo
InnoDB: Trx id counter is 0 562485504
140502 14:13:06  InnoDB: Starting an apply batch of log records to the database…
InnoDB: Progress in percents: 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99
InnoDB: Apply batch completed
InnoDB: Starting in background the rollback of uncommitted transactions
140502 14:13:06  InnoDB: Rolling back trx with id 0 562485192, 15 rows to undo
140502 14:13:06  InnoDB: Started; log sequence number 108 1058059648
140502 14:13:06  InnoDB: Assertion failure in thread 1873206128 in file ../../../storage/innobase/fsp/fsp0fsp.c line 1593
InnoDB: Failing assertion: frag_n_used > 0
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.1/en/forcing-recovery.html
InnoDB: about forcing recovery.
140502 14:13:06 – mysqld got signal 6 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=16777216
read_buffer_size=131072
max_used_connections=0
max_threads=151
threads_connected=0
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 345919 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong…
stack_bottom = (nil) thread_stack 0x30000
140502 14:13:06 [Note] Event Scheduler: Loaded 0 events
140502 14:13:06 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.1.41-3ubuntu12.10'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  (Ubuntu)
/usr/sbin/mysqld(my_print_stacktrace+0x2d) [0xb7579cbd]
/usr/sbin/mysqld(handle_segfault+0x494) [0xb7245854]
[0xb6fc0400]
/lib/tls/i686/cmov/libc.so.6(abort+0x182) [0xb6cc5a82]
/usr/sbin/mysqld(+0x4867e9) [0xb74647e9]
/usr/sbin/mysqld(btr_page_free_low+0x122) [0xb74f1622]
/usr/sbin/mysqld(btr_compress+0x684) [0xb74f4ca4]
/usr/sbin/mysqld(btr_cur_compress_if_useful+0xe7) [0xb74284e7]
/usr/sbin/mysqld(btr_cur_pessimistic_delete+0x332) [0xb7429e72]
/usr/sbin/mysqld(btr_node_ptr_delete+0x82) [0xb74f4012]
/usr/sbin/mysqld(btr_discard_page+0x175) [0xb74f41e5]
/usr/sbin/mysqld(btr_cur_pessimistic_delete+0x3e8) [0xb7429f28]
/usr/sbin/mysqld(+0x526197) [0xb7504197]
/usr/sbin/mysqld(row_undo_ins+0x1b1) [0xb7504771]
/usr/sbin/mysqld(row_undo_step+0x25f) [0xb74c210f]
/usr/sbin/mysqld(que_run_threads+0x58a) [0xb74a31da]

/usr/sbin/mysqld(trx_rollback_or_clean_all_without_sess+0x3e3) [0xb74ded43]
/lib/tls/i686/cmov/libpthread.so.0(+0x596e) [0xb6f9f96e]
/lib/tls/i686/cmov/libc.so.6(clone+0x5e) [0xb6d65a4e]
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.

Any recommendations?
mysql

I hoped to solve the /usr/bin/mysqld segfault error with server reboot as I though the problem is caused by the fact libc library was updated, but even a reboot did not solve it.

I've investigated online for a solution and found following MySQL corruption and recovery article.

The solution outlined there is very simple and comes to adding the line:
 

innodb_force_recovery = 1


to /etc/mysql/my.cnf

Assuming the mysql server is not running before restarting mariadb server.

1. Make a backup (Dump) of all MySQL tables

mysql:~# mysqldump -A > dump.sql

2. Drop all databases which need recovery.
You can do that from mysql cli or phpmyadmin

3. Stop mysqld.

mysql:~# /etc/init.d/mysql restart

4.  Remove /var/lib/mysql/ib*

mysql:~# rm -rf /var/lib/mysql/ib*

5. Comment out innodb_force_recovery in /etc/mysql/my.cnf

6. Restart mysqld. Look at mysql error log.
If everything is fine and you have problems with broken or missing databases the best thing next is to stop again mariadb and

7. Restore databases from the dump

mysql:~# mysql < dump.sql

 

 

 

Virtual Keyboard for Linux and other Freedom respecting operating Systems

Monday, July 30th, 2018

How to install and Use Linux Virtual Keyboard and other freedom respecting Operating Systems

  •  Looking for a quick way to use VIRTUAL KEYBOARD ON LINUX COMPUTER OPERATING SYSTEM, you can do it just this 1 task in 3 simple steps  ???
    – Logical question emerges, WHY ??? would you need a virtual keyboard on Free Software OS such as Linux?
    Well, just because sometimes it is much more secure to use a Virtual Keyboard, especially if you have doubt that your keyboard has been tapped or a Key Logger (Sniffer), intercepting the Keyboard IN / OUT jacks, is installed on the computer or you might have sit on a computer of ,a friend running Linux, and you want to make sure he did not install sniffer to intercept your ,SSH login passwords and ,later hack into your Servers, after stealing, the password

 

  • Assuming you're on : – Debian / Ubuntu Linux, or other of the numerous IT systems such as ,FreeBSD / OpeBSD etc. out there, you can run simply this commands:

     

  •  apt-get install –yes florence
    * A. To make it, easily invokable for laters, create a small bash, shell script in directory; – location /usr/bin/virtual-keyboard like, the one below:

    vim /usr/bin/virtual-keyboard

    * B.. INside the file Place following 1 liner code
     

    #!/bin/sh
    /usr/bin/florence

     

    * C… To later invoke it any time:
    Press ALT + F2 (or use Run Command Dialog in GNOME / KDE / Windomaker / IceWM whatever or any other crazy graphic environment of your choice and run:

    /usr/bin/virtual-keyboard

 

Saint King Boris feast and a Holy Liturgy in the Great Basilica of Pliska – 2nd of May 2018

Thursday, May 3rd, 2018

Pliska-Great-Basilica-air-view-biggest-medieval-church-Europe

Pliska is the first capital of Bulgaira and an ancient city that traces its roots back to the Baptizmal of Bulgaria by Saint King Boris (The Baptizer of Bulgarian lands and Slavonic People) the feast is of a great importance for Bulgaria to be pressent on the map of Europe over the last XV centuries since Bulgaria's establishment in 681 A.D. 

 

Pliska which is an enormous architectural complex dated from IX / X century hsa played a pivotal role for the Baptizmal process of Bulgarian nation as it was King Boris's palace at that time and the Christianization of Bulgarians and Slavonic (natioons) people started exactly in the newly built so called Great Basilica of Pliska  which was an archibishop seat and a central Monastery and Palace Church of Bulgaria, the building was completed approximately in 875 A.D. and was the biggest Cathedral church in Europe for about 1000 years onwards with the gigantic area of 2,920 square meters (31, 400 ft).

Old_Basilica_in_Pliska-walls-reconstruction

20 century reconsturction of the Church walls based on descriptions from history annals 

The Basillica was build on top of ruins of older Bulgar pagan temple and has martyrium (a place dedicated to a martyr who has been martyred there) as many of the ancient Churches had. It is believed that the martyrium may be Saint (Boyan) Enravota – the first Bulgarian saint (circa 833). The basillica has a form of a Cross shaped mausuleum.

 

Old_Basilica_in_Pliska-left-part-of-the-alter

Reconstruction of walls left part of the Altar


The archibishop residence was north and south of the basillica on the northern yard there was a residential building with a bath which included hypocaust (an ancient water warming system). Building south of the cathedral was used as school and scriptorium. Nearby the basillica are situated necropolis (most likely monk graves).

Old_Basilica_in_Pliska_Sarcophagus-near-altar

Pliska Sarcophagus near the Great Basillica Church altar

As with every basilica of that time there is a special allocated place of the Church for nobles. Yard northside was used as a kitchen and a dining room and accomodated other monsatic buildings.


The Old Basilica reconstruciton original model should have looked something similar to below picture which is found in the Museum (which holds a little part of the artefacts found) situated about 1.5 kilometer from the Basilica.

Old_Basilica_in_Pliska_Reconstruction-model

A presumable reconstruction model of the Old Basilica

On 2nd of May KIng Boris Feast in 2015 for a first time since many centuries in memoriam of 1150 years since Christianization of Bulgaria in the Great Basilica was served an Eastern Orthodox Christian (an open air) Holy Liturgy headed by the Bulgarian Patriarch Neofit with many Metropolitans guests from other Eastern Orthodox national Churches where the ex-president Rosen Plevneliev and the Living ancestor King Simeon the III-rd was present.

1150-years-baptismal-of-bulgaria-feast-patriarch-Neofit-Pliska-Great-Basilica

Patriarch Neofit Patriarch and head of Bulgarian Orthodox Church blessing in Great Basilica y. – 2018

The event in 2015 was continued with the idea to become an yearly event on the feast of Saint King Boris each 2nd of May yesterday in 2018 by the Metropolitan of Varna and Veliko Preslav his eminence Metropoltian Ioan (John).

metropolitan-holy-liturgy-Preslav-2018

Orthodox Christian Believers in the Great Basilica Preslav 2018 (People gathered being transported by free of charge buses organized by the Eparchy from Dobrich, Varna, Shumen, Targovishte 

holy-liturgy-2018-Preslav-1-fr-Peter-reading-Apostle

Fr. Peter reading The Apostle readings

his-eminence-Metropolitan-of-Varna-and-Veliki-Preslav_Ioan

Metropolitan Ioan (John) of Varna and Veliki Preslav Blessing in Holy Liturgy 2018

Holy-Sacraments-in_Great-Basilica-2018-metropolitan-Ioan-and-his-deacon

The Holy Altar with the Holy Sacraments in Pliska Basilica in 2018 (Metropolitan Ioan and his deacon)

The Church service was a big gathering for the region (if we take the fact that the current town Pliska has about 900 citizens and is in practice a village), the overall present people were perhaps about 200 to 250. More than 1/3 of the priests (38 priests) from the whole Eparchy served the service together with the Bishop. The weather before the service was rainy but just after the service and onwards, it came sunny again just like it was saint King Boris with all the Christian saints rejoiced to see so many Christians gather together just like the times when Christianity started to spread from this same place over the lands of Bulgaria country and later during the golden age throughout the territories of Bulgarian Empire and all the Slavonic borderlines such as nowadays Romania, Serbia, Ukraine and Russia. 

The whole set of pictures from saint King Boris feast in Pliska can be seen on the official  website of Varna and Veliko Preslav eparchy here

Ancient Orthodox Christian Monastery Patleina and Great Preslav the First Bulgarian Kingdom Capital city near Shumen

Wednesday, February 14th, 2018

Great-Preslav-the-capital-of-the-golden-age

Recently I had the great oportunity to Visit another two great History Landmarks of my homeland Bulgaria the monastery Patleina (Saint Pantaleon (known in Slavonic tradition as Panteleimon) – in memoriam of the great healer Christian Saint) where according to Orthodox Church tradition and archeologists is the historical monastery where Saint (Tzar) King Boris the Baptizer of Bulgaria has concluded his early life as humble monk praying for the heavenly blessing and prosperity of his young Christinized Country Bulgaria.

Saint Panteleimon is famous in west Roman-Catholic Christian tradition to be one of the Fourteen Holy Helpers a patron of doctors and herbalists especially famous during the middle ages 14-19 century summoned by people as a shield against various diseases and asked for help for untreatable sicknesses.

Besides the famous monastery Patleina region is a natural reserve 40,62 hectars in North-East Bulgaria, after 893 just few years after the Christianization of Bulgaria in 865 by King Boris the region is famous for being a host of a world famous ancient Presval Literary School near the region are found the oldest known cyriic datable inscriptionsJust to name a few of the famous Preslav Literary School authors that is Saint Naum of Preslav, Bishop Constantine of Preslav (Author of Alphabet Prayer), Historics and various other ancient Bulgarian spiritual literature, John the Exarch, Tchernorizets Hrabar etc.

Notable to see near Patleina monastery are Orthodox Christian icon craftman workshops.

Ceramic_icon-workshops-Patleina-region

Ceramic Icon Workshops near Patleina region (picture source Wikipedia)

Patleina-monastery-signature
 

Just 6 km away from Patleina is situated another Great Historical Landmark, the first Bulgarian capital Great Preslav (Veliki Preslav).
 

9th-century-monastery-Patleina-remains-Bulgaria

9th century Monastery Patleina Church wall remains – picture source Pravoslavieto.com

Patlejna-9th-century-King-Boris-Monastery

Patlejna (Also known as Saint King Boris Monastery) IX century entrance Church wall remains
 

Patleina-monastery-alter-view-near-Shumen-Bulgaria

Patleina IX century monastery Church alter view
 

St._Theodor-ceraminc-icon-Veliki-Preslav

Veliki Preslav the famous Ceramic Icon found during archeological excavations in Patleina Monastery (2 km away from Veliki Preslav)
The icon style reminds of Byzantine art of 10 century, the icon image looks very much alike saint Theodor Stratilates though it not 100% proven whose image does the icon depicts – It is interesting fact that Patleinas craftman workshops are believed to be the biggest ceramic workshops on Balkans in 10th century consisting of a separate ceramic plates – picture source Wikipedia

Besides its  ancient monastery and archeological excavations Patleina region is famous for being a reserve keeping one rare type of tree Carob Tree (or Ceratonia siliqua).

Just about 300 meters away from the ancient monastery Patleina there is functioning monastery that was recently restored bought back by Bulgarian Orthodox Church from the Government, the monastery building started in the 1930s (in which Bulgaria was in a serious spiritual and economical boom, however the building completed right before the completion of the Second World War before 9.09.1944 the date at which the Third Bulgarian Kingdom finished its existence and the communist came to power.

Monastery-Saint-Panteleimon-near-Patleina-region-Shumen

Saint Panteleimon Monastery Patleina region – (notice the unusual architecture, the whole monastery is one big rounded monastic wall

Saint_Panteleimon-Monastery-Patleina-region-monastic-walls

Saint Panteleimon Monastery near Veliki Preslav (picture source pravoslavieto.org)

The monastery saint Panteleimon was planned as in impressive building as in the 30's there was a hope for a restoration of Patleina as a spiritual center that will elevate itself to the importance it once had in the 9th century for the raise of Bulgarian Empire, the building is 4 floor and has the shocking number of 120 rooms. 
The fresh air and nature all surrounding it makes it a great place for rest for people who are overwhelmed by their daily stressful life and as far as I understood there is possibility for renting some of the rooms for a couple of days for pilgrims or even tourists.

Currently the whole enormous monastery is inhabited by only one monk Hieromonk Father Constantne who is also appointed as an Abbot of the Monastery and is in process of doing his best for restoration of the monastery and building a monastic brotherhood.

Hieromonk_Konstantin-Patleina-monastery-near-Shumen

Hieromonk Konstantin (the only abbot and monk of saint Panteleimon monastery near Shumen)

As it is visible from the picture (the boxes under the icon of the Virgin Mary) the monastery has holy relics of Saint Panteleimon and relics of few other saints. What is interesting about father Konstantin is he used to be working in the Bulgarian Atomic Central (AEC Kozlodoi) scientific work before he converted to become a fervent Christian. 

After father Konstantin understood God is a reality he decided to dedicate his life to Christ and he used to be a (novice – neophyte) in the Holy Mouth Athos monk republic in the Bulgarian Monastery saint George for 8 years!!!

It seems in this years father Konstantin realized he would be more helpful to become a monk in Bulgaria instead of Holy Mount Athos as here in Bulgaria, there is a lack of monks at the moment Bulgaria has according to what I heard a maximum of about 200 (monks and nuns) which is critically low.

Holy Liturgy Church service is being served regularly in the small monastic Church which is more looking like a Chapel and is on the first floor situated in the monastery building.

Father Constantine is famous for his hospitally just like most of the Holy Athos monks and right after the Sunday Service it is a tradition to feed all his guests, as a guests we were richly fed and we shared his lunch table just like Agape (the dinners of love of ancient Christian times).

To find out more about Saint Panteleimon Monastery check out the monastery official site here.

Very near to Patleina Monastery there are other great landmarks for those interested in archeology as Veliki Preslav (Great Preslav) the first Bulgarian Capital and Ancient Center of Orthodoxy who had a major influence on Russian Cyrillic and Orthodoxy and from where most likely many Christian icons and books were transferred later to Russia to help them raise their spirituality after Baptizmal of Russia is situated just few km away.

Fortress-Great-Preslav_entrance-walls

Entrance Walls fortress Great Preslav (picture source Izvora)

Fortress-Walls-Veliki_Preslav_walls_Enev

Fortress Walls Great Preslav – The First Bulgarian Capital (ruins from the IX-Xth century) – picture source Wikipedia

In the historical Veliki Preslav the most notable archeological remain is the so called Golden Church

Great-Preslav_Golden_Church_Klearchos_2

Great Preslav the famous Golden Church – the Church was partially restored to show the visitor how approximately it might have looked like in the X-XII century (The church was found during archeological excavations in 1927-1928 by Krystio Miatev and Jordan Gospodinov – according to  written sources from 907 by Tudor Doksov, the Church coincides with the Church built by Simeon I the Great who proclaimed himself an Emperor of Bulgaria 

Veliki-Preslav-The-Golden-Church-the-Round-Church

Preslav the Golden Church (also famous as the Round Church) – picture source fonstola.ru

The-Round-Church-Cross-on-a-stone

Cross on a Stone The Round Church – photograped by Angel Pavlov


Great-Preslav-Patleina-region-visit-view

Patleina and Great Preslav region view

Another Memorial Church worthy to visit in the region is situated on another fortress nearby to Great Preslav. The Memorial Church has the size of a Cathedral and is one of the biggest Cathedral Churches in Bulgaria.
 

Saint-Cyril-and-Methodius-Cathedral-Memorial-Church-Veliki-Preslav

Saint Cyril and Saint Methodius Memorial Church (In veneration of the creators of Glagolic Alphabet, later Bulgarian Cyrillic Alphabet)

St. St. Cyril and Methodius Church was planned as a memoriam of Saint Methodius (1000 years since his passing to Christ) planned to be build earlier in 1890s but delayed and later built in memorium of the Christianization of Bulgaria and 1000 years passed since the glorious Rule of King Simeon I, the Church building works completed thanks to Metropolitan Simeon in 1926 (the building works were delayed from 1897 'till 1918) A.D..

Very near the Church, one can see the ruins of another old Monastery dating again from the IX – X century, the construction is of a massive craved stone and was a masterpiece in the X century.

As an ex monastery the place can be considered holy as well and some people says it along with Veliki Preslov are a Gracious Energy filled places that can positively impact the human spirit.

Very near is situated also an Archeological Museum containing some of the artefacts found in Patleina, Veliki Preslav and the surrounding monasteries  – the museum is in the list of 100 Bulgarian National Museums and Landmarks.

Archeological-Museum-Great-Preslav-near-Patleina-and-Veliki-preslav-remains

Archeological Museum Great Preslav – picture source 100 Bulgarian National Landmarks

The museum has a very unique collection of artefacts and the entrance costs of 5 EUR is bearable, note that it's closed on Sundays.
The museum has an overall of 35 000 objects but unfortunately only 1700 artefacts are shown for the visitor, it is notable for holding one of the largest collection middle age Byzantine insignias, The Preslav Golden Treasure, bronze silver and ceramic artefacts.

The archeologists so far has determined that Veliki Preslav with all its Past Greatness and enormous territory, Churches and surrounding monasteries nearby has been planned as a rivalry Empire city to Constantinople and Preslav will definitely impress you if you're keen into archeology and history, hope you visit it soon, Enjoy! 🙂