file Archives - ☩ Walking in Light with Christ - Faith, Computing, Diary ☩ Walking in Light with Christ

Posts Tagged ‘file’

Install Zabbix Proxy configure and connect to Zabbix server on CentOS Linux

Thursday, May 4th, 2023

Install Zabbix Proxy configure and connect to Zabbix server on CentOS Linux

1. Why use Zabbix-Proxy hidden advantages of using Zabbix-Proxy ?

Proxy can be used for many purposes and can provide many hidden benefits, just to name few of them:

Offload Zabbix Server when monitoring thousands of devices
Monitor remote locations
Monitor locations having unreliable communications
Simplify maintenance of distributed monitoring
Improved Security (Zabbix server can be restricted to be connectable only by the set of connected Zabbix Proxy / Proxies

advantages-of-using-zabbix-proxy-instead-of-direct-connect-monitored-hosts-to-zabbix-server-diagram

A Zabbix proxy is the ideal solution if you have numerous hosts with multiple slow items that are affecting the performance of the server simply because processes are spending most of the time simply waiting for a response. A proxy can collect information from all hosts using its internal processes and then send raw historical data to the server. The time needed to connect and receive the host response will be on the proxy site, and the server performance will not be affected at all. A proxy just sends raw values to the server, and the server itself does not have to connect to the host to get the data.

2. Install zabbix-proxy-sqlite3 rpm package from Zabbix Official Repositories download page

Zabbix repository provides choice of 3 packages named as follows:

zabbix-proxy-mysql
zabbix-proxy-pgsql
zabbix-proxy-sqlite3

where the last value of the name (after zabbix-proxy) represents database type of the package — MySQL, PostgreSQL and SQLite respectively.

To not bother installing MySQL / PostgreSQL separate database servers, a lightweight choice is to use the sqlite3 db version.
As I prefer zabbix-proxy data to be stored inside a flat database, thus I choose to use zabbix-proxy-sqlite3.

[root@sysadminshelp:/root ]# yum info zabbix-proxy-sqlite3-5.0.31-1.el7.x86_64
Заредени плъгини: fastestmirror
Loading mirror speeds from cached hostfile
* base: mirrors.netix.net
* epel: fedora.ipacct.com
* extras: mirrors.netix.net
* remi: remi.mirror.karneval.cz
* remi-php74: remi.mirror.karneval.cz
* remi-safe: remi.mirror.karneval.cz
* updates: mirrors.netix.net
Инсталирани пакети
Име : zabbix-proxy-sqlite3
Архитект. : x86_64
Версия : 5.0.31
Издание : 1.el7
Обем : 4.4 M
Хранилище : installed
Обобщение : Zabbix proxy for SQLite3 database
URL : http://www.zabbix.com/
Лиценз : GPLv2+
Описание : Zabbix proxy with SQLite3 database support.

My experience to try to install thethe default CentOS RPM package for zabbix-proxy-sqlite3 provided by default
RPM package that came with CentOS did not work as expected and trying to install / configure and use it via

[root@sysadminshelp:/root ]# yum install zabbix-proxy-sqlite3.x86_64 -y

[root@sysadminshelp:/root ]# vi /etc/zabbix/zabbix_proxy.conf

Led me to a nasty errors seen in /var/log/zabbixsrv/zabbix_proxy.log like:

May 1st 2023, 08:42:45.020 zabbix_server cannot set list of PSK ciphersuites: file ssl_lib.c line 1314: error:1410D0B9:SSL routines:SSL_CTX_set_cipher_list:no cipher match
May 1st 2023, 08:42:45.018 zabbix_server cannot set list of PSK ciphersuites: file ssl_lib.c line 1314: error:1410D0B9:SSL routines:SSL_CTX_set_cipher_list:no cipher match
May 1st 2023, 08:42:45.013 zabbix_server cannot set list of PSK ciphersuites: file ssl_lib.c line 1314: error:1410D0B9:SSL routines:SSL_CTX_set_cipher_list:no cipher match
May 1st 2023, 08:42:45.013 zabbix_server cannot set list of PSK ciphersuites: file ssl_lib.c line 1314: error:1410D0B9:SSL routines:SSL_CTX_set_cipher_list:no cipher match
May 1st 2023, 08:42:45.011 zabbix_server cannot set list of PSK ciphersuites: file ssl_lib.c line 1314: error:1410D0B9:SSL routines:SSL_CTX_set_cipher_list:no cipher match

After some googling and reading some threads came upon this one https://support.zabbix.com/browse/ZBXNEXT-3604, there is exmplaed errors preventing the configured zabbix-proxy
to start are caused by the zabbix-proxy-sqlite3 package provided by Redhat (due to openssl incompitability bug or something ).

As one of people in the discussion pointed out the quickest workaround suggested is simply to use the official Zabbix Repository packages for zabbix-proxy-sqlite3, in order to not waste anymore time on this
trivial stuff to install it, simply run:

[root@sysadminshelp:/root ]# rpm -Uvh \
https://repo.zabbix.com/zabbix/5.0/rhel/7/x86_64/zabbix-proxy-sqlite3-5.0.31-1.el7.x86_64.rpm

Alternative way if you seem to not have the machine connected to the internet is simply download the package with wget / lynx / curl / w3m from another machine
that can reach the Internet upload the package via the local LAN or VPN and install it:

# wget https://repo.zabbix.com/zabbix/5.0/rhel/7/x86_64/zabbix-proxy-sqlite3-5.0.31-1.el7.x86_64.rpm …

[root@sysadminshelp:/root ]# rpm -ivh zabbix-proxy-sqlite3-5.0.31-1.el7.x86_64.rpm

NOTE ! Before you install proxy, keep in mind that your proxy version must match the Zabbix server version !

3. Generate a PSK random secret key and set proper permissions for zabbix-proxy directories

[root@sysadminshelp:/root ]# cd /etc/zabbix/

[root@sysadminshelp:/root ]# openssl rand -hex 32 >> /etc/zabbix/zabbix_proxy.psk
[root@sysadminshelp:/root ]# chown root:zabbix zabbix_proxy.psk [root@sysadminshelp:/root ]# vi /etc/zabbix/zabbix_proxy.conf [root@sysadminshelp:/root ]# mkdir -p /var/lib/zabbix-proxy/sqlite3db
[root@sysadminshelp:/root ]# chown -R zabbix:zabbix /var/lib/zabbix-proxy
[root@sysadminshelp:/var/lib/zabbixsrv/sqlite3db]# sqlite3 zabbix_proxy
SQLite version 3.7.17 2013-05-20 00:56:22
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> .databases
seq name file
— ————— ———————————————————-
0 main /var/lib/zabbixsrv/sqlite3db/zabbix_proxy
sqlite> [root@sysadminshelp:/root ]# vi /etc/zabbix_proxy.conf
#DBName=zabbix_proxy
DBName=/var/lib/zabbixsrv/sqlite3db/zabbix_proxy

4. Configure zabbix proxy to be able to connect to Zabbix Server

[root@sysadminshelp:/root ]# vi /etc/zabbix/zabbix_proxy.conf    ############ GENERAL PARAMETERS #################
   ProxyMode=0
   Server=192.168.1.28
   ServerPort=10051
   Hostname=zabbix-proxy
   ListenPort=10051
   SourceIP=10.168.1.55
   LogFile=/var/log/zabbix/zabbix_proxy.log
   LogFileSize=1
   DebugLevel=2
   PidFile=/var/run/zabbix/zabbix_proxy.pid
   DBName=/var/lib/zabbix-proxy/sqlite3db/zabbix_proxy.db
   DBUser=zabbix

   ######### PROXY SPECIFIC PARAMETERS #############
   ProxyOfflineBuffer=24
   HeartbeatFrequency=60
   ConfigFrequency=120

   ############ ADVANCED PARAMETERS ################
   StartPollersUnreachable=3
   StartHTTPPollers=3
   JavaGateway=127.0.0.1
   JavaGatewayPort=10052
   StartJavaPollers=5
   SNMPTrapperFile=/var/log/snmptrap/snmptrap.log
   StartSNMPTrapper=1
   CacheSize=32M
   Timeout=4
   ExternalScripts=/usr/lib/zabbix/externalscripts
   LogSlowQueries=3000

   ####### TLS-RELATED PARAMETERS #######
   TLSConnect=psk
   TLSAccept=psk
   TLSPSKIdentity=PSK zabbix-proxy-fqdn-hostname
   TLSPSKFile=/etc/zabbix/zabbix_proxy.psk

5. Check and make sure the installed zabbix proxy as well as the zabbix_proxy server zabbix_agentd client and zabbix_server are at the same major version release

a) Check zabbix proxy version

[root@sysadminshelp:/etc/zabbix]# zabbix_proxy -V
zabbix_proxy (Zabbix) 5.0.31
Revision f64a07aefca 30 January 2023, compilation time: Jan 30 2023 09:55:10

Copyright (C) 2023 Zabbix SIA
License GPLv2+: GNU GPL version 2 or later <https://www.gnu.org/licenses/>.
This is free software: you are free to change and redistribute it according to
the license. There is NO WARRANTY, to the extent permitted by law.

This product includes software developed by the OpenSSL Project
for use in the OpenSSL Toolkit (http://www.openssl.org/).

Compiled with OpenSSL 1.0.1e-fips 11 Feb 2013
Running with OpenSSL 1.0.1e-fips 11 Feb 2013
[root@sysadminshelp:/etc/zabbix]#

b) check zabbix_agentd version

[root@sysadminshelp:/etc/zabbix]# zabbix_agentd -V
zabbix_agentd (daemon) (Zabbix) 5.0.30
Revision 2c96c38fb4b 28 November 2022, compilation time: Nov 28 2022 11:27:43

Copyright (C) 2022 Zabbix SIA
License GPLv2+: GNU GPL version 2 or later <https://www.gnu.org/licenses/>.
This is free software: you are free to change and redistribute it according to
the license. There is NO WARRANTY, to the extent permitted by law.

This product includes software developed by the OpenSSL Project
for use in the OpenSSL Toolkit (http://www.openssl.org/).

Compiled with OpenSSL 1.0.1e-fips 11 Feb 2013
Running with OpenSSL 1.0.1e-fips 11 Feb 2013

c) Check zabbix server version

[root@zabbix:~]# zabbix_server -V
zabbix_server (Zabbix) 5.0.30
Revision 2c96c38fb4b 28 November 2022, compilation time: Nov 28 2022 09:19:03

Copyright (C) 2022 Zabbix SIA
License GPLv2+: GNU GPL version 2 or later <https://www.gnu.org/licenses/>.
This is free software: you are free to change and redistribute it according to
the license. There is NO WARRANTY, to the extent permitted by law.

This product includes software developed by the OpenSSL Project
for use in the OpenSSL Toolkit (http://www.openssl.org/).

Compiled with OpenSSL 1.1.1d 10 Sep 2019
Running with OpenSSL 1.1.1n 15 Mar 2022

6. Starting the zabbix-proxy for a first time

Before beginning with installation make sure selinux is disabled, as it might cause some issues with Zabbix

[root@sysadminshelp:/etc/zabbix]# sestatus
SELinux status: disabled

If you need to have the selinux enabled you will have to allow the zabbix-proxy into selinux as well:

# cd /tmp
# grep zabbix_proxy /var/log/audit/audit.log | grep denied | audit2allow -m zabbix_proxy > zabbix_proxy.te
# grep zabbix_proxy /var/log/audit/audit.log | grep denied | audit2allow -M zabbix_proxy
# semodule -i zabbix_proxy.pp

[root@sysadminshelp:/etc/zabbix]# systemctl start zabbix-proxy

Also lets enable zabbix-proxy to automatically start it on next server reboot / boot.

root@sysadminshelp:/etc/zabbix]# systemctl enable zabbix-proxy

Normally running zabbix-proxy should provide a status messages like:

[root@sysadminshelp:/etc/zabbix]# systemctl status zabbix-proxy
● zabbix-proxy.service – Zabbix Proxy
Loaded: loaded (/usr/lib/systemd/system/zabbix-proxy.service; disabled; vendor preset: disabled)
Active: active (running) since чт 2023-05-04 14:58:36 CEST; 2h 59min ago
Process: 8500 ExecStop=/bin/kill -SIGTERM $MAINPID (code=exited, status=0/SUCCESS)
Process: 8504 ExecStart=/usr/sbin/zabbix_proxy -c $CONFFILE (code=exited, status=0/SUCCESS)
Main PID: 8506 (zabbix_proxy)
CGroup: /system.slice/zabbix-proxy.service
├─8506 /usr/sbin/zabbix_proxy -c /etc/zabbix/zabbix_proxy.conf
├─8507 /usr/sbin/zabbix_proxy: configuration syncer [synced config 40521 bytes in 0.0…
├─8508 /usr/sbin/zabbix_proxy: trapper #1 [processed data in 0.000808 sec, waiting fo…
├─8509 /usr/sbin/zabbix_proxy: trapper #2 [processed data in 0.005028 sec, waiting fo…
├─8510 /usr/sbin/zabbix_proxy: trapper #3 [processed data in 0.001240 sec, waiting fo…
├─8511 /usr/sbin/zabbix_proxy: trapper #4 [processed data in 0.004378 sec, waiting fo…
├─8512 /usr/sbin/zabbix_proxy: trapper #5 [processed data in 0.004991 sec, waiting fo…
├─8513 /usr/sbin/zabbix_proxy: preprocessing manager #1 [queued 0, processed 3 values…
├─8514 /usr/sbin/zabbix_proxy: preprocessing worker #1 started
├─8515 /usr/sbin/zabbix_proxy: preprocessing worker #2 started
├─8516 /usr/sbin/zabbix_proxy: preprocessing worker #3 started
├─8517 /usr/sbin/zabbix_proxy: heartbeat sender [sending heartbeat message success in…
├─8518 /usr/sbin/zabbix_proxy: data sender [sent 0 values in 0.005241 sec, idle 1 sec…
├─8519 /usr/sbin/zabbix_proxy: housekeeper [deleted 4501 records in 0.011462 sec, idl…
├─8520 /usr/sbin/zabbix_proxy: http poller #1 [got 0 values in 0.000248 sec, idle 5 s…
├─8521 /usr/sbin/zabbix_proxy: http poller #2 [got 0 values in 0.000239 sec, idle 5 s…
├─8522 /usr/sbin/zabbix_proxy: http poller #3 [got 0 values in 0.000328 sec, idle 5 s…
├─8523 /usr/sbin/zabbix_proxy: discoverer #1 [processed 0 rules in 0.000261 sec, idle…
├─8524 /usr/sbin/zabbix_proxy: history syncer #1 [processed 0 values in 0.000009 sec,…
├─8525 /usr/sbin/zabbix_proxy: history syncer #2 [processed 0 values in 0.000007 sec,…
├─8526 /usr/sbin/zabbix_proxy: history syncer #3 [processed 0 values in 0.000014 sec,…
├─8527 /usr/sbin/zabbix_proxy: history syncer #4 [processed 0 values in 0.000021 sec,…
├─8528 /usr/sbin/zabbix_proxy: java poller #1 [got 0 values in 0.000017 sec, idle 5 s…
├─8529 /usr/sbin/zabbix_proxy: java poller #2 [got 0 values in 0.000019 sec, idle 5 s…
├─8530 /usr/sbin/zabbix_proxy: java poller #3 [got 0 values in 0.000019 sec, idle 5 s…
├─8531 /usr/sbin/zabbix_proxy: java poller #4 [got 0 values in 0.000018 sec, idle 5 s…
├─8532 /usr/sbin/zabbix_proxy: java poller #5 [got 0 values in 0.000013 sec, idle 5 s…
├─8533 /usr/sbin/zabbix_proxy: snmp trapper [processed data in 0.000026 sec, idle 1 s…
├─8534 /usr/sbin/zabbix_proxy: self-monitoring [processed data in 0.000034 sec, idle …
├─8535 /usr/sbin/zabbix_proxy: task manager [processed 0 task(s) in 0.000169 sec, idl…
├─8536 /usr/sbin/zabbix_proxy: poller #1 [got 0 values in 0.000012 sec, idle 5 sec]
├─8537 /usr/sbin/zabbix_proxy: poller #2 [got 0 values in 0.000021 sec, idle 5 sec]
├─8538 /usr/sbin/zabbix_proxy: poller #3 [got 0 values in 0.000039 sec, idle 5 sec]
├─8539 /usr/sbin/zabbix_proxy: poller #4 [got 0 values in 0.000024 sec, idle 5 sec]
├─8540 /usr/sbin/zabbix_proxy: poller #5 [got 0 values in 0.000019 sec, idle 5 sec]
├─8541 /usr/sbin/zabbix_proxy: unreachable poller #1 [got 0 values in 0.000011 sec, i…
├─8542 /usr/sbin/zabbix_proxy: unreachable poller #2 [got 0 values in 0.000018 sec, i…
├─8543 /usr/sbin/zabbix_proxy: unreachable poller #3 [got 0 values in 0.000041 sec, i…
└─8544 /usr/sbin/zabbix_proxy: icmp pinger #1 [got 0 values in 0.000022 sec, idle 5 s…

май 04 14:58:36 sysadminshelp systemd[1]: Stopped Zabbix Proxy.
май 04 14:58:36 sysadminshelp systemd[1]: Starting Zabbix Proxy…
май 04 14:58:36 sysadminshelp systemd[1]: Started Zabbix Proxy.

zabbix-server-zabbix-proxy-and-zabbix-clients-overview-diagram

7. Configure zabbix-agentd to use your just new brand new zabbix-proxy

Here is my sample configuration file:

[root@sysadminshelp:/etc/zabbix]# grep -v \# /etc/zabbix/zabbix_agentd.conf | sed '/^$/d'
PidFile=/var/run/zabbix/zabbix_agentd.pid
LogFile=/var/log/zabbix/zabbix_agentd.log
LogFileSize=0
Server=zabbix-proxy
ServerActive=zabbix-proxy:10051
ListenIP
Hostname=sysadminshelp
BufferSend=30
BufferSize=100
Include=/etc/zabbix/zabbix_agentd.d/*.conf

Note that the ServerActive given "zabbix-proxy" should be resolvable from the host, or even better you might want to put the IP of the Proxy if
you don't have at least a pseudo Hostname already configured inside /etc/hosts or actual DNS 'A' Active record configured inside a properly resolving
DNS server configured on the host via /etc/resolv.conf.

8. Create and Configure new proxy into the Zabbix-server host

Go to the zabbix server web interface URL into menus:

zabbix-administration-proxy-config

Administration -> Proxies (Proxy)

Click on ;

Create Proxy button (uppper right corner)

*Proxy name: usually-your-host-pingable-fqdn
Proxy mode: Active
Proxy address: 192.168.1.50
Description: pcfreak zabbix proxy

Administration -> Proxies -> Encryption

From "Connection to proxy"

Untick "No encryption"

and

Tick "PSK"

zabbix-administration-proxy-config-encryption

*PSK Identity: PSK proxy
*PSK: Put the key here (copy from /etc/zabbix/zabbix_proxy.psk generated steps earlier with openssl)

[root@sysadminshelp:/etc/zabbix]# cat zabbix_proxy.psk
faddbd96be00ac42c892fda5201634df25d51f3ndbbbf6cee9d354b2817092a28

Press the "Update" Button

zabbix-administration-proxy-config-encryption1

and go again to Proxies and check the zabbix-proxy is connected to the server and hosts configured to use the zabbix proxy reporting frequently.

To make sure that the configured new hosts to use the Zabbix Proxy instead of direct connection to Zabbix Server, go to Latest Data and check whether the configured Hostnames to connect to the Zabbix-Proxy continues to sent Data still.

9. Debugging problems with zabix-proxy and zabbix-agentd connectivity to proxy

In case of troubles check out what is going on inside the Zabbix Proxy / Agent and Server log files

[root@sysadminshelp:/etc/zabbix]# tail -n 50 /var/log/zabbix/zabbix_proxy.log

6832:20230504:134032.281 Starting Zabbix Proxy (active) [zabbix-proxy]. Zabbix 5.0.31 (revision f
64a07aefca).
6832:20230504:134032.281 **** Enabled features ****
6832:20230504:134032.281 SNMP monitoring: YES
6832:20230504:134032.281 IPMI monitoring: YES
6832:20230504:134032.281 Web monitoring: YES
6832:20230504:134032.281 VMware monitoring: YES
6832:20230504:134032.281 ODBC: YES
6832:20230504:134032.281 SSH support: YES
6832:20230504:134032.281 IPv6 support: YES
6832:20230504:134032.281 TLS support: YES
6832:20230504:134032.281 **************************
6832:20230504:134032.281 using configuration file: /etc/zabbix/zabbix_proxy.conf
6832:20230504:134032.291 current database version (mandatory/optional): 05000000/05000005
6832:20230504:134032.291 required mandatory version: 05000000
6832:20230504:134032.292 proxy #0 started [main process]
6833:20230504:134032.292 proxy #1 started [configuration syncer #1]
6833:20230504:134032.329 received configuration data from server at "192.168.1.28", datalen 40521
6834:20230504:134032.392 proxy #2 started [trapper #1]
6835:20230504:134032.401 proxy #3 started [trapper #2]
6836:20230504:134032.402 proxy #4 started [trapper #3]
6838:20230504:134032.405 proxy #6 started [trapper #5]
6837:20230504:134032.409 proxy #5 started [trapper #4]
6843:20230504:134032.409 proxy #11 started [heartbeat sender #1]
6845:20230504:134032.412 proxy #13 started [housekeeper #1]
6847:20230504:134032.412 proxy #15 started [discoverer #1]
8526:20230504:145836.512 proxy #20 started [history syncer #3]
8517:20230504:145836.512 proxy #11 started [heartbeat sender #1]
8530:20230504:145836.515 proxy #24 started [java poller #3]
8531:20230504:145836.517 proxy #25 started [java poller #4]
8532:20230504:145836.520 proxy #26 started [java poller #5]
8536:20230504:145836.522 proxy #30 started [poller #1]
8527:20230504:145836.525 proxy #21 started [history syncer #4]
8535:20230504:145836.525 proxy #29 started [task manager #1]
8533:20230504:145836.528 proxy #27 started [snmp trapper #1]
8539:20230504:145836.528 proxy #33 started [poller #4]
8538:20230504:145836.529 proxy #32 started [poller #3]
8534:20230504:145836.532 proxy #28 started [self-monitoring #1]
8544:20230504:145836.532 proxy #38 started [icmp pinger #1]
8543:20230504:145836.532 proxy #37 started [unreachable poller #3]
8542:20230504:145836.535 proxy #36 started [unreachable poller #2]
8541:20230504:145836.537 proxy #35 started [unreachable poller #1]
8540:20230504:145836.540 proxy #34 started [poller #5]
8507:20230504:150036.453 received configuration data from server at "192.168.1.28", datalen 40521
8507:20230504:150236.503 received configuration data from server at "192.168.1.28", datalen 40521
8507:20230504:150436.556 received configuration data from server at "192.168.1.28", datalen 40521
8507:20230504:150636.608 received configuration data from server at "192.168.1.28", datalen 40521
8507:20230504:150836.662 received configuration data from server at "192.168.1.28", datalen 40521
…

[root@sysadminshelp:/etc/zabbix]# tail -n 10 /var/log/zabbix-agent/zabbix_agentd.log
3096166:20230504:182840.461 agent #1 started [collector]
3096167:20230504:182840.462 agent #2 started [listener #1]
3096168:20230504:182840.463 agent #3 started [listener #2]
3096169:20230504:182840.464 agent #4 started [listener #3]
3096170:20230504:182840.464 agent #5 started [active checks #1]

If necessery to Debug further and track some strange errors, you might want to increase the DebugLevel to lets say DebugLevel=5

5 – extended debugging (produces even more information)

If checking both zabbix_agentd.log and zabbix_proxy.log cannot give you enough of a hint on what might be the issues you face with your userparameter scripts or missing Monitored data etc. and hopefully you have access to the zabbix-server machine, check out the zabbix server log as well

[root@zabbix:~]# tail -n 100 /var/log/zabbix/zabbix_server.log

3145027:20230504:182641.556 sending configuration data to proxy "zabbix-proxy" at "192.168.1.50", datalen 40521, bytes 6120 with compression ratio 6.6
3145029:20230504:182716.529 cannot send list of active checks to "192.168.1.30": host [pcfrxenweb] not found
3145028:20230504:182731.959 cannot send list of active checks to "192.168.1.30": host [pcfrxenweb] not found
3145029:20230504:182756.634 cannot send list of active checks to "192.168.1.30": host [pcfrxenweb] not found

Wrapping it up

In this article, we have learned how to install and configure a zabbix-proxy server and prepare a PSK encryption secret key for it.
We learned also how to connect this server to the central zabbix monitoring host machine in Active mode, so both Zabbix proxy and server can communicate in a secure crypted form,
as well as how to set zabbix_agentd clients to connect to the zabbix proxy
which will from itself send its data to the Central Zabbix server host as well as how to Debug and hopefully solve issues with communication between Zabbix client -> Zabbix Proxy -> Zabbix server.

I know this article, does not say anything revolutionary and there is plenty of posts online talking about how to run yourself a zabbix proxy and make in your home or corporate network,
but I thought to write it down as by writting it and reading a bit more on the topic of Zabbix Server / Proxy / Agent, that give myself a better overview on how this technologies work and such an article will give myself an easier step by step guide to follow,
in future when I have to configure Zabbix Environments for personal hobby or professionally for customers.
Hope you enjoyed. Cheers ! 🙂

Tags: cause, Click, configured, curl, download, file, history, Hope, howto, incompitability, Install Zabbix Proxy, lib, log, match, menus, messages, network, page, pcfreak, Press, psk, rpm, server version, task, unreachable, YES, zabbix-proxy
Posted in Linux, System Administration, Zabbix | No Comments »

Switching from PasswordSafe to Keepass database, migrating .psafe3 to .kdbx format howto

Thursday, February 23rd, 2023

I have been using PasswordSafe for many years within my job location as system administrator on the Windows computers I do use as dumb hosts to administrate remotely via ssh servers, develop code in bash / perl or just store different SysAdmin management tools and interfaces passwords. The reason behind was simply that I come out from a Linux background as I've used for daily Sysadmin job for many years GNU / Linux and there I always prefer GNOME (gnome GTK interface) in favour of KDE's (QT Library), and whence I came to work for the "Evil" Windows oriented world of corporations for the sake of Outlook use and Office 365 as well as Citrix accessibility i've become forced by the circumstances to use Windows.
Hence for a PasswordManager for Windows back in the years, I preferred the simplicity of interface of PasswordSafe instead of Keepass which always reminded me of the nasty KDE.
PasswordSafe is really cool and a handy program and it works well, but recetnly when I had to store many many passwords and easily navigate through each of it I realized, by observing colleagues, that KeePass as of time of writting this article is much more Powerful and easy to use, as I can see all records of a searched passwords on a Single screen, instead of scrolling like crazy with PasswordSafe through the passowrds.

I didn’t really feel like cutting and pasting every field for all my passwords (plus I started experiencing some PasswordSafe copy / paste passwords issues – maybe not related to PasswordSafe itself so this was the turning point I decided to migrate to Keepass.

For that, started looking at the import export functions for each program.

After a quick search, I found few articles online explaining on how the migration of PasswordSafe to KeePass can be easily handled as the versions of Keepass and Password safe are moving all the time, of course usually some of the guides to be found online are never competely upto date, so I had to slightly modify one of the articles and come up with this one 🙂 .

My PasswordSafe program that keeps my account password records and notes is version is
V 3.59 built on May 28 2022 and is running on my Windows 10 OS 64 bit release
The installed KeePass version to where I have migrated the Pwsafe password database Successfully is 2.48 64 Bit

Use the Password Safe function to export to XML file Format
(File -> Export To -> XML Format )
Import the text file into KeePass
(File->Import From-> Password Safe XML file)

This process worked quite fine. All of the passwords were imported .
Despite the importing (expected small glitches – please recheck that all was imported fine, before joy), the process is quicker than copy/pasting every field for each entry.

For those of you who are more worried about security than I am, you know this is a very insecure method to transfer passwords. For others, you may wish to export the (unencrypted) text file to a Veracrypt – that is a Truecypt fork (as nowadays obsolete unmaintaned and probably insecury) – a Free Open-Source On-The-Fly Disk Encryption Software to prepare Veracrypt partition and / or use Eraser on the text file once you’re finished with it or use another of the free Veracrypt open-source (free software) alternatives such DiskCryptor or even the proprietary Windows BitLocker / CipherShed / Axcrypt or some other encryption alternative software for Windows XP / 2000 / 7 10 that is out there.

NB! Please don’t do this on a public computer or a PC that you don't administrate.
You never know who might find your passwords or might be sniffing on your OS, as today there are so many devices that perhaps are hacked and listening and collecting password datas 🙂

That's it now I enjoy my KeePass but I'm thankful to PasswordSafe developers, who have easified my password management Virtual life for years 🙂
Any hints on how you migrated PasswordSafe to Keepass are mostly welcome. Also will be nice to hear of hard-core PasswordSafe hints or plugins that can power-up the password storage, maybe I can get convinced back to return back to PasswordSafe 🙂

Tags: didn, Evil Windows, export, file, fine, hosts, howto, imported, passwords, process, ssh servers, text, use, xml
Posted in Computer Security, Educational, Security, System Administration | No Comments »

How to dump and restore Zabbix database to do monitoring database snapshot, PostGreSQL continuous archiving Point in time recovery setup

Monday, November 21st, 2022

postgresql-db-backup-and-continuous-archiving-on-linux-keep-zabbix-backup-regularly

If you have set up a Zabbix server that is using as a database storage postgresql at home or company, then you will need to make sure you don’t loose any data by producing regular postgresql backups. This small article will show how to do the db backup in conventional way with pg_dump / pg_restore as well as how to set the continuous archiving point in time recovery end point recovery so called PITR..

1. Connect to the database and some postgres very basics

-bash-4.2$ psql DBNAME USERNAME

-bash-4.2$ psql zabbix zabbix

After you access a PostgreSQL database, you can run SQL queries and more. Here are some common psql commands:

•   To view help for psql commands, type \?.
•   To view help for SQL commands, type \h.
•   To view information about the current database connection, type \conninfo.
•   To list the database's tables and their respective owners, type \dt.
•   To list all of the tables, views, and sequences in the database, type \z.
•   To exit the psql program, type \q.

Fundamental backup approaches with postgres

As with everything that contains valuable data, PostgreSQL databases should be backed up regularly. While the procedure is essentially simple, it is important to have a clear understanding of the underlying techniques and assumptions.

There are three fundamentally different approaches to backing up PostgreSQL data:

•   SQL dump
•   File system level backup
•   Continuous archiving

Each has its own strengths and weaknesses; each is discussed in turn in the following sections.

2. Creating SQL Dump of postgresql target database

2.1 Manual SQL dump (custom dump format)

Use pg_dump's custom dump format. If PostgreSQL was built on a system with the zlib compression library installed, the custom dump format will compress data as it writes it to the output file.
This will produce dump file sizes similar to using gzip, but it has the added advantage that tables can be restored selectively.
The following command dumps a database using the custom dump format:

# sudo su – postgres
-bash-4.2% cd /var/lib/pgsql/9.5/backups/

-bash-4.2$ pg_dump -Fc zabbix > /var/lib/pgsql/9.5/backups/zabbixdbdump`date "+%d%m%y%H%M%S"`.gz

2.2 Schedule backups with cron job

To automate the job schedule a cron job to do the work, somethjing like below:

# Minute Hour Day of Month Month Day of Week Command
# (0-59) (0-23) (1-31) (1-12 or Jan-Dec) (0-6 or Sun-Sat)

@daily pg_dump -Fc zabbix > /var/lib/pgsql/9.5/backups/Zabbix_db_dump`date "+\%d\%m\%y\%H\%M\%S"`.gz

3. Restore a database from the dump file

3.1. Restoring a database using pg_restore (usually used)

A custom-format dump is not a script for psql, but instead must be restored with pg_restore, for example:

# pg_restore -d zabbix /var/lib/pgsql/9.5/backups/Zabbix_db_dump281118151005.gz

3.2. Second Option restoing with psql after creating a restore database

# su postgres

-bash-4.2$ psql template1

CREATE DATABASE zabbix OWNER zabbix ENCODING 'UTF8';
\q
-bash-4.2$ psql zabbix < /var/lib/pgsql/9.5/backups/zabbixdbdump291117151002.gz
exit

NOTE: if you get a permission denied error when trying to restore, check the Unix permissions on the backup file and all the parent directories.

4. Continuous Archiving and Point-in-Time Recovery (PITR) backups

At all times, PostgreSQL maintains a write ahead log (WAL) in the pg_xlog/ subdirectory of the cluster's data directory. The log records every change made to the database's data files. This log exists primarily for crash-safety purposes: if the system crashes, the database can be restored to consistency by "replaying" the log entries made since the last checkpoint. However, the existence of the log makes it possible to use a third strategy for backing up databases: we can combine a file-system-level backup with backup of the WAL files. If recovery is needed, we restore the file system backup and then replay from the backed-up WAL files to bring the system to a current state. This approach is more complex to administer than either of the previous approaches, but it has some significant benefits:

Setting Up the WAL Archiving

In an abstract sense, a running PostgreSQL system produces an indefinitely long sequence of WAL records. The system physically divides this sequence into WAL segment files, which are normally 16MB a piece (although the segment size can be altered when building PostgreSQL). The segment files are given numeric names that reflect their position in the abstract WAL sequence. When not using WAL archiving, the system normally creates just a few segment files and then "recycles" them by renaming no-longer-needed segment files to higher segment numbers. It's assumed that segment files whose contents precede the checkpoint-before-last are no longer of interest and can be recycled.

When archiving WAL data, we need to capture the contents of each segment file once it is filled, and save that data somewhere before the segment file is recycled for reuse.
Depending on the application and the available hardware, there could be many different ways of "saving the data somewhere": we could copy the segment files to an NFS-mounted directory on another machine, write them onto a tape drive (ensuring that you have a way of identifying the original name of each file), or batch them together and burn them onto CDs, or something else entirely. To provide the database administrator with flexibility, PostgreSQL tries not to make any assumptions about how the archiving will be done. Instead, PostgreSQL lets the administrator specify a shell command to be executed to copy a completed segment file to wherever it needs to go. The command could be as simple as a cp, or it could invoke a complex shell script — it's all up to you.

To enable WAL archiving, set the wal_level configuration parameter to archive or higher, archive_mode to on, and specify the shell command to use in the archive_command configuration parameter.

In practice these settings will always be placed in the postgresql.conf e.g. (/etc/postgresql/12/main/postgresql.conf) file.

In archive_command, %p is replaced by the path name of the file to archive, while %f is replaced by only the file name. (The path name is relative to the current working directory, i.e., the cluster's data directory.)

Use %% if you need to embed an actual % character in the command.

The simplest useful command to enable PITR would, be something like:

# – Archiving config setction

archive_mode = on                    # enables archiving; off, on, or always
# (change requires restart)

archive_command = '/bin/xz -2 -z < %p > /var/lib/pgsql/9.5/archivedir/%f'               # command to use to archive a logfile segment

archive_timeout = 1h            # force a logfile segment switch after this
                                                   # number of seconds; 0 disables

Tags: dumps, example, file, How to, point in time, postgresql, segment, setup, snapshot, WAL
Posted in Backups, Linux, Postgresql, System Administration | No Comments »

How to monitor Haproxy Application server backends with Zabbix userparameter autodiscovery scripts

Friday, May 13th, 2022

Haproxy is doing quite a good job in High Availability tasks where traffic towards multiple backend servers has to be redirected based on the available one to sent data from the proxy to.

Lets say haproxy is configured to proxy traffic for App backend machine1 and App backend machine2.

Usually in companies people configure a monitoring like with Icinga or Zabbix / Grafana to keep track on the Application server is always up and running. Sometimes however due to network problems (like burned Network Switch / router or firewall misconfiguration) or even an IP duplicate it might happen that Application server seems to be reporting reachable from some monotoring tool on it but unreachable from Haproxy server -> App backend machine2 but reachable from App backend machine1. And even though haproxy will automatically switch on the traffic from backend machine2 to App machine1. It is a good idea to monitor and be aware that one of the backends is offline from the Haproxy host.
In this article I'll show you how this is possible by using 2 shell scripts and userparameter keys config through the autodiscovery zabbix legacy feature.
Assumably for the setup to work you will need to have as a minimum a Zabbix server installation of version 5.0 or higher.

1. Create the required haproxy_discovery.sh and haproxy_stats.sh scripts

You will have to install the two scripts under some location for example we can put it for more clearness under /etc/zabbix/scripts

[root@haproxy-server1 ]# mkdir /etc/zabbix/scripts

[root@haproxy-server1 scripts]# vim haproxy_discovery.sh
#!/bin/bash
#
# Get list of Frontends and Backends from HAPROXY
# Example: ./haproxy_discovery.sh [/var/lib/haproxy/stats] FRONTEND|BACKEND|SERVERS
# First argument is optional and should be used to set location of your HAPROXY socket
# Second argument is should be either FRONTEND, BACKEND or SERVERS, will default to FRONTEND if not set
#
# !! Make sure the user running this script has Read/Write permissions to that socket !!
#
## haproxy.cfg snippet
# global
# stats socket /var/lib/haproxy/stats mode 666 level admin

HAPROXY_SOCK=""/var/run/haproxy/haproxy.sock
[ -n “$1” ] && echo $1 | grep -q ^/ && HAPROXY_SOCK="$(echo $1 | tr -d '\040\011\012\015')"

if [[ “$1” =~ (25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?):[0-9]{1,5} ]];
then
HAPROXY_STATS_IP="$1"
QUERYING_METHOD="TCP"
fi

QUERYING_METHOD="${QUERYING_METHOD:-SOCKET}"

query_stats() {
if [[ ${QUERYING_METHOD} == “SOCKET” ]]; then
echo "show stat" | socat ${HAPROXY_SOCK} stdio 2>/dev/null
elif [[ ${QUERYING_METHOD} == “TCP” ]]; then
echo "show stat" | nc ${HAPROXY_STATS_IP//:/ } 2>/dev/null
fi
}

get_stats() {
echo "$(query_stats)" | grep -v "^#"
}

[ -n “$2” ] && shift 1
case $1 in
B*) END="BACKEND" ;;
F*) END="FRONTEND" ;;
S*)
for backend in $(get_stats | grep BACKEND | cut -d, -f1 | uniq); do
for server in $(get_stats | grep "^${backend}," | grep -v BACKEND | grep -v FRONTEND | cut -d, -f2); do
serverlist="$serverlist,\n"'\t\t{\n\t\t\t"{#BACKEND_NAME}":"'$backend'",\n\t\t\t"{#SERVER_NAME}":"'$server'"}'
done
done
echo -e '{\n\t"data":[\n’${serverlist#,}’]}'
exit 0
;;
*) END="FRONTEND" ;;
esac

for frontend in $(get_stats | grep "$END" | cut -d, -f1 | uniq); do
felist="$felist,\n"'\t\t{\n\t\t\t"{#'${END}'_NAME}":"'$frontend'"}'
done
echo -e '{\n\t"data":[\n’${felist#,}’]}'

[root@haproxy-server1 scripts]# vim haproxy_stats.sh
#!/bin/bash
set -o pipefail

if [[ “$1” = /* ]]
then
HAPROXY_SOCKET="$1"
shift 0
else
if [[ “$1” =~ (25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?):[0-9]{1,5} ]];
then
HAPROXY_STATS_IP="$1"
QUERYING_METHOD="TCP"
shift 1
fi
fi

pxname="$1"
svname="$2"
stat="$3"

DEBUG=${DEBUG:-0}
HAPROXY_SOCKET="${HAPROXY_SOCKET:-/var/run/haproxy/haproxy.sock}"
QUERYING_METHOD="${QUERYING_METHOD:-SOCKET}"
CACHE_STATS_FILEPATH="${CACHE_STATS_FILEPATH:-/var/tmp/haproxy_stats.cache}"
CACHE_STATS_EXPIRATION="${CACHE_STATS_EXPIRATION:-1}" # in minutes
CACHE_INFO_FILEPATH="${CACHE_INFO_FILEPATH:-/var/tmp/haproxy_info.cache}" ## unused
CACHE_INFO_EXPIRATION="${CACHE_INFO_EXPIRATION:-1}" # in minutes ## unused
GET_STATS=${GET_STATS:-1} # when you update stats cache outsise of the script
SOCAT_BIN="$(which socat)"
NC_BIN="$(which nc)"
FLOCK_BIN="$(which flock)"
FLOCK_WAIT=15 # maximum number of seconds that "flock" waits for acquiring a lock
FLOCK_SUFFIX='.lock'
CUR_TIMESTAMP="$(date '+%s')"

debug() {
[ “${DEBUG}” -eq 1 ] && echo "DEBUG: $@" >&2 || true
}

debug "SOCAT_BIN => $SOCAT_BIN"
debug "NC_BIN => $NC_BIN"
debug "FLOCK_BIN => $FLOCK_BIN"
debug "FLOCK_WAIT => $FLOCK_WAIT seconds"
debug "CACHE_FILEPATH => $CACHE_FILEPATH"
debug "CACHE_EXPIRATION => $CACHE_EXPIRATION minutes"
debug "HAPROXY_SOCKET => $HAPROXY_SOCKET"
debug "pxname => $pxname"
debug "svname => $svname"
debug "stat => $stat"

# check if socat is available in path
if [ “$GET_STATS” -eq 1 ] && [[ $QUERYING_METHOD == “SOCKET” && -z “$SOCAT_BIN” ]] || [[ $QUERYING_METHOD == “TCP” && -z “$NC_BIN” ]]
then
echo 'ERROR: cannot find socat binary'
exit 126
fi

# if we are getting stats:
# check if we can write to stats cache file, if it exists
# or cache file path, if it does not exist
# check if HAPROXY socket is writable
# if we are NOT getting stats:
# check if we can read the stats cache file
if [ “$GET_STATS” -eq 1 ]
then
if [ -e “$CACHE_FILEPATH” ] && [ ! -w “$CACHE_FILEPATH” ]
then
echo 'ERROR: stats cache file exists, but is not writable'
exit 126
elif [ ! -w ${CACHE_FILEPATH%/*} ]
then
echo 'ERROR: stats cache file path is not writable'
exit 126
fi
if [[ $QUERYING_METHOD == “SOCKET” && ! -w $HAPROXY_SOCKET ]]
then
echo "ERROR: haproxy socket is not writable"
exit 126
fi
elif [ ! -r “$CACHE_FILEPATH” ]
then
echo 'ERROR: cannot read stats cache file'
exit 126
fi

# index:name:default
MAP="
1:pxname:@
2:svname:@
3:qcur:9999999999
4:qmax:0
5:scur:9999999999
6:smax:0
7:slim:0
8:stot:@
9:bin:9999999999
10:bout:9999999999
11:dreq:9999999999
12:dresp:9999999999
13:ereq:9999999999
14:econ:9999999999
15:eresp:9999999999
16:wretr:9999999999
17:wredis:9999999999
18:status:UNK
19:weight:9999999999
20:act:9999999999
21:bck:9999999999
22:chkfail:9999999999
23:chkdown:9999999999
24:lastchg:9999999999
25:downtime:0
26:qlimit:0
27:pid:@
28:iid:@
29:sid:@
30:throttle:9999999999
31:lbtot:9999999999
32:tracked:9999999999
33:type:9999999999
34:rate:9999999999
35:rate_lim:@
36:rate_max:@
37:check_status:@
38:check_code:@
39:check_duration:9999999999
40:hrsp_1xx:@
41:hrsp_2xx:@
42:hrsp_3xx:@
43:hrsp_4xx:@
44:hrsp_5xx:@
45:hrsp_other:@
46:hanafail:@
47:req_rate:9999999999
48:req_rate_max:@
49:req_tot:9999999999
50:cli_abrt:9999999999
51:srv_abrt:9999999999
52:comp_in:0
53:comp_out:0
54:comp_byp:0
55:comp_rsp:0
56:lastsess:9999999999
57:last_chk:@
58:last_agt:@
59:qtime:0
60:ctime:0
61:rtime:0
62:ttime:0
"

_STAT=$(echo -e "$MAP" | grep :${stat}:)
_INDEX=${_STAT%%:*}
_DEFAULT=${_STAT##*:}

debug "_STAT => $_STAT"
debug "_INDEX => $_INDEX"
debug "_DEFAULT => $_DEFAULT"

# check if requested stat is supported
if [ -z “${_STAT}” ]
then
echo "ERROR: $stat is unsupported"
exit 127
fi

# method to retrieve data from haproxy stats
# usage:
# query_stats "show stat"
query_stats() {
if [[ ${QUERYING_METHOD} == “SOCKET” ]]; then
echo $1 | socat ${HAPROXY_SOCKET} stdio 2>/dev/null
elif [[ ${QUERYING_METHOD} == “TCP” ]]; then
echo $1 | nc ${HAPROXY_STATS_IP//:/ } 2>/dev/null
fi
}

# a generic cache management function, that relies on 'flock'
check_cache() {
local cache_type="${1}"
local cache_filepath="${2}"
local cache_expiration="${3}"
local cache_filemtime
cache_filemtime=$(stat -c '%Y' "${cache_filepath}" 2> /dev/null)
if [ $((cache_filemtime+60*cache_expiration)) -ge ${CUR_TIMESTAMP} ]
then
debug "${cache_type} file found, results are at most ${cache_expiration} minutes stale.."
elif "${FLOCK_BIN}" –exclusive –wait "${FLOCK_WAIT}" 200
then
cache_filemtime=$(stat -c '%Y' "${cache_filepath}" 2> /dev/null)
if [ $((cache_filemtime+60*cache_expiration)) -ge ${CUR_TIMESTAMP} ]
then
debug "${cache_type} file found, results have just been updated by another process.."
else
debug "no ${cache_type} file found, querying haproxy"
query_stats "show ${cache_type}" > "${cache_filepath}"
fi
fi 200> "${cache_filepath}${FLOCK_SUFFIX}"
}

# generate stats cache file if needed
get_stats() {
check_cache 'stat' "${CACHE_STATS_FILEPATH}" ${CACHE_STATS_EXPIRATION}
}

# generate info cache file
## unused at the moment
get_info() {
check_cache 'info' "${CACHE_INFO_FILEPATH}" ${CACHE_INFO_EXPIRATION}
}

# get requested stat from cache file using INDEX offset defined in MAP
# return default value if stat is ""
get() {
# $1: pxname/svname
local _res="$("${FLOCK_BIN}" –shared –wait "${FLOCK_WAIT}" "${CACHE_STATS_FILEPATH}${FLOCK_SUFFIX}" grep $1 "${CACHE_STATS_FILEPATH}")"
if [ -z “${_res}” ]
then
echo "ERROR: bad $pxname/$svname"
exit 127
fi
_res="$(echo $_res | cut -d, -f ${_INDEX})"
if [ -z “${_res}” ] && [[ “${_DEFAULT}” != “@” ]]
then
echo "${_DEFAULT}"
else
echo "${_res}"
fi
}

# not sure why we'd need to split on backslash
# left commented out as an example to override default get() method
# status() {
# get "^${pxname},${svnamem}," $stat | cut -d\ -f1
# }

# this allows for overriding default method of getting stats
# name a function by stat name for additional processing, custom returns, etc.
if type get_${stat} >/dev/null 2>&1
then
debug "found custom query function"
get_stats && get_${stat}
else
debug "using default get() method"
get_stats && get "^${pxname},${svname}," ${stat}
fi

! NB ! Substitute in the script /var/run/haproxy/haproxy.sock with your haproxy socket location

You can download the haproxy_stats.sh here and haproxy_discovery.sh here

2. Create the userparameter_haproxy_backend.conf

[root@haproxy-server1 zabbix_agentd.d]# cat userparameter_haproxy_backend.conf
#
# Discovery Rule
#

# HAProxy Frontend, Backend and Server Discovery rules
UserParameter=haproxy.list.discovery[*],sudo /etc/zabbix/scripts/haproxy_discovery.sh SERVER
UserParameter=haproxy.stats[*],sudo /etc/zabbix/scripts/haproxy_stats.sh $2 $3 $4

# support legacy way

UserParameter=haproxy.stat.downtime[*],sudo /etc/zabbix/scripts/haproxy_stats.sh $2 $3 downtime

UserParameter=haproxy.stat.status[*],sudo /etc/zabbix/scripts/haproxy_stats.sh $2 $3 status

UserParameter=haproxy.stat.last_chk[*],sudo /etc/zabbix/scripts/haproxy_stats.sh $2 $3 last_chk

3. Create new simple template for the Application backend Monitoring and link it to monitored host

create-configuration-template-backend-monitoring

create-template-backend-monitoring-macros

Go to Configuration -> Hosts (find the host) and Link the template to it

4. Restart Zabbix-agent, in while check autodiscovery data is in Zabbix Server

[root@haproxy-server1 ]# systemctl restart zabbix-agent

Check in zabbix the userparameter data arrives, it should not be required to add any Items or Triggers as autodiscovery zabbix feature should automatically create in the server what is required for the data regarding backends to be in.

To view data arrives go to Zabbix config menus:

Configuration -> Hosts -> Hosts: (lookup for the haproxy-server1 hostname)

The autodiscovery should have automatically created the following prototypes

Now if you look inside Latest Data for the Host you should find some information like:

HAProxy Backend [backend1] (3 Items)

HAProxy Server [backend-name_APP/server1]: Connection Response
2022-05-13 14:15:04           History

HAProxy Server [backend-name/server2]: Downtime (hh:mm:ss)
2022-05-13 14:13:57   20:30:42       History

HAProxy Server [bk_name-APP/server1]: Status
2022-05-13 14:14:25   Up (1)       Graph
       ccnrlb01   HAProxy Backend [bk_CCNR_QA_ZVT] (3 Items)

HAProxy Server [bk_name-APP3/server1]: Connection Response
2022-05-13 14:15:05           History

HAProxy Server [bk_name-APP3/server1]: Downtime (hh:mm:ss)
2022-05-13 14:14:00   20:55:20       History

HAProxy Server [bk_name-APP3/server2]: Status
2022-05-13 14:15:08   Up (1)

To make alerting in case if a backend is down which usually you would like only left thing is to configure an Action to deliver alerts to some email address.

Tags: admin, configured, esac, file, frontend, Haproxy Application, How to, index, information, Network Switch, scripts, servers, shell scripts, sid, Zabbix Server
Posted in Bash Scripting, System Optimization, Zabbix | No Comments »

Configure own Media streaming minidlna Linux server to access data from your Smart TV

Friday, February 18th, 2022

If you happen to buy or already own or just have to install a Smart TV to be connected with a LAN Network to a Linux based custom built NAS (Network Attached Storage) server. You might benefit of the smart TV to Share and Watching the Disk Storage Pictures, Music, Video files from the NAS to the Smart TV using the Media Server protocol.

You have certainly already faced the Media Server at your life on many locations in stores and Mall Buildings, because virtually any reoccuring advertisements, movies projected on the TVs, Kids entertainment or Floor and Buildings Room location schedules or timeline promition schedules are streamed using the Media Server protocol, for many years now. Thus having a brief idea about Media Server proto existence is foundamental stuff to be aware of for sysadmins and programmers.

Shortly about DLNA UPnP Media Streaming Protocol

Assuming that your Smart TV has been already connected to your Wireless Router 2.4Ghz or 5Ghz Wifi, one would think that the easiest way to share the files with the SmartTV is via something like a simple SAMBA Linux server via smb:// cifs:// protocols or via the good old NFS Server, however most of Samsung Smart TV and many other in year 2022 does not have embedded support for Samba SMB / CIFS Protocol but instead have support for the DLNA (Digital Living Network Alliance) streaming support. DLNA is part of the UPnP (Universal Plug and Play) Protocols, UPnP is also known to those using and familiar with Windows Operating Systems realm simply as UPnP AV Media server or Windows Media server.
Windows Media server for those who never heard it or used it allows you to build a Playlists with Media files Video and Audio data files, that can be then later played remotely via a Local LAN or even long distance over TCP / IP remote side connected Internet network.

1. Set up and Stream data via Media server on Windows PC / notebook with integrated Windows Media server

Windows Media server configuration on Windows 7, 10 and 11 is a relatively easy to configure via:

Network and Sharing Center -> Media Streaming Options -> Turn on Media Streaming

Then you have to define the name of the Media Library, configure whether Media server should show
on the Local Netework for other conected devices and Allow or Block access from the other network present devices.

2. Using a more advanced Media Server to get rid about the limitation of DLNA set of supported file codecs.

The Windows default embedded DLNA server is the easiest and fastest one to set up, but it’s not necessarily the best option.
Due to the way DLNA works, you can only stream certain types of media codecs supported by the server. If you have other types of media not defaultly supported and defined by DLNA win server, it just won’t work.

Thus thanksfully it was developed other DLNA servers improve this by offering real-time transcoding.
If you try to play an unsupported file, they’ll transcode it on-the-fly, streaming the video in a supported format to your DLNA device.
Just to name few of the DLNA Media Streaming servers that have supported for larger MPG Video, MP3 / MP4 and other Audio formats encodings,
you can try Plex or the Universal Media Server both of which are free to use under freeware license and have versions for Linux and Mac OS.

3. Setting up a free as in freedom DLNA server MiniDLNA (ReadyMedia) on GNU / Linux

ReadyMedia (formerly known as MiniDLNA) is a simple media server software, with the aim of being fully compliant with DLNA/UPnP-AV clients. It was originally developed by a NETGEAR employee for the ReadyNAS product line.

MiniDNLA daemon serves media files (music, pictures, and video) to clients on a network. Linux Media servers clients you can use to test or scan your network for existent Media servers are multiple perhaps the most famous ones are applications such as totem (for QT users) and Kodi (for KDE).
The devices that can be used with minidlna are devices such as portable media players (iPod), Smartphones, Televisions, Tablets, and gaming systems (such as PS3 and Xbox 360) etc.

ReadyMedia is a simple, lightweight, the downside of it is It does not have a web interface for administration and must be configured by editing a text file. But for a simple Video streaming in most cases does a great job.

3.1 Install the minidlna software package

Minidlna is available out of the box on most linux distributions (Fedora / CentOS / Debian / Ubuntu etc.) as of year 2022.

Install on Debian Linux (Deb based distro)

media-server:~# apt install minidlna –yes

Install on Fedora / CentOS (other RPM based distro)

media-server:~# yum install -y minidlna

3.2 Configure minidlna

– /etc/minidlna.conf – main config file
Open with text editor and set user= , media_dir= , port=, friendly_name= , network_interface= variables as minimum.
To be add minidlnad support symlinks to external file locations, set also wide_links=yes

media-server:~# vim /etc/minidlna.conf

#user=minidlna
user=root
media_dir=/var/www/owncloud/data
network_interface=eth0,eth1

# Port number for HTTP traffic (descriptions, SOAP, media transfer).
# This option is mandatory (or it must be specified on the command-line using
# "-p").
port=8200
# Name that the DLNA server presents to clients.
# Defaults to "hostname: username".
#friendly_name=
friendly_name=DLNAServer Linux
# set this to yes to allow symlinks that point outside user-defined media_dirs.
wide_links=yes
# Automatic discovery of new files in the media_dir directory.
#inotify=yes

Keep in mind that it is supported to provide separete media_dir and provide different USB / External Hard Drive or SD Card sources separated only by content be it Video, Audio or Pictures short named in config as (A,V,P).

media_dir=P,/media/usb/photos
media_dir=V,/media/external-disk/videos
media_dir=A,/media/sd-card/music

You might want to diasble / ineable the inotify depending on your liking, if you don't plan to place new files automated to the NAS and don't care to get indexed and streamed from the Media server you can disable it with inotify=no otherwise keep that on.

– /etc/default/minidlna – additional startup config to set minidlnad (daemon) options such as setup to run with admin superuser root:root
(usually it is safe to leave it empty and set the user=root, whether needed straight from /etc/minidlna.conf
That's all now go on and launch the minidlna and enable it to automatically boot on Linux boot.

media-server:~# systemctl start minidlna
media-server:~# systemctl enable minidlna
media-server:~# systemctl status minidlna

3.3 Rebuilt minidlna database with data indexed files

If you need to re- generate minidlna's database.
To do so stop the minidlna server with the

media-server:~# systemctop stop minidlna

command, then issue the following command (both commands should be run as root):

media-server:~# minidlna -R

Since this command might kept in the background and keep the minidlna server running with incorrect flags, after a minute or two kill minidlna process and relaunch the server via sysctl.

media-server:~# killall -9 minidlna
media-server:~# systemctl start minidlna

3.4 Permission Issues / Scanning issues

If you plan to place files in /home directory. You better have a seperate partition or folder *outside* your "home" directory devoted to your media. Default user with which minidlna runs is minidlna, this could prevent some files with root or other users being red. So either run minidlna daemon as root or as other user with whom all media files should be accessible.
If service runs as root:root, and still getting some scanning issues, check permissions on your files and remove special characters from file names.

media-server:~# tail -10 /var/log/minidlna/minidlna.log
[2022/02/17 22:51:36] scanner.c:489: warn: Unsuccessful getting details for /var/www/owncloud/data/Videos/Family-Videos/FILE006.MPG
[2022/02/17 22:52:08] scanner.c:819: warn: Scanning /var/www/owncloud/data finished (10637 files)!
[2022/02/17 22:52:08] playlist.c:135: warn: Parsing playlists…
[2022/02/17 22:52:08] playlist.c:269: warn: Finished parsing playlists.
minidlna.c:1126: warn: Starting MiniDLNA version 1.3.0.
minidlna.c:1186: warn: HTTP listening on port 8200
scanner.c:489: warn: Unsuccessful getting details for /var/www/owncloud/data/admin/files/origin/External SD card/media/Viber Images/IMG-4477de7b1eee273d5e6ae25236c5c223-V.jpg
scanner.c:489: warn: Unsuccessful getting details for /var/www/owncloud/data/Videos/Family-Video/FILE006.MPG
playlist.c:135: warn: Parsing playlists…
playlist.c:269: warn: Finished parsing playlists.

3.5. Fix minidlna Inotify errors

In /etc/sysctl.conf

Add:

fs.inotify.max_user_watches=65536

in a blank line at end of file and do

media-server:~# sysctl -p

Debugging minidlna problems, index errors, warnings etc

minidlna does write by default to /var/log/minidlna/minidlna.log inspect the log closely and you should get most of the time what is wrong with it.
Note that some files might not get indexed because minidlna won't support the strange file codecs such as SWF encoding, if you have some important files to stream that are not indexed by minidlna, then install and try one of the more sophisticated free software Media Servers for Linux:

MediaTomb
UniversalMediaServer
Plex

plex-media-streaming-server-screenshot

Note that most Linux users from my quick research shows, MediaTomb is the preferred advanced features Open Source Linux Media Server of choice for most of the guys.

mediatomb-linux-media-streaming-server-picture.jpg.webp

4. Test minidlna Linux servers works, getting information of other DLNA Servers on the network

media-server:~# lynx -dump http://127.0.0.1:8200
MiniDLNA status

Media library

Audio files 0
Video files 455
Image files 10182

Connected clients

ID Type IP Address HW Address Connections
0 Samsung Series [CDEFJ] 192.168.1.11 7C:0A:3D:88:A6:FA 0
1 Generic DLNA 1.5 192.168.0.241 00:16:4E:1D:48:05 0
2 Generic DLNA 1.5 192.168.1.18 00:16:3F:0D:45:05 0
3 Unknown 127.0.0.1 FF:FF:FF:FF:FF:FF 0

-1 connections currently open

Note that there is -1 connections (no active connections) currently to the server.
The 2 Generic DLNA 1.5 IPs are another DLNA servers provided by a OpenXEN hosted Windows 7 Virtual machines, that are also broadcasting their existence in the network. The Samsung Series [CDEFJ] is the DLNA client on the Samsung TV found, used to detect and stream data from the just configured Linux dlna server.

The DLNA Protocol enabled devices on a network as you can see are quite easy to access, querying localhost on the 8200 server dumps, what minidlna knows, the rest of IPs connecting should not be able to receive this info. But anyways since the minidlna does not have a special layers of security to access it, but the only way to restrict is filtering the 8200 port, it is a very good idea to put a good iptables firewall on the machine to allow only the devices that should have access to the data.

Further more if you happen to need to access the Media files on Linux from GUI you might use some client as upmentioned totem, VLC or if you need something more feature rich Java eezUPnP .

That's all folks !
Enjoy your media on the TV 🙂

Tags: access, configured, data, default user, Digital Living Network Alliance, DLNA, Enjoy, file, file names, Install, linux?, Media, Open Source Linux Media Server, org, owncloud, root root, setting, software package, stuff, support, wiki, www
Posted in Entertainment, Linux, Video Streaming | No Comments »

Log rsyslog script incoming tagged string message to separate external file to prevent /var/log/message from string flood

Wednesday, December 22nd, 2021

rsyslog_logo-log-external-tag-scripped-messages-to-external-file-linux-howto

If you're using some external bash script to log messages via rsyslogd to some of the multiple rsyslog understood data tubes (called in rsyslog language facility levels) and you want Rsyslog to move message string to external log file, then you had the same task as me few days ago.

For example you have a bash shell script that is writting a message to rsyslog daemon to some of the predefined facility levels be it:

kern,user,cron, auth etc. or some local

and your logged script data ends under the wrong file location /var/log/messages , /var/log/secure , var/log/cron etc. However you need to log everything coming from that service to a separate file based on the localX (fac. level) the usual way to do it is via some config like, as you would usually do it with rsyslog variables as:

local1.info /var/log/custom-log.log

# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none;local0.none;local1.none /var/log/messages

Note the local1.none is instructing the rsyslog not to log anything from local1 facility towards /var/log/message.
But what if this due to some weirdness in configuration of rsyslog on the server or even due to some weird misconfiguration in

/etc/systemd/journald.conf such as:

[Journal]
Storage=persistent
RateLimitInterval=0s
RateLimitBurst=0
SystemMaxUse=128M
SystemMaxFileSize=32M
MaxRetentionSec=1month
MaxFileSec=1week
ForwardToSyslog=yes
SplitFiles=none

Due to that config and especially the FowardToSyslog=yes, the messages sent via the logger tool to local1 still end up inside /var/log/messages, not nice huh ..

The result out of that is anything being sent with a predefined TAGGED string via the whatever.sh script which uses the logger command (if you never use it check man logger) to enter message into rsyslog with cmd like:

# logger -p local1.info -t TAG_STRING

# logger -p local2.warn test
# tail -2 /var/log/messages
Dec 22 18:58:23 pcfreak rsyslogd: — MARK —
Dec 22 19:07:12 pcfreak hipo: test

was nevertheless logged to /var/log/message.
Of course /var/log/message becomes so overfilled with "junk" shell script data not related to real basic Operating system adminsitration, so this prevented any critical or important messages that usually should come under /var/log/message / /var/log/syslog to be lost among the big quantities of other tagged tata reaching the log.

After many attempts to resolve the issue by modifying /etc/rsyslog.conf as well as the messed /etc/systemd/journald.conf (which by the way was generated with this strange values with an OS install time automation ansible stuff). It took me a while until I found the solution on how to tell rsyslog to log the tagged message strings into an external separate file. From my 20 minutes of research online I have seen multitudes of people in different Linux OS versions to experience the same or similar issues due to whatever, thus this triggered me to write this small article on the solution to rsyslog.

The solution turned to be pretty easy but requires some further digging into rsyslog, Redhat's basic configuration on rsyslog documentation is a very nice reading for starters, in my case I've used one of the Propery-based compare-operations variable contains used to select my tagged message string.

1. Add msg contains compare-operations to output log file and discard the messages

[root@centos bin]# vi /etc/rsyslog.conf

# config to log everything logged to rsyslog to a separate file
:msg, contains, "tag_string:/" /var/log/custom-script-log.log
:msg, contains, "tag_string:/" ~

Substitute quoted tag_string:/ to whatever your tag is and mind that it is better this config is better to be placed somewhere near the beginning of /etc/rsyslog.conf and touch the file /var/log/custom-script-log.log and give it some decent permissions such as 755, i.e.

1.1 Discarding a message

The tilda sign – ~

as placed to the end of the msg, contains is the actual one to tell the string to be discarded so it did not end in /var/log/messages.

Alternative rsyslog config to do discard the unwanted message once you have it logged is with the
rawmsg variable, like so:

# config to log everything logged to rsyslog to a separate file
:msg, contains, "tag_string:/" /var/log/custom-script-log.log
:rawmsg, isequal, "tag_string:/" stop

Other way to stop logging immediately after log is written to custom file across some older versions of rsyslog is via the &stop

:msg, contains, "tag_string:/" /var/log/custom-script-log.log
& stop

I don't know about other versions but Unfortunately the &stop does not work on RHEL 7.9 with installed rpm package rsyslog-8.24.0-57.el7_9.1.x86_64.

1.2 More with property based filters basic exclusion of string

Property based filters can do much more, you can for example, do regular expression based matches of strings coming to rsyslog and forward to somewhere.

To select syslog messages which do not contain any mention of the words fatal and error with any or no text between them (for example, fatal lib error), type:

:msg, !regex, "fatal .* error"

2. Create file where tagged data should be logged and set proper permissions

[root@centos bin]# touch /var/log/custom-script-log.log
[root@centos bin]# chmod 755 /var/log/custom-script-log.log

3. Test rsyslogd configuration for errors and reload rsyslog

[root@centos ]# rsyslogd -N1
rsyslogd: version 8.24.0-57.el7_9.1, config validation run (level 1), master config /etc/rsyslog.conf
rsyslogd: End of config validation run. Bye.
[root@centos ]# systemctl restart rsyslog
[root@centos ]# systemctl status rsyslog
● rsyslog.service – System Logging Service
Loaded: loaded (/usr/lib/systemd/system/rsyslog.service; enabled; vendor preset: enabled)
Active: active (running) since Wed 2021-12-22 13:40:11 CET; 3h 5min ago
Docs: man:rsyslogd(8)
http://www.rsyslog.com/doc/
Main PID: 108600 (rsyslogd)
CGroup: /system.slice/rsyslog.service
└─108600 /usr/sbin/rsyslogd -n

4. Property-based compare-operations supported by rsyslog table

Compare-operation	Description
`contains`	Checks whether the provided string matches any part of the text provided by the property. To perform case-insensitive comparisons, use `contains_i` .
`isequal`	Compares the provided string against all of the text provided by the property. These two values must be exactly equal to match.
`startswith`	Checks whether the provided string is found exactly at the beginning of the text provided by the property. To perform case-insensitive comparisons, use `startswith_i` .
`regex`	Compares the provided POSIX BRE (Basic Regular Expression) against the text provided by the property.
`ereregex`	Compares the provided POSIX ERE (Extended Regular Expression) regular expression against the text provided by the property.
`isempty`	Checks if the property is empty. The value is discarded. This is especially useful when working with normalized data, where some fields may be populated based on normalization result.

5. Rsyslog understanding Facility levels

Here is a list of facility levels that can be used.

Note: The mapping between Facility Number and Keyword is not uniform over different operating systems and different syslog implementations, so among separate Linuxes there might be diference in the naming and numbering.

Facility Number	Keyword	Facility Description
0	kern	kernel messages
1	user	user-level messages
2	mail	mail system
3	daemon	system daemons
4	auth	security/authorization messages
5	syslog	messages generated internally by syslogd
6	lpr	line printer subsystem
7	news	network news subsystem
8	uucp	UUCP subsystem
9		clock daemon
10	authpriv	security/authorization messages
11	ftp	FTP daemon
12	–	NTP subsystem
13	–	log audit
14	–	log alert
15	cron	clock daemon
16	local0	local use 0 (local0)
17	local1	local use 1 (local1)
18	local2	local use 2 (local2)
19	local3	local use 3 (local3)
20	local4	local use 4 (local4)
21	local5	local use 5 (local5)
22	local6	local use 6 (local6)
23	local7	local use 7 (local7)

6. rsyslog Severity levels (sublevels) accepted by facility level

As defined in RFC 5424, there are eight severity levels as of year 2021:

Code	Severity	Keyword	Description	General Description
0	Emergency	emerg (panic)	System is unusable.	A "panic" condition usually affecting multiple apps/servers/sites. At this level it would usually notify all tech staff on call.
1	Alert	alert	Action must be taken immediately.	Should be corrected immediately, therefore notify staff who can fix the problem. An example would be the loss of a primary ISP connection.
2	Critical	crit	Critical conditions.	Should be corrected immediately, but indicates failure in a primary system, an example is a loss of a backup ISP connection.
3	Error	err (error)	Error conditions.	Non-urgent failures, these should be relayed to developers or admins; each item must be resolved within a given time.
4	Warning	warning (warn)	Warning conditions.	Warning messages, not an error, but indication that an error will occur if action is not taken, e.g. file system 85% full – each item must be resolved within a given time.
5	Notice	notice	Normal but significant condition.	Events that are unusual but not error conditions – might be summarized in an email to developers or admins to spot potential problems – no immediate action required.
6	Informational	info	Informational messages.	Normal operational messages – may be harvested for reporting, measuring throughput, etc. – no action required.
7	Debug	debug	Debug-level messages.	Info useful to developers for debugging the application, not useful during operations.

7. Sample well tuned configuration using severity and facility levels and immark, imuxsock, impstats

Below is sample config using severity and facility levels

# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none;local0.none;local1.none /var/log/messages

Note the local0.none; local1.none tells rsyslog to not log from that facility level to /var/log/messages.

If you need a complete set of rsyslog configuration fine tuned to have a proper logging with increased queues and included configuration for loggint to remote log aggegator service as well as other measures to prevent the system disk from being filled in case if something goes wild with a logging service leading to a repeatedly messages you might always contact me and I can help 🙂
Other from that sysadmins might benefit from a sample set of configuration prepared with the Automated rsyslog config builder or use some fine tuned config for rsyslog-8.24.0-57.el7_9.1.x86_64 on Redhat 7.9 (Maipo) rsyslog_config_redhat-2021.tar.gz.

To sum it up rsyslog though looks simple and not an important thing to pre

Tags: config, daemons, email, Facility Number, file, flood, Informational, log messages, none, property, rsyslog, type, use, variables, Warning
Posted in Linux, Remote System Administration, System Administration | No Comments »

KVM Virtual Machine RHEL 8.3 Linux install on Redhat 8.3 Linux Hypervisor with custom tailored kickstart.cfg

Friday, January 22nd, 2021

If you don't have tried it yet Redhat and CentOS and other RPM based Linux operationg systems that use anaconda installer is generating a kickstart file after being installed under /root/{anaconda-ks.cfg,initial-setup- ks.cfg,original-ks.cfg} immediately after the OS installation completes. Using this Kickstart file template you can automate installation of Redhat installation with exactly the same configuration as many times as you like by directly loading your /root/original-ks.cfg file in RHEL installer.

Here is the official description of Kickstart files from Redhat:

"The Red Hat Enterprise Linux installation process automatically writes a Kickstart file that contains the settings for the installed system. This file is always saved as /root/anaconda-ks.cfg. You may use this file to repeat the installation with identical settings, or modify copies to specify settings for other systems."

Kickstart files contain answers to all questions normally asked by the text / graphical installation program, such as what time zone you want the system to use, how the drives should be partitioned, or which packages should be installed. Providing a prepared Kickstart file when the installation begins therefore allows you to perform the installation automatically, without need for any intervention from the user. This is especially useful when deploying Redhat based distro (RHEL / CentOS / Fedora …) on a large number of systems at once and in general pretty useful if you're into the field of so called "DevOps" system administration and you need to provision a certain set of OS to a multitude of physical servers or create or recreate easily virtual machines with a certain set of configuration.

1. Create /vmprivate storage directory where Virtual machines will reside

First step on the Hypervisor host which will hold the future created virtual machines is to create location where it will be created:

[root@redhat ~]# lvcreate –size 140G –name vmprivate vg00
[root@redhat ~]# mkfs.ext4 -j -b 4096 /dev/mapper/vg00-vmprivate
[root@redhat ~]# mount /dev/mapper/vg00-vmprivate /vmprivate

To view what is the situation with Logical Volumes and VG group names:

[root@redhat ~]# vgdisplay -v|grep -i vmprivate -A7 -B7
Segments 1
Allocation inherit
Read ahead sectors auto
– currently set to 8192
Block device 253:0

— Logical volume —
LV Path /dev/vg00/vmprivate
LV Name vmprivate
VG Name vg00
LV UUID VVUgsf-FXq2-TsMJ-QPLw-7lGb-Dq5m-3J9XJJ
LV Write Access read/write
LV Creation host, time main.hostname.com, 2021-01-20 17:26:11 +0100
LV Status available
# open 1
LV Size 150.00 GiB

Note that you'll need to have the size physically available on a SAS / SSD Hard Drive physically connected to Hypervisor Host.

To make the changes Virtual Machines storage location directory permanently mounted add to /etc/fstab

/dev/mapper/vg00-vmprivate /vmprivate ext4 defaults,nodev,nosuid 1 2

[root@redhat ~]# echo '/dev/mapper/vg00-vmprivate /vmprivate ext4 defaults,nodev,nosuid 1 2' >> /etc/fstab

2. Second we need to install the following set of RPM packages on the Hypervisor Hardware host

[root@redhat ~]# yum install qemu-kvm qemu-img libvirt virt-install libvirt-client virt-manager libguestfs-tools virt-install virt-top -y

3. Enable libvirtd on the host

[root@redhat ~]# lsmod | grep -i kvm
[root@redhat ~]# systemctl enable libvirtd

4. Configure network bridging br0 interface on Hypervisor

In /etc/sysconfig/network-scripts/ifcfg-eth0 you need to include:

NM_CONTROLED=NO

Next use nmcli redhat configurator to create the bridge (you can use ip command instead) but since the tool is the redhat way to do it lets do it their way ..

[root@redhat ~]# nmcli connection delete eno3
[root@redhat ~]# nmcli connection add type bridge autoconnect yes con-name br0 ifname br0
[root@redhat ~]# nmcli connection modify br0 ipv4.addresses 10.80.51.16/26 ipv4.method manual
[root@redhat ~]# nmcli connection modify br0 ipv4.gateway 10.80.51.1
[root@redhat ~]# nmcli connection modify br0 ipv4.dns 172.20.88.2
[root@redhat ~]# nmcli connection add type bridge-slave autoconnect yes con-name eno3 ifname eno3 master br0
[root@redhat ~]# nmcli connection up br0

5. Prepare a working kickstart.cfg file for VM

Below is a sample kickstart file I've used to build a working fully functional Virtual Machine with Red Hat Enterprise Linux 8.3 (Ootpa) .

#version=RHEL8
#install
# Run the Setup Agent on first boot
firstboot --enable
ignoredisk --only-use=vda
# Use network installation
#url --url=http://hostname.com/rhel/8/BaseOS
##url --url=http://171.23.8.65/rhel/8/os/BaseOS
# Use text mode install
text
#graphical
# System language
#lang en_US.UTF-8
keyboard --vckeymap=us --xlayouts='us'
# Keyboard layouts
##keyboard us
lang en_US.UTF-8
# Root password
rootpw $6$gTiUCif4$YdKxeewgwYCLS4uRc/XOeKSitvDJNHFycxWVHi.RYGkgKctTMCAiY2TErua5Yh7flw2lUijooOClQQhlbstZ81 --iscrypted
# network-stuff
# place ip=your_VM_IP, netmask, gateway, nameserver hostname 
network --bootproto=static --ip=10.80.21.19 --netmask=255.255.255.192 --gateway=10.80.21.1 --nameserver=172.30.85.2 --device=eth0 --noipv6 --hostname=FQDN.VMhost.com --onboot=yes
# if you need just localhost initially configured uncomment and comment above
##network В --device=lo --hostname=localhost.localdomain
# System authorization information
authconfig --enableshadow --passalgo=sha512 --enablefingerprint
# skipx
skipx
# Firewall configuration
firewall --disabled
# System timezone
timezone Europe/Berlin
# Clear the Master Boot Record
##zerombr
# Repositories
## Add RPM repositories from KS file if necessery
#repo --name=appstream --baseurl=http://hostname.com/rhel/8/AppStream
#repo --name=baseos --baseurl=http://hostname.com/rhel/8/BaseOS
#repo --name=inst.stage2 --baseurl=http://hostname.com ff=/dev/vg0/vmprivate
##repo --name=rhsm-baseos В  В --baseurl=http://172.54.8.65/rhel/8/rhsm/x86_64/BaseOS/
##repo --name=rhsm-appstream --baseurl=http://172.54.8.65/rhel/8/rhsm/x86_64/AppStream/
##repo --name=os-baseos В  В  В --baseurl=http://172.54.9.65/rhel/8/os/BaseOS/
##repo --name=os-appstream В  --baseurl=http://172.54.8.65/rhel/8/os/AppStream/
#repo --name=inst.stage2 --baseurl=http://172.54.8.65/rhel/8/BaseOS
# Disk partitioning information set proper disk sizing
##bootloader --location=mbr --boot-drive=vda
bootloader --append=" crashkernel=auto tsc=reliable divider=10 plymouth.enable=0 console=ttyS0 " --location=mbr --boot-drive=vda
# partition plan
zerombr
clearpart --all --drives=vda --initlabel
part /boot --size=1024 --fstype=ext4 --asprimary
part swap --size=1024
part pv.01 --size=30000 --grow --ondisk=vda
##part pv.0 --size=80000 --fstype=lvmpv
#part pv.0 --size=61440 --fstype=lvmpv
volgroup s pv.01
logvol / --vgname=s --size=15360 --name=root --fstype=ext4
logvol /var/cache/ --vgname=s --size=5120 --name=cache --fstype=ext4 --fsoptions="defaults,nodev,nosuid"
logvol /var/log --vgname=s --size=7680 --name=log --fstype=ext4 --fsoptions="defaults,nodev,noexec,nosuid"
logvol /tmp --vgname=s --size=5120 --name=tmp --fstype=ext4 --fsoptions="defaults,nodev,nosuid"
logvol /home --vgname=s --size=5120 --name=home --fstype=ext4 --fsoptions="defaults,nodev,nosuid"
logvol /opt --vgname=s --size=2048 --name=opt --fstype=ext4 --fsoptions="defaults,nodev,nosuid"
logvol /var/log/audit --vgname=s --size=3072 --name=audit --fstype=ext4 --fsoptions="defaults,nodev,nosuid"
logvol /var/spool --vgname=s --size=2048 --name=spool --fstype=ext4 --fsoptions="defaults,nodev,nosuid"
logvol /var --vgname=s --size=7680 --name=var --fstype=ext4 --fsoptions="defaults,nodev,nosuid"
# SELinux configuration
selinux --disabled
# Installation logging level
logging --level=debug
# reboot automatically
reboot
###
%packages
@standard
python3
pam_ssh_agent_auth
-nmap-ncat
#-plymouth
#-bpftool
-cockpit
#-cryptsetup
-usbutils
#-kmod-kvdo
#-ledmon
#-libstoragemgmt
#-lvm2
#-mdadm
-rsync
#-smartmontools
-sos
-subscription-manager-cockpit
# Tune Linux vm.dirty_background_bytes (IMAGE-439)
# The following tuning causes dirty data to begin to be background flushed at
# 100 Mbytes, so that it writes earlier and more often to avoid a large build
# up and improving overall throughput.
echo "vm.dirty_background_bytes=100000000" >> /etc/sysctl.conf
# Disable kdump
systemctl disable kdump.service
%end

Important note to make here is the MD5 set root password string in (rootpw) line this string can be generated with openssl or mkpasswd commands :

Method 1: use openssl cmd to generate (md5, sha256, sha512) encrypted pass string

[root@redhat ~]# openssl passwd -6 -salt xyz test
$6$xyz$rjarwc/BNZWcH6B31aAXWo1942.i7rCX5AT/oxALL5gCznYVGKh6nycQVZiHDVbnbu0BsQyPfBgqYveKcCgOE0

Note: passing -1 will generate an MD5 password, -5 a SHA256 encryption and -6 SHA512 encrypted string (logically recommended for better security)

Method 2: (md5, sha256, sha512)

[root@redhat ~]# mkpasswd –method=SHA-512 –stdin

The option –method accepts md5, sha-256 and sha-512
Theoretically there is also a kickstart file generator web interface on Redhat's site here however I never used it myself but instead use above kickstart.cfg

6. Install the new VM with virt-install cmd

Roll the new preconfigured VM based on above ks template file use some kind of one liner command line like below:

[root@redhat ~]# virt-install -n RHEL8_3-VirtualMachine –description "CentOS 8.3 Virtual Machine" –os-type=Linux –os-variant=rhel8.3 –ram=8192 –vcpus=8 –location=/vmprivate/rhel-server-8.3-x86_64-dvd.iso –disk path=/vmprivate/RHEL8_3-VirtualMachine.img,bus=virtio,size=70 –graphics none –initrd-inject=/root/kickstart.cfg –extra-args "console=ttyS0 ks=file:/kickstart.cfg"

7. Use a tiny shell script to automate VM creation

For some clarity and better automation in case you plan to repeat VM creation you can prepare a tiny bash shell script:

#!/bin/sh
KS_FILE='kickstart.cfg';
VM_NAME='RHEL8_3-VirtualMachine';
VM_DESCR='CentOS 8.3 Virtual Machine';
RAM='8192';
CPUS='8';
# size is in Gigabytes
VM_IMG_SIZE='140';
ISO_LOCATION='/vmprivate/rhel-server-8.3-x86_64-dvd.iso';
VM_IMG_FILE_LOC='/vmprivate/RHEL8_3-VirtualMachine.img';

virt-install -n "$VMNAME" –description "$VM_DESCR" –os-type=Linux –os-variant=rhel8.3 –ram=8192 –vcpus=8 –location="$ISO_LOCATION" –disk path=$VM_IMG_FILE,bus=virtio,size=$IMG_VM_SIZE –graphics none –initrd-inject=/root/$KS_FILE –extra-args "console=ttyS0 ks=file:/$KS_FILE"

A copy of virt-install.sh script can be downloaded here

Wait for the installation to finish it should be visualized and if all installation is smooth you should get a login prompt use the password generated with openssl tool and test to login, then disconnect from the machine by pressing CTRL + ] and try to login via TTY with

[root@redhat ~]# virst list –all
Id Name State
—————————
2 RHEL8_3-VirtualMachine running

…

[root@redhat ~]# virsh console RHEL8_3-VirtualMachine

redhat8-login-prompt

One last thing I recommend you check the official documentation on Kickstart2 from CentOS official website

In case if you later need to destroy the VM and the respective created Image file you can do it with:

[root@redhat ~]# virsh destroy RHEL8_3-VirtualMachine
[root@redhat ~]# virsh undefine RHEL8_3-VirtualMachine

Don't forget to celebreate the success and give this nice article a credit by sharing this nice tutorial with a friend or by placing a link to it from your blog 🙂

Enjoy !

Tags: access, addresses, Administration, after, ALL, allows, and, Below, better security, cfg, connection, custom, file, installation, mount dev, need, Redhat, use, virtual machines, vm
Posted in Everyday Life, Linux, System Administration, Various, Virtual Machines | 3 Comments »

Install and configure IBM TSM Backup Archive client on CentOS, Redhat, Fedora Linux

Friday, November 19th, 2021

In this article I'll explain how to install IBM Spectrum Protect client well known over the years with TSM (Tivoli Service Manager) which as of time of writting perhaps the best backup solution for Linux / Unix server. The install process is rather trivial, but there are few configuration things worthy to mention and explain. IBM Tivoli is a 2 component system consisting of Client and Server just like most solutions, we know thus before proceeding you should already have setup a functional backup server with attached and configured storages but this is a topic for another future article.

1. Login and configure TSM server to add the backup host machine you'll be connecting

If you don't have the access to remote server if you're in a Corporation network and you don't have the Admin credentials to Tivoli Server, ask a colleague who has access to add you the hostname you would like to backup.
He should generate you as a minimum a connect password and hand it back to you in encrypted form (GPG / PGP etc.). Besides setting the new client to connect password, it is necessery to communicate when and how frequently the backup is to be prepared and respectively he should set the required backup schedule configuration etc.

2. Check OS version

As minimum this article assumes you'll be using some kind of RPM based distribution.

[root@centos ~]# cat /etc/redhat-release

CentOS Linux release 7.9.2009 (Core)

In my case this was CentOS, any other RPM based distribution should work. For specifics check out

README_*.htm

3. Remove version lock If version lock is there

On our servers we do use one feature of yum called versionlock to prepare some colleague to install or update some package with yum by mistake.

If you don't know versionlock and you want to install and use it you will have to install RPM package yum-plugin-versionlock

[root@centos ~]# yum install -y yum-plugin-versionlock
…

Versionlock is used for Restricting a Package to a Fixed Version Number with yum.

In case if you have versionlock installed to list the versionlock restricted versions to not be ovewritten do:

[root@centos ~]# yum versionlock list
…
….
……
0:gskcrypt64-8.0-55.14.*
0:lm_sensors-libs-3.4.0-8.20160601gitf9185e5.el7.*
0:sysstat-10.1.5-19.el7.*
0:TIVsm-API64-8.1.10-0.*
versionlock list done

To clean up already set lock for current installed packages

[root@centos ~]# yum versionlock clear

4. Install TSM Backup API and Backup Archiver Client RPMs

If you have already created a /etc/yum.repos.d/3party.repo with a mirror of IBM binaries and repository is enabled you can perhaps do something like:

[root@centos ~]# yum install -y Tivsm-BA

Otherwise go and follow official IBM Tivoli installation instructions

In short once .rpms are downloaded from IBM’s site, this comes to:

4.1. Install the required dependent library gskcrypt64

[root@centos ~]# rpm -U gskcrypt64-8.x.x.x.linux.x86_64.rpm gskssl64-8.x.x.x.linux.x86_64.rpm

4.2. Install Tivoli Storage Manager API

Application programming interface (API) is API which contains the Tivoli Storage Manager API shared libraries and samples, default directory install location is /opt/tivoli/tsm/client/api/bin64

[root@centos ~]# rpm -i TIVsm-API64.x86_64.rpm

4.3. Install the backup-archive client

[root@centos ~]# rpm -i TIVsm-BA.x86_64.rpm

Its most files will be stored under dir /opt/tivoli/tsm/client/ba/bin.

This directory is considered to be the default installation directory for many backup-archive client files. The sample system-options file (dsm.sys.smp) is written to this directory. If the DSM_DIR environment variable is not set, the dsmc executable file, the resource files, and the dsm.sys file are stored in this directory.

If DSM_CONFIG is not set, the client user-options file must be in this directory.

If you do not define DSM_LOG, writes messages to the dsmerror.log and dsmsched.log files in the current working directory.

Optionally: Install the Common Inventory Technology package that is used by the API. This package depends on the API so it must be installed after the API package is installed.

[root@centos ~]# rpm -i TIVsm-APIcit.x86_64.rpm

Optionally: Install the Common Inventory Technology package the client uses to send PVU metrics to the server. This package depends on the client package so it must be installed after the client package is installed.

[root@centos ~]# rpm -i TIVsm-BAcit.x86_64.rpm

If finally no optional stuff is installed the minumum for operational (Tivoli) Spectrum Protect is to at least below 2 RPMs being installed:

[root@centos ~]# rpm -qa |grep -i TivSM

TIVsm-BA-8.1.10-0.x86_64

TIVsm-API64-8.1.10-0.x86_64

5. Create /var/tsm log directory

[root@centos ~]# mkdir /var/tsm
[root@centos ~]# chown root:root /var/tsm

6. Hardcode to /etc/hosts TSM server hosts / addresses somewhere at File EOF

[root@centos ~]# vim /etc/hosts

Includfe at End of File

### TSM Server Records ###

10.50.8.1 tsmserver1.backup-server-fqdn.com tsmserver1

10.50.8.2 tsmserver2.backup-server-fqdn.com tsmserver2

Hardcoding records is a good idea if you don't use DNS at all, e.g. for security /etc/resolv.conf is empty or if you have configured the machine Name Record resolve order to be first read from /etc/hosts. For those who don't know in Linux this is done via /etc/nsswitch.conf (for more how it is configured – man nsswitch.conf command)

7. Create /opt/tivoli/tsm/client/ba/bin/{dsm.sys,dsm.opt,inclexcl.tsm}
configs

dsm.sys main config mandatory file

[root@centos ~]# vi /opt/tivoli/tsm/client/ba/bin/dsm.sys

Paste inside and save :wq!:

*=================================================================

*

* File:     /usr/tivoli/tsm/client/ba/bin/dsm.sys

*

* Who:

*

* Project: TSM

*

* Purpose: Tivoli Storage Manager client user options

*

* Release: Tivoli BA client 5.3.4

*

* Author:

*

*=================================================================

*

* Modification history:

*

* [Chg.]   [When]       [Who]           [What]

* 1.       19.11.2021      Georgi Georgiev    Creation.

*

*=================================================================

* TSM Server Location

*   Servername           tsmserver1

*   COMMmethod           TCPip

*   TCPPort              1500

*   TCPServeraddress     tsmserver1.backup-server-fqdn.com

*   Passwordaccess       generate

*   SCHEDLOGNAME         /logs/tivoli/sched.log

*   SCHEDLOGRETENTION    21 D

*   SCHEDMODE            PROMPT

*   ERRORLOGNAME         /logs/tivoli/dsmerror.log

*   ERRORLOGRETENTION    30 D

*   INCLEXCL             /opt/tivoli/tsm/client/ba/bin/inclexcl.tsm

*=========================================================================

*TSM SERVER

   Servername           tsmserver2

   COMMmethod           TCPip

   TCPPort              1400

   TCPServeraddress     tsmserver2.backup-server-fqdn.com

   NodeName             server-hostname1

   Passwordaccess       generate

   SCHEDLOGNAME         /var/tsm/sched.log

   SCHEDLOGRETENTION    21 D

   SCHEDMODE            prompted

   MANAGEDServices      schedule

   ERRORLOGNAME         /var/tsm/dsmerror.log

   ERRORLOGRETENTION    30 D

*   INCLEXCL             /opt/tivoli/tsm/client/ba/bin/inclexcl.tsm

##########

TCPServeraddress – should equal to the complete hostname where the TSM backup server is configured

Nodename – should be your server hostname from where you create backup usually returned by hostname command

dsm.opt Additional options file

[root@centos ~]# vi /opt/tivoli/tsm/client/ba/bin/dsm.opt

*=================================================================

*

* File: /opt/tivoli/tsm/client/ba/bin/dsm.opt

*

*

* Project: TSM

*

* Purpose: Tivoli Storage Manager client user options

*

* Release: Tivoli BA client 5.3.4

*

* Author:

*

*=================================================================

*

* Modification history:

*

* [Chg.] [When] [Who] [What]

*

*

*=================================================================

* Quiet

SERVERNAME TSMSERVER2

* Sample config you might want to modify it according to your preferences

Archsymlinkasfile no

Subdir yes

dateformat 2

followsymbolic yes

guitreeviewafterbackup yes

quiet

DOMAIN "/"

DOMAIN "/boot"

DOMAIN "/home"

* below line is commented
* DOMAIN "/var"

########

Define what directories files / shuld be excluded from backup

# vi /opt/tivoli/tsm/client/ba/bin/inclexcl.tsm

exclude.dir /tmp

exclude.dir /var/cache

exclude /var/lock/*

exclude.dir /var/log

exclude /var/spool/*

exclude.dir /var/tmp

exclude.dir /var/tsm

exclude.dir /var/run

Note ! Within dsm.sys you have also to have define SCHEDMODE prompted instead of SCHEDMODE polling:

Config should be as:

SCHEDMODE prompted

managedservices schedule

8. Authenticate to tsmserver2 host for a first time to remote Tivoli Backup Serv

Run the Tivoli connect client dsmc

# dsmc

→ empty username (if TSMServer configured so or otherwise provide the user provided by an Admin)
→ enter password given by TSM Admin

q sched → should answer with a schedule

tsm> q sched

    Schedule Name: INC_DAILY_PROD

      Description: 03:30 Days Schedule Prod

   Schedule Style: Classic

           Action: Incremental

          Options:

          Objects:

         Priority: 5

   Next Execution: 13 Hours and 38 Minutes

         Duration: 2 Hours

           Period: 1 Day

      Day of Week: Any

            Month:

     Day of Month:

    Week of Month:

           Expire: Never

    Schedule Name: Archives_365_DAYS_prod

      Description: 365 Days Archive monthly Prod

   Schedule Style: Classic

           Action: Archive

          Options: -archmc=ARCHIVE_365_DAYS-description="Archive of Logfiles" -deletefiles

          Objects: /var/log/*.gz

         Priority: 5

   Next Execution: 492 Hours and 38 Minutes

         Duration: 2 Hours

           Period: 1 Month

      Day of Week: Any

            Month:

     Day of Month:

    Week of Month:

           Expire: Never

tsm>

As you see above you get some details on when backup is to be executed, e.g. what time it is scheduled (in this case it is 03:30 Days Schedule Prod early morning), how often it is supposed to re-run, what is it priority, what is the type of backup (i.e. incremental) and how long is left until next execution.

incr command→ Start the backup (will create first incremental full backup)

q fi command→ To show (list) backuped FileSystems

tsm> q fi

#     Last Incr Date          Type    File Space Name

——————————————————————————–

1     10-11-2021 12:08:43     EXT2    /

2     10-11-2021 12:09:01     EXT4    /boot

3     10-11-2021 12:09:03     EXT4    /home

4     10-11-2021 12:09:24     EXT4    /var

5     10-11-2021 12:03:17     EXT4    /vz

As you can see we have a recent backup of our OpenVZ machine Hypervisor host 🙂

quit → logout from dsmc

9. Check entires within defined /var/tsm/{dsmsched.log, sched.log, dsmerror.log}

If problems connecting check /var/tsm/dsmerror.log

Once incr backup command is finished check that all is backupped normal within /var/tsm/sched.log

10. Start dsmcad process

[root@centos ~]# /etc/init.d/dsmcad start

[root@centos ~]# systemctl start dsmcad

11. Add service to auto start on server boot time

Depending on where, the machine is old legacy System V system or a systemd RedHat / CentOS you should either add it in chkconfig tool to auto start or enable the systemctl service.

11.1. Enable dsmcad autoboot on SystemV legacy machine

[root@centos ~]# chkconfig –add dsmcad

11.2 Enable dmscad boot on recent systemd machine as of year 2021

[root@centos ~]# systemctl enable dsmcad.service

or

[root@centos ~]# systemctl start dsmcad.service

12. Enable back yum versionlock for all installed RPM packages (Optional)

[root@centos ~]# yum versionlock \*

Tags: bin, boot time, CentOS, clear, client, com, command, directory, DNS, file, good, idea, Install, installed, Last Incr Date Type File Space Name, Linux Unix, login, order, priority, read, Redhat, rpm, SCHEDMODE, server boot, server hostname, systemctl, Tivoli Backup Serv, type, username, var, yum versionlock
Posted in Backups, Linux, Various | 1 Comment »

CentOS disable SELinux permanently or one time on grub Linux kernel boot time

Saturday, July 24th, 2021

1. Office 365 cloud connected computer and a VirtualBox hosted machine with SELINUX preventing it to boot

At my job we're in process of migrating my old Lenovo Laptop Thinkpad model L560 Laptop to Dell Latitude 5510 wiith Intel Core i5 vPro CPU and 256 Gb SSD Hard Drive. The new laptops are generally fiine though they're not even a middle class computers and generally I prefer thinkpads. The sad thing out of this is our employee decided to migrate to Office 365 (again perhaps another stupid managerial decision out of an excel sheet wtih a balance to save some money …

As you can imagine Office 365 is not really PCI Standards compliant and not secure since our data is stored in Microsoft cloud and theoretically Microsoft has and owns our data or could wipe loose the data if they want to. The other obvious security downside I've noticed with the new "Secure PCI complaint laptop" is the initial PC login screen which by default offers fingerprint authentication or the even worse and even less secure face recognition, but obviosly everyhing becomes more and more crazy and people become less and less cautious for security if that would save money or centralize the data … In the name of security we completely waste security that is very dubious paradox I don't really understand but anyways, enough rant back to the main topic of this article is how to and I had to disable selinux?

As part of Migration I've used Microsoft OneDrive to copy old files from the Thinkpad to the Latitude (as on the old machine USB's are forbidden and I cannot copy over wiith a siimple USB driive, as well as II have no right to open the laptop and copy data from the Hard driive, and even if we had this right without breaking up some crazy company policy that will not be possible as the hard drive data on old laptop is encrypted, the funny thing is that the new laptop data comes encrypted and there is no something out of the box as BitDefender or McAffee incryption (once again, obviously our data security is a victim of some managarial decisions) …

2. OneDrive copy problems unable to sync some of the copied files to Onedrive

Anyways as the Old Laptop's security is quite paranoid and we're like Fort Nox, only port 80 and port 443 connections to the internet can be initiated to get around this harsh restrictions it was as simple to use a Virtualbox Virtual Machine. So on old laptop I've installed a CentOS 7 image which I used so far and I used one drive to copy my vbox .vdi image on the new laptop work machine.

The first head buml was the .vdi which seems to be prohibited to be copied to OneDrive, so to work around this I had to rename the origianl CentOS7.vdi to CentOS7.vdi-renamed on old laptop and once the data is in one drive copy my Vitualbox VM/ directory from one drive to the Dell Latitude machine and rename the .vdi-named towards .vdi as well as import it from the latest installed VirtualBox on the new machine.

3. Disable SELINUX from initial grub boot

So far so good but as usual happens with miigrations I've struck towards another blocker, the VM image once initiated to boot from Virtualbox badly crashed with some complains that selinux cannot be loaded.
Realizing CentOS 7 has the more or less meaningless Selinux, I've took the opportunity to disable SeLinux.

To do so I've booted the Kernel with Selinux disabled from GRUB2 loader prompt before Kernel and OS Userland boots.

I thought I need to type the information on the source in grub. What I did is very simple, on the Linux GRUB boot screen I've pressed

'e' keyboard letter

that brought the grub boot loader into edit mode.

Then I had to add selinux=0 on the edited selected kernel version, as shown in below screenshot:

Next to boot the Linux VM without Selinux enabled one time, just had to press together

Ctrl+X then add selinux=0 on the edited selected kernel version, that should be added as shown in the screenshot somewhere after the line of
root=/dev/mapper/…..

4. Permanently Disable Selinux on CentOS 7

Once I managed to boot Virtual Machine properly with Oracle Virtualbox, to permanently disabled selinux I had to:

Once booted into CentOS, to check the status of selinux run:

# sestatus
Copy
SELinux status: enabled
SELinuxfs mount: /sys/fs/selinux
SELinux root directory: /etc/selinux
Loaded policy name: targeted
Current mode: enforcing
Mode from config file: enforcing
Policy MLS status: enabled
Policy deny_unknown status: allowed
Max kernel policy version: 31

5. Disable SELinux one time with setenforce command

You can temporarily change the SELinux mode from targeted to permissive with the following command:

# setenforce 0

Next o permanently disable SELinux on your CentOS 7 next time the system boots, Open the /etc/selinux/config file and set the SELINUX mod parameter to disabled.

On CentOS 7 you can edit the kernel parameters in /etc/default/grub (in the GRUB_CMDLINE_LINUX= key) and set selinux=0 so on next VM / PC boot we boot with a SELINUX disabled for example add RUB_CMDLINE_LINUX=selinux=0 to the file then you have to regenerate your Grub config like this:

# grub2-mkconfig -o /etc/grub2.cfg
# grub2-mkconfig -o /etc/grub2-efi.cfg

Further on to disable SeLinux on OS level edit /etc/selinux

Default /etc/selinux/config with selinux enabled should look like so:

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing – SELinux security policy is enforced.
# permissive – SELinux prints warnings instead of enforcing.
# disabled – No SELinux policy is loaded.
SELINUX=enforcing
# SELINUXTYPE= can take one of these two values:
# targeted – Targeted processes are protected,
# mls – Multi Level Security protection.
SELINUXTYPE=targeted

To disable SeLinux modify the file to be something like:

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing – SELinux security policy is enforced.
# permissive – SELinux prints warnings instead of enforcing.
# disabled – No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of these two values:
# targeted – Targeted processes are protected,
# mls – Multi Level Security protection.
SELINUXTYPE=targeted

6. Check SELINUX status is disabled

# sestatus

SELinux status: disabled

So in this article shottly was explained shortly the fake security adopted by using Microsoft Cloud environment Offiice 365, my faced OneDrive copy issues (which prevented even my old laptop Virtual Machine to boot properly and the handy trick to rename the file that is unwilling to get copied from old PC towards m$ OneDrive as well as the grub trick to disable Selinux permanently from grub2.

Tags: boot time, CentOS, config, copy, file, Mode, Office, SELINUXTYPE, system, Virtualbox Virtual Machine
Posted in Linux, Security, System Administration | No Comments »

Short SSL generate new and self-signed certificates PEM, view and convert to and from PKCS12 to java key store cookbook commands cheat sheet

Tuesday, January 12th, 2021

Below is a short compilation of common used openssl commands (a kind of cookbook) helpful for sysadmins who has to commonly deal with OpenSSL certificates.

Lets say you have to generate new certificate / key and a PEM files, prepare self-signed certificates, show CSR / PEM or KEY ssl file contents, get information about certificate such as expiry date a type of encryption algorithm or sign certificate with self-signed authority convert PEM to PKCS12, convert from PKCS12 file format to .PEM, convert java X509 to java key store SSL encryptionor convert java key store format certificate to PKCS12, then below will be of use to you.

1. Generate Private RSA Key with 2048 bits

# openssl genrsa -out $ (hostname -f) .key 2048

2. Create CSR file

# openssl req -new -key $ (hostname -f) .key -out $ (hostname -f) .csr

3. Create a Self Certified Certificate:

# openssl x509 -req -days 30 -in $ (hostname -f) .csr -signkey $ (hostname -f) .key -out $ (hostname -f) .crt
Enter password:

# openssl rsa -in key.pem -out newkey.pem

4. Show CSR file content

# openssl req -in newcsr.csr -noout -text

5. Get Certificate version / serial number / signature algorithm / RSA key lenght / modulus / exponent etc.

# openssl x509 -in newcert.pem -noout -text

6. Server certificate as CA self signeded

# openssl ca -in newcert.csr -notext -out newcert.pem

7. Generate a certificate signing request based on an existing certificate

# openssl x509 -x509toreq -in certificate.crt -out CSR.csr -signkey privateKey.key

8. Convert .pem / .key / .crt file format to pkcs12 format

# openssl pkcs12 -export -in newcert.pem -inkey newkey.key -certfile ca.crt -out newcert.p12

9. Convert pkcs12 pfx to common .pem

# openssl pkcs12 -in mycert.pfx -out mycert.pem

10. The Formats available

# openssl x509 -inform the -in certificate.cer -out certificate.crt

11. Convert a pkcs # 7 certificate into PEM format

# openssl pkcs7 -in cert.p7c -inform DER -outform PEM -out certificate.p7b
# openssl pkcs7 -print_certs -in certificate.p7b -out certificate.pem

12. Convert X509 to java keystore file

# java -cp not-yet-commons-ssl-0.3.11.jar org.apache.commons.ssl.KeyStoreBuilder pass_for_new_keystore key.key certificate.crt

13. Convert java keystore file to pkcs12

# keytool -importkeystore -srckeystore keystore.jks -destkeystore intermediate.p12 -deststoretype PKCS12

Tags: Below, cheat sheet, file, generate, key, openssl, pem, say, Self Certified Certificate, view
Posted in System Administration, Web and CMS | No Comments »

☩ Walking in Light with Christ – Faith, Computing, Diary

Posts Tagged ‘file’

Switching from PasswordSafe to Keepass database, migrating .psafe3 to .kdbx format howto

How to monitor Haproxy Application server backends with Zabbix userparameter autodiscovery scripts

1. Create the required haproxy_discovery.sh and haproxy_stats.sh scripts

2. Create the userparameter_haproxy_backend.conf

3. Create new simple template for the Application backend Monitoring and link it to monitored host

4. Restart Zabbix-agent, in while check autodiscovery data is in Zabbix Server

Short SSL generate new and self-signed certificates PEM, view and convert to and from PKCS12 to java key store cookbook commands cheat sheet

Daily Bible quote

GET ARTICLE UPDATES

Useful blog? Help it:

Links to Other Places

Recent Posts

Ads

Categories

About Myself

Recent Comments

Top Post Views

blogtopsites

Posts Tagged ‘file’

1. Why use Zabbix-Proxy hidden advantages of using Zabbix-Proxy ?

2. Install zabbix-proxy-sqlite3 rpm package from Zabbix Official Repositories download page

NOTE ! Before you install proxy, keep in mind that your proxy version must match the Zabbix server version !

3. Generate a PSK random secret key and set proper permissions for zabbix-proxy directories

4. Configure zabbix proxy to be able to connect to Zabbix Server

5. Check and make sure the installed zabbix proxy as well as the zabbix_proxy server zabbix_agentd client and zabbix_server are at the same major version release

6. Starting the zabbix-proxy for a first time

7. Configure zabbix-agentd to use your just new brand new zabbix-proxy

8. Create and Configure new proxy into the Zabbix-server host

9. Debugging problems with zabix-proxy and zabbix-agentd connectivity to proxy

Wrapping it up

1. Connect to the database and some postgres very basics

Fundamental backup approaches with postgres

2. Creating SQL Dump of postgresql target database

2.1 Manual SQL dump (custom dump format)

2.2 Schedule backups with cron job

3. Restore a database from the dump file

3.1. Restoring a database using pg_restore (usually used)

3.2. Second Option restoing with psql after creating a restore database

4. Continuous Archiving and Point-in-Time Recovery (PITR) backups

Setting Up the WAL Archiving

1. Create the required haproxy_discovery.sh and haproxy_stats.sh scripts

2. Create the userparameter_haproxy_backend.conf

3. Create new simple template for the Application backend Monitoring and link it to monitored host

4. Restart Zabbix-agent, in while check autodiscovery data is in Zabbix Server

Shortly about DLNA UPnP Media Streaming Protocol

1. Set up and Stream data via Media server on Windows PC / notebook with integrated Windows Media server

2. Using a more advanced Media Server to get rid about the limitation of DLNA set of supported file codecs.

3. Setting up a free as in freedom DLNA server MiniDLNA (ReadyMedia) on GNU / Linux

3.1 Install the minidlna software package

3.2 Configure minidlna

3.3 Rebuilt minidlna database with data indexed files

3.4 Permission Issues / Scanning issues

3.5. Fix minidlna Inotify errors

4. Test minidlna Linux servers works, getting information of other DLNA Servers on the network

1. Add msg contains compare-operations to output log file and discard the messages

1.1 Discarding a message

The tilda sign – ~

1.2 More with property based filters basic exclusion of string

2. Create file where tagged data should be logged and set proper permissions

3. Test rsyslogd configuration for errors and reload rsyslog

4. Property-based compare-operations supported by rsyslog table

5. Rsyslog understanding Facility levels

6. rsyslog Severity levels (sublevels) accepted by facility level

7. Sample well tuned configuration using severity and facility levels and immark, imuxsock, impstats

1. Create /vmprivate storage directory where Virtual machines will reside

2. Second we need to install the following set of RPM packages on the Hypervisor Hardware host

3. Enable libvirtd on the host

4. Configure network bridging br0 interface on Hypervisor

5. Prepare a working kickstart.cfg file for VM

6. Install the new VM with virt-install cmd

7. Use a tiny shell script to automate VM creation

1. Login and configure TSM server to add the backup host machine you'll be connecting

2. Check OS version

3. Remove version lock If version lock is there

4. Install TSM Backup API and Backup Archiver Client RPMs

5. Create /var/tsm log directory

6. Hardcode to /etc/hosts TSM server hosts / addresses somewhere at File EOF

7. Create /opt/tivoli/tsm/client/ba/bin/{dsm.sys,dsm.opt,inclexcl.tsm} configs

8. Authenticate to tsmserver2 host for a first time to remote Tivoli Backup Serv

9. Check entires within defined /var/tsm/{dsmsched.log, sched.log, dsmerror.log}

10. Start dsmcad process

11. Add service to auto start on server boot time

12. Enable back yum versionlock for all installed RPM packages (Optional)

2. OneDrive copy problems unable to sync some of the copied files to Onedrive

3. Disable SELINUX from initial grub boot

4. Permanently Disable Selinux on CentOS 7

5. Disable SELinux one time with setenforce command

6. Check SELINUX status is disabled

Daily Bible quote

GET ARTICLE UPDATES

Useful blog? Help it:

Links to Other Places

Recent Posts

Ads

Categories

About Myself

Recent Comments

Tags

7. Create /opt/tivoli/tsm/client/ba/bin/{dsm.sys,dsm.opt,inclexcl.tsm}
configs