Posts Tagged ‘config’

« Older Entries

Install specific zabbix-agent version / Downgrade Zabbix Agent client to exact preferred old RPM version on CentOS / Fedora / RHEL Linux from repo

Wednesday, June 7th, 2023

zabbix-update-downgrade-on-centos-rhel-fedora-and-other-rpm-based-linux-zabbix-logo

 

In below article, I'll give you the short Update zabbix procedure to specific version release, if you need to have it running in tandem with rest of zabbix infra, as well as expain shortly how to downgrade zabbix version to a specific release number
to match your central zabbix-serveror central zabbix proxies.

The article is based on personal experience how to install / downgrade the specific zabbuix-agent  release on RPM based distros.
I know this is pretty trivial stuff but still, hope this might be useful to some sysadmin out there thus I decided to quickly blog it.

 

1. Prepare backup of zabbix_agentd.conf
 

# cp -rpf /etc/zabbix/zabbix_agentd.conf /home/your-user/zabbix_agentd.conf.bak.$(date +"%b-%d-%Y")

 

2. Create zabbix repo source file in yum.repos.d directory

# cd /etc/yum.repos.d/
# vim zabbix.repo 

 

[zabbix-5.0]

name=Zabbix 5.0 repo

baseurl=http://zabixx-rpm-mirrors-site.com/centos/external/zabbix-5.0/8/x86_64/

enabled=1

gpgcheck=0

 

3. Update zabbix-agent to a specific defined version

# yum search zabbix-agent –enablerepo zabbix-5.0

To update zabbix-agent for RHEL 7.*

# yum install zabbix-agent-5.0.34-1.el7.x86_64


For RHEL 8.*

# yum install zabbix-agent-5.0.34-1.el8.x86_64


4. Restart zabbix-agentd and check its status to make sure it works correctly
 

# systemctl status zabbix-agentd
# systemctl restart zabbix-agentd
# systemctl status zabbix-agentd


Go to zabbix-server WEB GUI interface and check that data is delivered as normally in Latest Data for the host fom recent time, to make sure host monitoring is continuing flawlessly as before change.

NB !: If yum use something like versionlock is enabled remove the versionlock for package and update then, otherwise it will (weirldly look) look like the package is missing.
I'm saying that because I've hit this issue and was wondering why i cannot install the zabbix-agent even though the version is listed, available and downloadable from the repository.


5. Downgrade agent-client to specific version (Install old version of Zabbix from Repo)
 

Sometimes by mistake you might have raised the Zabbix-agent version to be higher release than the zabbix-server's version and thus breach out the Zabbix documentation official recommendation to keep
up the zabbix-proxy, zabbix-server and zabbix-agent at the exactly same version major and minor version releases. 

If so, then you would want to decrease / downgrade the version, to match your Zabbix overall infrastructure exact version for each of Zabbix server -> Zabbix Proxy server -> Agent clients.

To downgrade the version, I prefer to create some backups, just in case for all /etc/zabbix/ configurations and userparameter scripts (from experience this is useful as sometimes some RPM binary update packages might cause /etc/zabbix/zabbix_agentd.conf file to get overwritten. To prevent from restoring zabbix_agentd.conf from your most recent backup hence, I prefer to just crease the zabbix config backups manually.
 

# cd /root

# mkdir -p /root/backup/zabbix-agent 

# tar -czvf zabbix_agent.tar.gz /etc/zabbix/

# tar -xzvf zabbix_agent.tar.gz 


Then list the available installable zabbix-agent versions
 

[root@sysadminshelp:~]# yum –showduplicates list zabbix-agent
Заредени плъгини: fastestmirror
Determining fastest mirrors
 * base: centos.uni-sofia.bg
 * epel: fedora.ipacct.com
 * extras: centos.uni-sofia.bg
 * remi: mirrors.uni-ruse.bg
 * remi-php74: mirrors.uni-ruse.bg
 * remi-safe: mirrors.uni-ruse.bg
 * updates: centos.uni-sofia.bg
Инсталирани пакети
zabbix-agent.x86_64                                                     5.0.30-1.el7                                                     @zabbix
Налични пакети
zabbix-agent.x86_64                                                     5.0.0-1.el7                                                      zabbix
zabbix-agent.x86_64                                                     5.0.1-1.el7                                                      zabbix
zabbix-agent.x86_64                                                     5.0.2-1.el7                                                      zabbix
zabbix-agent.x86_64                                                     5.0.3-1.el7                                                      zabbix
zabbix-agent.x86_64                                                     5.0.4-1.el7                                                      zabbix
zabbix-agent.x86_64                                                     5.0.5-1.el7                                                      zabbix
zabbix-agent.x86_64                                                     5.0.6-1.el7                                                      zabbix
zabbix-agent.x86_64                                                     5.0.7-1.el7                                                      zabbix
zabbix-agent.x86_64                                                     5.0.8-1.el7                                                      zabbix
zabbix-agent.x86_64                                                     5.0.9-1.el7                                                      zabbix
zabbix-agent.x86_64                                                     5.0.10-1.el7                                                     zabbix
zabbix-agent.x86_64                                                     5.0.11-1.el7                                                     zabbix
zabbix-agent.x86_64                                                     5.0.12-1.el7                                                     zabbix
zabbix-agent.x86_64                                                     5.0.13-1.el7                                                     zabbix
zabbix-agent.x86_64                                                     5.0.14-1.el7                                                     zabbix
zabbix-agent.x86_64                                                     5.0.15-1.el7                                                     zabbix
zabbix-agent.x86_64                                                     5.0.16-1.el7                                                     zabbix
zabbix-agent.x86_64                                                     5.0.17-1.el7                                                     zabbix
zabbix-agent.x86_64                                                     5.0.18-1.el7                                                     zabbix
zabbix-agent.x86_64                                                     5.0.19-1.el7                                                     zabbix
zabbix-agent.x86_64                                                     5.0.20-1.el7                                                     zabbix
zabbix-agent.x86_64                                                     5.0.21-1.el7                                                     zabbix
zabbix-agent.x86_64                                                     5.0.22-1.el7                                                     zabbix
zabbix-agent.x86_64                                                     5.0.23-1.el7                                                     zabbix
zabbix-agent.x86_64                                                     5.0.24-1.el7                                                     zabbix
zabbix-agent.x86_64                                                     5.0.25-1.el7                                                     zabbix
zabbix-agent.x86_64                                                     5.0.26-1.el7                                                     zabbix
zabbix-agent.x86_64                                                     5.0.27-1.el7                                                     zabbix
zabbix-agent.x86_64                                                     5.0.28-1.el7                                                     zabbix
zabbix-agent.x86_64                                                     5.0.29-1.el7                                                     zabbix
zabbix-agent.x86_64                                                     5.0.30-1.el7                                                     zabbix
zabbix-agent.x86_64                                                     5.0.31-1.el7                                                     zabbix
zabbix-agent.x86_64                                                     5.0.32-1.el7                                                     zabbix
zabbix-agent.x86_64                                                     5.0.33-1.el7                                                     zabbix
zabbix-agent.x86_64                                                     5.0.34-1.el7                                                     zabbix

 

Next lets install the most recent zabbix-versoin from the CentOS repo, which for me as of time of writting this article is 5.0.34.

# yum downgrade -y zabbix-agent-5.0.34-1.el7

# cp -rpf /root/backup/zabbix-agent/etc/zabbix/zabbix_agentd.conf /etc/zabbix/

# systemctl start zabbix-agent.service

# systemctl enable  zabbix-agent.service
 

# zabbix_agentd -V
zabbix_agentd (daemon) (Zabbix) 5.0.30
Revision 2c96c38fb4b 28 November 2022, compilation time: Nov 28 2022 11:27:43

Copyright (C) 2022 Zabbix SIA
License GPLv2+: GNU GPL version 2 or later <https://www.gnu.org/licenses/>.
This is free software: you are free to change and redistribute it according to
the license. There is NO WARRANTY, to the extent permitted by law.

This product includes software developed by the OpenSSL Project
for use in the OpenSSL Toolkit (http://www.openssl.org/).

Compiled with OpenSSL 1.0.1e-fips 11 Feb 2013
Running with OpenSSL 1.0.1e-fips 11 Feb 2013
 

That's all folks you should be at your custom selected preferred version of zabbix-agent.
Enjoy ! 🙂

Tags: , , , , , , , , ,
Posted in Linux, Linux Package Management, Monitoring, Various, Zabbix | 2 Comments »

Log rsyslog script incoming tagged string message to separate external file to prevent /var/log/message from string flood

Wednesday, December 22nd, 2021

rsyslog_logo-log-external-tag-scripped-messages-to-external-file-linux-howto

If you're using some external bash script to log messages via rsyslogd to some of the multiple rsyslog understood data tubes (called in rsyslog language facility levels) and you want Rsyslog to move message string to external log file, then you had the same task as me few days ago.

For example you have a bash shell script that is writting a message to rsyslog daemon to some of the predefined facility levels be it:
 

kern,user,cron, auth etc. or some local

and your logged script data ends under the wrong file location /var/log/messages , /var/log/secure , var/log/cron etc. However  you need to log everything coming from that service to a separate file based on the localX (fac. level) the usual way to do it is via some config like, as you would usually do it with rsyslog variables as:
 

local1.info                                            /var/log/custom-log.log

# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none;local0.none;local1.none        /var/log/messages


Note the local1.none is instructing the rsyslog not to log anything from local1 facility towards /var/log/message. 
But what if this due to some weirdness in configuration of rsyslog on the server or even due to some weird misconfiguration in

/etc/systemd/journald.conf such as:

[Journal]
Storage=persistent
RateLimitInterval=0s
RateLimitBurst=0
SystemMaxUse=128M
SystemMaxFileSize=32M
MaxRetentionSec=1month
MaxFileSec=1week
ForwardToSyslog=yes
SplitFiles=none

Due to that config and especially the FowardToSyslog=yes, the messages sent via the logger tool to local1 still end up inside /var/log/messages, not nice huh ..

The result out of that is anything being sent with a predefined TAGGED string via the whatever.sh script which uses the logger command  (if you never use it check man logger) to enter message into rsyslog with cmd like:
 

# logger -p local1.info -t TAG_STRING

# logger -p local2.warn test
# tail -2 /var/log/messages
Dec 22 18:58:23 pcfreak rsyslogd: — MARK —
Dec 22 19:07:12 pcfreak hipo: test


was nevertheless logged to /var/log/message.
Of course /var/log/message becomes so overfilled with "junk" shell script data not related to real basic Operating system adminsitration, so this prevented any critical or important messages that usually should come under /var/log/message / /var/log/syslog to be lost among the big quantities of other tagged tata reaching the log.

After many attempts to resolve the issue by modifying /etc/rsyslog.conf as well as the messed /etc/systemd/journald.conf (which by the way was generated with this strange values with an OS install time automation ansible stuff). It took me a while until I found the solution on how to tell rsyslog to log the tagged message strings into an external separate file. From my 20 minutes of research online I have seen multitudes of people in different Linux OS versions to experience the same or similar issues due to whatever, thus this triggered me to write this small article on the solution to rsyslog.

The solution turned to be pretty easy but requires some further digging into rsyslog, Redhat's basic configuration on rsyslog documentation is a very nice reading for starters, in my case I've used one of the Propery-based compare-operations variable contains used to select my tagged message string.
 

1. Add msg contains compare-operations to output log file and discard the messages

[root@centos bin]# vi /etc/rsyslog.conf

# config to log everything logged to rsyslog to a separate file
:msg, contains, "tag_string:/"         /var/log/custom-script-log.log
:msg, contains, "tag_string:/"    ~

Substitute quoted tag_string:/ to whatever your tag is and mind that it is better this config is better to be placed somewhere near the beginning of /etc/rsyslog.conf and touch the file /var/log/custom-script-log.log and give it some decent permissions such as 755, i.e.
 

1.1 Discarding a message


The tilda sign –  ~ 

as placed to the end of the msg, contains is the actual one to tell the string to be discarded so it did not end in /var/log/messages.

Alternative rsyslog config to do discard the unwanted message once you have it logged is with the
rawmsg variable, like so:

 

# config to log everything logged to rsyslog to a separate file
:msg, contains, "tag_string:/"         /var/log/custom-script-log.log
:rawmsg, isequal, "tag_string:/" stop

Other way to stop logging immediately after log is written to custom file across some older versions of rsyslog is via the &stop

:msg, contains, "tag_string:/"         /var/log/custom-script-log.log
& stop

I don't know about other versions but Unfortunately the &stop does not work on RHEL 7.9 with installed rpm package rsyslog-8.24.0-57.el7_9.1.x86_64.

1.2 More with property based filters basic exclusion of string 

Property based filters can do much more, you can for example, do regular expression based matches of strings coming to rsyslog and forward to somewhere.

To select syslog messages which do not contain any mention of the words fatal and error with any or no text between them (for example, fatal lib error), type:

:msg, !regex, "fatal .* error"

 

2. Create file where tagged data should be logged and set proper permissions
 

[root@centos bin]# touch /var/log/custom-script-log.log
[root@centos bin]# chmod 755 /var/log/custom-script-log.log


3. Test rsyslogd configuration for errors and reload rsyslog

[root@centos ]# rsyslogd -N1
rsyslogd: version 8.24.0-57.el7_9.1, config validation run (level 1), master config /etc/rsyslog.conf
rsyslogd: End of config validation run. Bye.
[root@centos ]# systemctl restart rsyslog
[root@centos ]#  systemctl status rsyslog 
● rsyslog.service – System Logging Service
   Loaded: loaded (/usr/lib/systemd/system/rsyslog.service; enabled; vendor preset: enabled)
   Active: active (running) since Wed 2021-12-22 13:40:11 CET; 3h 5min ago
     Docs: man:rsyslogd(8)
           http://www.rsyslog.com/doc/
 Main PID: 108600 (rsyslogd)
   CGroup: /system.slice/rsyslog.service
           └─108600 /usr/sbin/rsyslogd -n

 

4. Property-based compare-operations supported by rsyslog table
 

Compare-operation Description
contains Checks whether the provided string matches any part of the text provided by the property. To perform case-insensitive comparisons, use  contains_i .
isequal Compares the provided string against all of the text provided by the property. These two values must be exactly equal to match.
startswith Checks whether the provided string is found exactly at the beginning of the text provided by the property. To perform case-insensitive comparisons, use  startswith_i .
regex Compares the provided POSIX BRE (Basic Regular Expression) against the text provided by the property.
ereregex Compares the provided POSIX ERE (Extended Regular Expression) regular expression against the text provided by the property.
isempty Checks if the property is empty. The value is discarded. This is especially useful when working with normalized data, where some fields may be populated based on normalization result.

 


5. Rsyslog understanding Facility levels

Here is a list of facility levels that can be used.

Note: The mapping between Facility Number and Keyword is not uniform over different operating systems and different syslog implementations, so among separate Linuxes there might be diference in the naming and numbering.

Facility Number Keyword Facility Description
0 kern kernel messages
1 user user-level messages
2 mail mail system
3 daemon system daemons
4 auth security/authorization messages
5 syslog messages generated internally by syslogd
6 lpr line printer subsystem
7 news network news subsystem
8 uucp UUCP subsystem
9   clock daemon
10 authpriv security/authorization messages
11 ftp FTP daemon
12 – NTP subsystem
13 – log audit
14 – log alert
15 cron clock daemon
16 local0 local use 0 (local0)
17 local1 local use 1 (local1)
18 local2 local use 2 (local2)
19 local3 local use 3 (local3)
20 local4 local use 4 (local4)
21 local5 local use 5 (local5)
22 local6 local use 6 (local6)
23 local7 local use 7 (local7)


6. rsyslog Severity levels (sublevels) accepted by facility level

As defined in RFC 5424, there are eight severity levels as of year 2021:

Code Severity Keyword Description General Description
0 Emergency emerg (panic) System is unusable. A "panic" condition usually affecting multiple apps/servers/sites. At this level it would usually notify all tech staff on call.
1 Alert alert Action must be taken immediately. Should be corrected immediately, therefore notify staff who can fix the problem. An example would be the loss of a primary ISP connection.
2 Critical crit Critical conditions. Should be corrected immediately, but indicates failure in a primary system, an example is a loss of a backup ISP connection.
3 Error err (error) Error conditions. Non-urgent failures, these should be relayed to developers or admins; each item must be resolved within a given time.
4 Warning warning (warn) Warning conditions. Warning messages, not an error, but indication that an error will occur if action is not taken, e.g. file system 85% full – each item must be resolved within a given time.
5 Notice notice Normal but significant condition. Events that are unusual but not error conditions – might be summarized in an email to developers or admins to spot potential problems – no immediate action required.
6 Informational info Informational messages. Normal operational messages – may be harvested for reporting, measuring throughput, etc. – no action required.
7 Debug debug Debug-level messages. Info useful to developers for debugging the application, not useful during operations.


7. Sample well tuned configuration using severity and facility levels and immark, imuxsock, impstats
 

Below is sample config using severity and facility levels
 

# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none;local0.none;local1.none        /var/log/messages


Note the local0.none; local1.none tells rsyslog to not log from that facility level to /var/log/messages.

If you need a complete set of rsyslog configuration fine tuned to have a proper logging with increased queues and included configuration for loggint to remote log aggegator service as well as other measures to prevent the system disk from being filled in case if something goes wild with a logging service leading to a repeatedly messages you might always contact me and I can help 🙂
 Other from that sysadmins might benefit from a sample set of configuration prepared with the Automated rsyslog config builder  or use some fine tuned config  for rsyslog-8.24.0-57.el7_9.1.x86_64 on Redhat 7.9 (Maipo)   rsyslog_config_redhat-2021.tar.gz.

To sum it up rsyslog though looks simple and not an important thing to pre

Tags: , , , , , , , , , , , , , ,
Posted in Linux, Remote System Administration, System Administration | No Comments »

CentOS disable SELinux permanently or one time on grub Linux kernel boot time

Saturday, July 24th, 2021

selinux-artistic-penguin-logo-protect-data

 

1. Office 365 cloud connected computer and a VirtualBox hosted machine with SELINUX preventing it to boot

At my job we're in process of migrating my old Lenovo Laptop Thinkpad model L560 Laptop to Dell Latitude 5510 wiith Intel Core i5 vPro CPU and 256 Gb SSD Hard Drive.  The new laptops are generally fiine though they're not even a middle class computers and generally I prefer thinkpads. The sad thing out of this is our employee decided to migrate to Office 365 (again perhaps another stupid managerial decision out of an excel sheet wtih a balance to save some money … 

As you can imagine Office 365 is not really PCI Standards compliant and not secure since our data is stored in Microsoft cloud and theoretically Microsoft has and owns our data or could wipe loose the data if they want to. The other obvious security downside I've noticed with the new "Secure PCI complaint laptop" is the initial PC login screen which by default offers fingerprint authentication or the even worse  and even less secure face recognition, but obviosly everyhing becomes more and more crazy and people become less and less cautious for security if that would save money or centralize the data … In the name of security we completely waste security that is very dubious paradox I don't really understand but anyways, enough rant back to the main topic of this article is how to and I had to disable selinux?

As part of Migration I've used Microsoft OneDrive to copy old files from the Thinkpad to the Latitude (as on the old machine USB's are forbidden and I cannot copy over wiith a siimple USB driive, as well as II have no right to open the laptop and copy data from the Hard driive, and even if we had this right without breaking up some crazy company policy that will not be possible as the hard drive data on old laptop is encrypted, the funny thing is that the new laptop data comes encrypted and there is no something out of the box as BitDefender or McAffee incryption (once again, obviously our data security is a victim of some managarial decisions) …
 

2. OneDrive copy problems unable to sync some of the copied files to Onedrive


Anyways as the Old Laptop's security is quite paranoid and we're like Fort Nox, only port 80 and port 443 connections to the internet can be initiated to get around this harsh restrictions it was as simple to use a Virtualbox Virtual Machine. So on old laptop I've installed a CentOS 7 image which I used so far and I used one drive to copy my vbox .vdi image on the new laptop work machine.

The first head buml was the .vdi which seems to be prohibited to be copied to OneDrive, so to work around this I had to rename the origianl CentOS7.vdi to CentOS7.vdi-renamed on old laptop and once the data is in one drive copy my Vitualbox VM/ directory from one drive to the Dell Latitude machine and rename the .vdi-named towards .vdi as well as import it from the latest installed VirtualBox on the new machine.
 

3. Disable SELINUX from initial grub boot


So far so good but as usual happens with miigrations I've struck towards another blocker, the VM image once initiated to boot from Virtualbox badly crashed with some complains that selinux cannot be loaded.
Realizing CentOS 7 has the more or less meaningless Selinux, I've took the opportunity to disable SeLinux.

To do so I've booted the Kernel with Selinux disabled from GRUB2 loader prompt before Kernel and OS Userland boots.

 

 

I thought I need to type the information on the source in grub. What I did is very simple, on the Linux GRUB boot screen I've pressed

'e' keyboard letter

that brought the grub boot loader into edit mode.

Then I had to add selinux=0 on the edited selected kernel version, as shown in below screenshot:

selinux-disable-from-grub.png

Next to boot the Linux VM without Selinux enabled one time,  just had to press together

Ctrl+X then add selinux=0 on the edited selected kernel version, that should be added as shown in the screenshot somewhere after the line of
root=/dev/mapper/….. 

4. Permanently Disable Selinux on CentOS 7


Once I managed to boot Virtual Machine properly with Oracle Virtualbox, to permanently disabled selinux I had to:

 

Once booted into CentOS, to check the status of selinux run:

 

# sestatus
Copy
SELinux status:                 enabled
SELinuxfs mount:                /sys/fs/selinux
SELinux root directory:         /etc/selinux
Loaded policy name:             targeted
Current mode:                   enforcing
Mode from config file:          enforcing
Policy MLS status:              enabled
Policy deny_unknown status:     allowed
Max kernel policy version:      31

 

5. Disable SELinux one time with setenforce command


You can temporarily change the SELinux mode from targeted to permissive with the following command:

 

# setenforce 0


Next o permanently disable SELinux on your CentOS 7 next time the system boots, Open the /etc/selinux/config file and set the SELINUX mod parameter to disabled.

On CentOS 7 you can  edit the kernel parameters in /etc/default/grub (in the GRUB_CMDLINE_LINUX= key) and set selinux=0 so on next VM / PC boot we boot with a SELINUX disabled for example add   RUB_CMDLINE_LINUX=selinux=0 to the file then you have to regenerate your Grub config like this:
 

# grub2-mkconfig -o /etc/grub2.cfg
# grub2-mkconfig -o /etc/grub2-efi.cfg


Further on to disable SeLinux on OS level edit /etc/selinux
 

Default /etc/selinux/config with selinux enabled should look like so:

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#       enforcing – SELinux security policy is enforced.
#       permissive – SELinux prints warnings instead of enforcing.
#       disabled – No SELinux policy is loaded.
SELINUX=enforcing
# SELINUXTYPE= can take one of these two values:
#       targeted – Targeted processes are protected,
#       mls – Multi Level Security protection.
SELINUXTYPE=targeted


To disable SeLinux modify the file to be something like:

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#       enforcing – SELinux security policy is enforced.
#       permissive – SELinux prints warnings instead of enforcing.
#       disabled – No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of these two values:
#       targeted – Targeted processes are protected,
#       mls – Multi Level Security protection.
SELINUXTYPE=targeted

6. Check SELINUX status is disabled

# sestatus

SELinux status:                 disabled

So in this article shottly was explained shortly the fake security adopted by using Microsoft Cloud environment Offiice 365, my faced OneDrive copy issues (which prevented even my old laptop Virtual Machine to boot properly and the handy trick to rename the file that is unwilling to get copied from old PC towards m$ OneDrive as well as the grub trick to disable Selinux permanently from grub2.

Tags: , , , , , , , , ,
Posted in Linux, Security, System Administration | No Comments »

Update reverse sshd config with cronjob to revert if sshd reload issues

Friday, February 12th, 2021

Update-reverse-sshd-config-with-cronjob-to-revert-if-sshd-reload-issues

Say you're doing ssh hardening modifying /etc/ssh/sshd_config for better system security or just changing options in sshd due to some requirements. But you follow the wrong guide and you placed some ssh variable which is working normally on newer SSH versions ssh OpenSSH_8.0p1 / or 7 but the options are applied on older SSH server and due to that restarting sshd via /etc/init.d/… or systemctl restart sshd cuts your access to remote server located in a DC and not attached to Admin LAN port, and does not have a working ILO or IDRAC configured and you have to wait for a couple of hours for some Support to go to the server Room / Rack / line location to have access to a Linux physical tty console and fix it by reverting the last changes you made to sshd and restarting.

Thus logical question comes what can you do to assure yourself you would not cut your network access to remote machine after modifying OpenSSHD and normal SSHD restart?

There is an old trick, I'm using for years now but perhaps if you're just starting with Linux as a novice system administrator or a server support guy you would not know it, it is as simple as setting a cron job for some minutes to periodically overwrite the sshd configuration with a copy of the old working version of sshd before modification.

Here is this nice nify trick which saved me headache of call on technical support line to ValueWeb when I was administering some old Linux servers back in the 2000s

root@server:~# crontab -u root -e

# create /etc/ssh/sshd_config backup file
cp -rpf /etc/ssh/sshd_config /etc/ssh/sshd_config_$(date +%d-%m-%y)
# add to cronjob to execute every 15 minutes and ovewrite sshd with the working version just in case
*/15 * * * * /bin/cp -rpf /etc/ssh/sshd_config_$(date +%d-%m-%y) /etc/ssh/sshd_config && /bin/systemctl restart sshd
# restart sshd 
cp -rpf /etc/ssh/sshd_config_$(date +%d-%m-%y) /etc/ssh/sshd_config && /bin/systemctl restart sshd


Copy paste above cron definitions and leave them on for some time. Do the /etc/ssh/sshd_config modifications and once you're done restart sshd by lets say

root@server:~#  killall -HUP sshd 


If the ssh connectivity continues to work edit the cron job again and delete all lines and save again.
If you're not feeling confortable with vim as a text editor (in case you're a complete newbie and you don't know) how to get out of vim. Before doing all little steps you can do on the shell with  export EDITOR=nano or export EDITOR=mcedit cmds,this will change the default text editor on the shell. 

Hope this helps someone… Enjoy 🙂

Tags: , , , , , , , , , , ,
Posted in Linux, Networking, Remote System Administration, System Administration, Various | 2 Comments »

Postfix copy every email to a central mailbox (send a copy of every mail sent via mail server to a given email)

Wednesday, October 28th, 2020

Postfix-logo-always-bcc-email-option-send-all-emails-to-a-single-address-with-postfix.svg

Say you need to do a mail server migration, where you have a local configured Postfix on a number of Linux hosts named:

Linux-host1
Linux-host2
Linux-host3

etc.


all configured to send email via old Email send host (MailServerHostOld.com) in each linux box's postfix configuration's /etc/postfix/main.cf.
Now due to some infrastructure change in the topology of network or anything else, you need to relay Mails sent via another asumably properly configured Linux host relay (MailServerNewHost.com).

Usually such a migrations has always a risk that some of the old sent emails originating from local running scripts on Linux-host1, Linux-Host2 … or some application or anything else set to send via them might not properly deliver emails to some external Internet based Mailboxes via the new relayhost MailServerNewHost.com.

E.g. in /etc/postfix/main.cf Linux-Host* machines, you have below config after the migration:

relayhost = [MailServerNewHost.com]

Lets say that you want to make sure, that you don't end up with lost emails as you can't be sure whether the new email server will deliver correctly to the old repicient emails. What to do then?

To make sure will not end up in undelivered state and get lost forever after a week or so (depending on the mail queue configuration retention period made on Linux sent MTAs and mailrelay MailServerNewHost.com, it is a very good approach to temprorary set all email communication that will be sent via MailServerNewHost.com a BCC emaills (A Blind Carbon Copy) of each sent mail via relay that is set on your local configured Postfix-es on Linux-Host*.

In postfix to achieve that it is very easy all you have to do is set on your MailServerNewHost.com a postfix config variable always_bcc smartly included by postfix Mail Transfer Agent developers for cases exactly like this.

To forward all passed emails via the mail server just place in the end of /etc/postfix/mail.conf after login via ssh on MailServerNewHost.com

always_bcc=All-Emails@your-diresired-redirect-email-address.com


Now all left is to reload the postfix to force the new configuration to get loaded on systemd based hosts as it is usually today do:

# systemctl reload postfix


Finally to make sure all works as expected and mail is sent do from do a testing via local MTAs. 
 

Linux-Host:~# echo -e "Testing body" | mail -s "testing subject" -r "testing@test.com" georgi.stoyanov@remote-user-email-whatever-address.com

Linux-Host:~# echo -e "Testing body" | mail -s "testing subject" -r "testing@test.com" georgi.stoyanov@sample-destination-address.com


As you can see I'm using the -r to simulate a sender address, this is a feature of mailx and is not available on older Linux Os hosts that are bundled with mail only command.
Now go to and open the All-Emails@your-diresired-redirect-email-address.com in Outlook (if it is M$ Office 365 MX Shared mailbox), Thunderbird or whatever email fetching software that supports POP3 or IMAP (in case if you configured the common all email mailbox to be on some other Postfix / Sendmail / Qmail MTA). and check whether you started receiving a lot of emails 🙂

That's all folks enjoy ! 🙂

Tags: , , , , , , , , , , , , , , , , , , ,
Posted in Linux, Postfix, System Administration | No Comments »

Linux: Howto Disable logging for all VirtualHosts on Apache and NGINX Webservers one liner

Wednesday, July 1st, 2020

disable-apache-nginx-logging-for-all-virtualhosts
Did you happen to administer Apache Webservers or NGINX webservers whose logs start to grow so rapidly that are flooding the disk too quickly?
Well this happens sometimes and it also happens that sometimes you just want to stop logging especially, to offload disk writting.

There is an easy way to disable logging for requests and errors (access_log and error_log usually residing under /var/log/httpd or /var/log/nginx ) for  all configured Virtual Domains with a short one liner, here is how.

Before you start  Create backup of /etc/apache2/sites-enabled / or /etc/nginx to be able to revert back to original config.

# cp -rpf /etc/apache2/sites-enabled/ ~/

# cp -rpf /etc/nginx/ ~/


1. Disable Logging for All  Virtual Domains configured for Apache Webserver

First lets print what the command will do to make sure we don't mess something

# find /home/hipo/sites-enabled/* -exec echo sed -i 's/#*[Cc]ustom[Ll]og/#CustomLog/g' {} \;


You will get some output like

find /home/hipo//sites-enabled/* -exec echo sed -i 's/#*[Cc]ustom[Ll]og/#CustomLog/g' {} \;

find /etc/apache2/sites-enabled/* -exec sed -i 's/#*[Cc]ustom[Ll]og/#CustomLog/g' {} \;
find /etc/apache2/sites-enabled/* -exec sed -i 's/#*[Ee]rror[Ll]og/#ErrorLog/g' {} \;

2. Disable Logging for All configured Virtual Domains for NGINX Webserver
 

find /etc/nginx/sites-enabled/* -exec sed -i 's/#*access_log/#access_log/g' {} \;
find /etc/nginx/sites-enabled/* -exec sed -i 's/#*error_log/#error_log/g' {} \;

f course above substituations that will comment out with '#' occurances from file configs of only default set access_log and error_log / access.log, error.log 
for machines where there is no certain convention on file naming and there are multiple domains in custom produced named log files this won't work.

This one liner was inspired from a friend's daily Martin Petrov. Martin blogged initially about this nice tip for those reading Cyrillic check out mpetrov.net, so. Thanks Marto ! 🙂

Tags: , , , , , , , , , , , , , , , , , , ,
Posted in Everyday Life, Nginx, Web and CMS | No Comments »

Nginx increase security by putting websites into Linux jails howto

Monday, August 27th, 2018

linux-jail-nginx-webserver-increase-security-by-putting-it-and-its-data-into-jail-environment

If you're sysadmining a large numbers of shared hosted websites which use Nginx Webserver to interpret PHP scripts and serve HTML, Javascript, CSS … whatever data.

You realize the high amount of risk that comes with a possible successful security breach / hack into a server by a malicious cracker. Compromising Nginx Webserver by an intruder automatically would mean that not only all users web data will get compromised, but the attacker would get an immediate access to other data such as Email or SQL (if the server is running multiple services).

Nowadays it is not so common thing to have a multiple shared websites on the same server together with other services, but historically there are many legacy servers / webservers left which host some 50 or 100+ websites.

Of course the best thing to do is to isolate each and every website into a separate Virtual Container however as this is a lot of work and small and mid-sized companies refuse to spend money on mostly anything this might be not an option for you.

Considering that this might be your case and you're running Nginx either as a Load Balancing, Reverse Proxy server etc. , even though Nginx is considered to be among the most secure webservers out there, there is absolutely no gurantee it would not get hacked and the server wouldn't get rooted by a script kiddie freak that just got in darknet some 0day exploit.

To minimize the impact of a possible Webserver hack it is a good idea to place all websites into Linux Jails.

linux-jail-simple-explained-diagram-chroot-jail

For those who hear about Linux Jail for a first time,
chroot() jail is a way to isolate a process / processes and its forked children from the rest of the *nix system. It should / could be used only for UNIX processes that aren't running as root (administrator user), because of the fact the superuser could break out (escape) the jail pretty easily.

Jailing processes is a concept that is pretty old that was first time introduced in UNIX version 7 back in the distant year 1979, and it was first implemented into BSD Operating System ver. 4.2 by Bill Joy (a notorious computer scientist and co-founder of Sun Microsystems). Its original use for the creation of so called HoneyPot – a computer security mechanism set to detect, deflect, or, in some manner, counteract attempts at unauthorized use of information systems that appears completely legimit service or part of website whose only goal is to track, isolate, and monitor intruders, a very similar to police string operations (baiting) of the suspect. It is pretty much like а bait set to collect the fish (which in this  case is the possible cracker).

linux-chroot-jail-environment-explained-jailing-hackers-and-intruders-unix

BSD Jails nowadays became very popular as iPhones environment where applications are deployed are inside a customly created chroot jail, the principle is exactly the same as in Linux.

But anyways enough talk, let's create a new jail and deploy set of system binaries for our Nginx installation, here is the things you will need:

1. You need to have set a directory where a copy of /bin/ls /bin/bash /bin/,  /bin/cat … /usr/bin binaries /lib and other base system Linux system binaries copy will reside.

 

server:~# mkdir -p /usr/local/chroot/nginx

 


2. You need to create the isolated environment backbone structure /etc/ , /dev, /var/, /usr/, /lib64/ (in case if deploying on 64 bit architecture Operating System).

 

server:~# export DIR_N=/usr/local/chroot/nginx;
server:~# mkdir -p $DIR_N/etc
server:~# mkdir -p $DIR_N/dev
server:~# mkdir -p $DIR_N/var
server:~# mkdir -p $DIR_N/usr
server:~# mkdir -p $DIR_N/usr/local/nginx
server:~# mkdir -p $DIR_N/tmp
server:~# chmod 1777 $DIR_N/tmp
server:~# mkdir -p $DIR_N/var/tmp
server:~# chmod 1777 $DIR_N/var/tmp
server:~# mkdir -p $DIR_N/lib64
server:~# mkdir -p $DIR_N/usr/local/

 

3. Create required device files for the new chroot environment

 

server:~# /bin/mknod -m 0666 $D/dev/null c 1 3
server:~# /bin/mknod -m 0666 $D/dev/random c 1 8
server:~# /bin/mknod -m 0444 $D/dev/urandom c 1 9

 

mknod COMMAND is used instead of the usual /bin/touch command to create block or character special files.

Once create the permissions of /usr/local/chroot/nginx/{dev/null, dev/random, dev/urandom} have to be look like so:

 

server:~# ls -l /usr/local/chroot/nginx/dev/{null,random,urandom}
crw-rw-rw- 1 root root 1, 3 Aug 17 09:13 /dev/null
crw-rw-rw- 1 root root 1, 8 Aug 17 09:13 /dev/random
crw-rw-rw- 1 root root 1, 9 Aug 17 09:13 /dev/urandom

 

4. Install nginx files into the chroot directory (copy all files of current nginx installation into the jail)
 

If your NGINX webserver installation was installed from source to keep it latest
and is installed in lets say, directory location /usr/local/nginx you have to copy /usr/local/nginx to /usr/local/chroot/nginx/usr/local/nginx, i.e:

 

server:~# /bin/cp -varf /usr/local/nginx/* /usr/local/chroot/nginx/usr/local/nginx

 


5. Copy necessery Linux system libraries to newly created jail
 

NGINX webserver is compiled to depend on various libraries from Linux system root e.g. /lib/* and /lib64/* therefore in order to the server work inside the chroot-ed environment you need to transfer this libraries to the jail folder /usr/local/chroot/nginx

If you are curious to find out which libraries exactly is nginx binary dependent on run:

server:~# ldd /usr/local/nginx/usr/local/nginx/sbin/nginx

        linux-vdso.so.1 (0x00007ffe3e952000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f2b4762c000)
        libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007f2b473f4000)
        libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007f2b47181000)
        libcrypto.so.0.9.8 => /usr/local/lib/libcrypto.so.0.9.8 (0x00007f2b46ddf000)
        libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f2b46bc5000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f2b46826000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f2b47849000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f2b46622000)


The best way is to copy only the libraries in the list from ldd command for best security, like so:

 

server: ~# cp -rpf /lib/x86_64-linux-gnu/libthread.so.0 /usr/local/chroot/nginx/lib/*
server: ~# cp -rpf library chroot_location
…
etc.

 

However if you're in a hurry (not a recommended practice) and you don't care for maximum security anyways (you don't worry the jail could be exploited from some of the many lib files not used by nginx and you don't  about HDD space), you can also copy whole /lib into the jail, like so:

 

server: ~# cp -rpf /lib/ /usr/local/chroot/nginx/usr/local/nginx/lib

 

NOTE! Once again copy whole /lib directory is a very bad practice but for a time pushing activities sometimes you can do it …


6. Copy /etc/ some base files and ld.so.conf.d , prelink.conf.d directories to jail environment
 

 

server:~# cp -rfv /etc/{group,prelink.cache,services,adjtime,shells,gshadow,shadow,hosts.deny,localtime,nsswitch.conf,nscd.conf,prelink.conf,protocols,hosts,passwd,ld.so.cache,ld.so.conf,resolv.conf,host.conf}  \
/usr/local/chroot/nginx/usr/local/nginx/etc

 

server:~# cp -avr /etc/{ld.so.conf.d,prelink.conf.d} /usr/local/chroot/nginx/nginx/etc


7. Copy HTML, CSS, Javascript websites data from the root directory to the chrooted nginx environment

 

server:~# nice -n 10 cp -rpf /usr/local/websites/ /usr/local/chroot/nginx/usr/local/


This could be really long if the websites are multiple gigabytes and million of files, but anyways the nice command should reduce a little bit the load on the server it is best practice to set some kind of temporary server maintenance page to show on the websites index in order to prevent the accessing server clients to not have interrupts (that's especially the case on older 7200 / 7400 RPM non-SSD HDDs.)
 

 

8. Stop old Nginx server outside of Chroot environment and start the new one inside the jail


a) Stop old nginx server

Either stop the old nginx using it start / stop / restart script inside /etc/init.d/nginx (if you have such installed) or directly kill the running webserver with:

 

server:~# killall -9 nginx

 

b) Test the chrooted nginx installation is correct and ready to run inside the chroot environment

 

server:~# /usr/sbin/chroot /usr/local/chroot/nginx /usr/local/nginx/nginx/sbin/nginx -t
server:~# /usr/sbin/chroot /usr/local/chroot/nginx /usr/local/nginx/nginx/sbin/nginx

 

c) Restart the chrooted nginx webserver – when necessery later

 

server:~# /usr/sbin/chroot /nginx /usr/local/chroot/nginx/sbin/nginx -s reload

 

d) Edit the chrooted nginx conf

If you need to edit nginx configuration, be aware that the chrooted NGINX will read its configuration from /usr/local/chroot/nginx/nginx/etc/conf/nginx.conf (i'm saying that if you by mistake forget and try to edit the old config that is usually under /usr/local/nginx/conf/nginx.conf

 

 

Tags: , , , , , , , , , , , , , , , ,
Posted in Computer Security, Linux, Nginx, System Administration | No Comments »

How to enable Gravis UltraSound in DOSBox for enhanced music experience in DOS programs and Games

Tuesday, October 31st, 2017

DOSBox

Gravis UltraSound Classic

 

Gravis UltraSound

Gravis UltraSound or GUS is a sound card for the IBM PC compatible systems.
It was lunched in 1992 and is notable for it's ability to use real-world sound recordings (wavetable) of a musical instruments rather than artificial computer-generated waveforms.
As one of my friends used to say back then: "it sounds like a CD".

To enable GUS in DOSBox all you need to do is:

1. Download the archive with the GUS files from https://alex.www.pc-freak.net/files/GUS/ULTRASND.zip. Extract the archive (there is already a directory in it so you don't have to create one) preferably where you keep your DOSBox stuff (like Games).

2. Find your DOSBox config file. Depending on the version or host OS, the dosbox conf file is located either inside the user profile folder or inside the same folder as dosbox.exe. In Windows 7 the config file is located at

"C:\Users\Fred\AppData\Local\VirtualStore\Program Files (x86)\dosbox.conf"

where "Fred" is your username.

In GNU/Linux it's in "/home/Fred/.dosbox/dosbox.conf" where "Fred" is your username.

The name of the conf file may also have dosbox version (for example –

"dosbox-0.74.conf").

Open it with a text editor like notepad (Windows) or equvalent for GNU/Linux (vi, Kate, gedit…). Locate "[gus]" section (without the quotes) and edit it so it looks like this:

[gus]
#      gus: Enable the Gravis Ultrasound emulation.
#  gusrate: Sample rate of Ultrasound emulation.
#           Possible values: 44100, 48000, 32000, 22050, 16000, 11025, 8000, 49716.
#  gusbase: The IO base address of the Gravis Ultrasound.
#           Possible values: 240, 220, 260, 280, 2a0, 2c0, 2e0, 300.
#   gusirq: The IRQ number of the Gravis Ultrasound.
#           Possible values: 5, 3, 7, 9, 10, 11, 12.
#   gusdma: The DMA channel of the Gravis Ultrasound.
#           Possible values: 3, 0, 1, 5, 6, 7.
# ultradir: Path to Ultrasound directory. In this directory
#           there should be a MIDI directory that contains
#           the patch files for GUS playback. Patch sets used
#           with Timidity should work fine.

gus=true
gusrate=44100
gusbase=240
gusirq=5
gusdma=3
ultradir=C:\ULTRASND

Then save the dosbox conf file.

3. Start DOSBox and mount "ULTRASND" directory to "C:".

You can do that with

mount c (directory to ULTRASND)

For example if you have extracted the archive in "C:\Games" it has created "C:\Games\ULTRASND" and the command you will have to write in DOSBox is

mount c c:\Games

(example: if your game is in "C:\Games\Heroes2" and your GUS directory is "C:\Games\ULTRSND" (if you have extracted the archive "C:\Games\") then you "mount c c:\Games" and you are set)

or for GNU/Linux if you have extracted the archive in "/home/Fred/Games" it has created "/home/Fred/Games/ULTRASND" and the command you will have to write in DOSBox is

mount c /home/Fred/Games (where "Fred" is your user name).

(example: if your game is in "/home/Fred/Games/Heroes2" and your GUS directory is "/home/Fred/Games/ULTRSND" (if you have extracted the archive" /home/Fred/Games/") in  then you "mount c /home/Fred/Games" and you are set)

You can make this automatic so you don't have to write it everytime by adding this command in the end (bottom) part of your dosbox conf file and save it.

You're practicly ready. All you need to do now is set Gravis UltraSound in your game or application setup (for example with the file "setup.exe") with IO: 240, IRQ 5 and DMA 3. If you prefer you previous sound card you can do that by selecting it again from the setup without disabling GUS from the dosbox conf file.

Happy listening!

Gravis Ultrasound

 

Article written by Alex

Tags: , , , , , , , , ,
Posted in Entertainment, Games, Games Linux, Linux, Linux Audio & Video, System Administration, Windows | 4 Comments »

Fix dosbox – setting: cycles=auto. If the game runs too fast try a fixed cycles amount in DOSBox’s options. Exit to error: DRC64:Unhandled memory reference

Thursday, October 12th, 2017

fix-dosbox-error-setting-cycles-auto-game-runs-too-fast-exit-err-drc64-unhandled-memory-reference

I've recently installed Debian GNU / Linux 9.1 Stretch and on Ubuntu 16.10 Yakkety Yak and as an old school true gamer addict decided to play some games to recall the good old memories from the past of the glorious DOS games times, when all was simple people was real and there was no terrible people dividers such as Facebook  / GooglePlus, no skype fuzzy craziness and no people watching all time at their mobile smart phones like zoombies .

Well anyways, so I wanted to play my favourite DOS games Terminal Velocity, StarGunner, Raptor, Heroes of Might and Magic 2, Doom II, Duke 3d etc. etc. the list goes on and on – BTW What's your favourite old school DOS Game??

So I issues in Terminal:

$ dosbox TV.EXE

DOSBox version 0.74
Copyright 2002-2010 DOSBox Team, published under GNU GPL.
—
CONFIG:Loading primary settings from config file /home/zetathon/.dosbox/dosbox-0.74.conf
MIXER:Got different values from SDL: freq 44100, blocksize 512
ALSA:Can't subscribe to MIDI port (65:0) nor (17:0)
MIDI:Opened device:none
DOSBox switched to max cycles, because of the setting: cycles=auto. If    the game runs too fast try a fixed cycles amount in DOSBox's options.
Exit to error: DRC64:Unhandled memory reference

 

So what is the solution to  Exit to error: DRC64:Unhandled memory reference
error and why it happens?

Well, as the error clearly states the whole problem
comes from a default dosbox dos emulator configuration, that's the
setting:
 

cycles=auto


Perhaps the CPU beat of a Core I7 CPUs is too strong for some of the old games which are expecting a slower PC with less memory that's why the exception

Exit to Error: DRC64:Unhandled memory reference
is spit out by Dosemu.

So how to solve DRC64:Unhandled memory reference dosemu error?

By default DOSBOX places its configuration at  Linux / BSD / *nix-es at home folder /dosbox – e.g. ~/.dosbox/ the naming of dosbox.conf might differ according to dosbox version and the respective Linux / BSD distro, on Debian and Ubuntu Linux usually the naming includes a version number.

On Debian 9 Strecth Linux dosbox config is at ~/.dosbox/dosbox-0.74.conf

Edit the file
 

$ vim ~/.dosbox/dosbox-0.74.conf


and change inside the value for:

 

 

 

core=auto

to

core=normal

Then give dosbox with the erroring game another try, it worked for me
for me and everything seemed to run fine.

I've only run a handful of games but so far it seems to be working well.

If it does work but performance is degraded and tooks a huch hit on Gameplay you better try to substitute
 

core=normal

to
 

core=simple


AND HOORAY IT WORKS AGAIN! THE GAME RUN SUCCESSFULLY JOY TO THE WORLD! 🙂

Tags: , , , , , , , , , , , , ,
Posted in Games Linux, Linux and FreeBSD Desktop | No Comments »

« Older Entries