System Administration Archives - ☩ Walking in Light with Christ - Faith, Computing, Diary ☩ Walking in Light with Christ

Archive for the ‘System Administration’ Category

Zabbix Power Shell PS1 script to write zero or one if string is matched inside log file

Monday, December 2nd, 2024

At work we had setup zabbix log file processing for few servers for a service that is doing a Monitoring Health Checks for a a special application via an encrypted strong encrypted tunnel. The app based on the check reports whether the remote side has processed data or not.
As me and my team are not maintainers of the zabbix-server where the zabbix-agents are sending the data, there is a multiple content of data being sent in simply "" empty strings via a zabbix Item setup. Those empty strings however gets stored in the zabbix-server database and since this check is made frequently about 500 hundred records of empty string lines are being written to the zabbix server, we got complaint by the zabbix adminsitrators, that we have to correct our Monitoring setup to not flood the zabbix-server.

Since zabbix cannot catch up the "" empty string and we cannot supress the string from being written in the Item, we needed a way to change the monitoring so that the configured Application check returns 1 (on error) and 0 (on success).

Zabbix even though advanced has a strange when zabbix log[] function, e.g.

log[/path/to/log,,,,skip]

log function, used to analyze a log file and cut out last or first lines of a file simmilar to UNIX's head and tail over log files this is described in the zabbix log file monitoring here . If a string is matched it can return string 1, but if nothing gets matched the result is empty string "" and this empty string cannot be used in a way to analyze the data with Item is used.

There is plenty of discussions online for this weird behavior and many people do offer different approaches to solve the strange situation, but as we have tried with our colleagues sys admins none of those really worked out.

Thus we decided to use the classical way to work around, e.g. to simply use a powershell script that would check a number of lines inside a provided log file analyze if a string gets found and print out value of "1" if the string is matched or "0" "if not and this PS1 script to be set to run via a standard zabbix userparameter script.

This worked well, as all of us are mainly managing Linux systems, and we don't have enough knowledge on powershell we have used our internal Aartificial Intelligence (AI) clone tool to LibreChat – A free and open source ChatGPT clone.

LibreChat includes OpenAI's models, but also others — both open-source and closed-source — and its website promises "seamless integration" with AI services from OpenAI, Azure, Anthropic, and Google — as well as GPT-4, Gemini Vision, and many others. ("Every AI in one place," explains LibreChat's home page.) Plugins even let you make requests to DALL-E or Stable Diffusion for image generations. (LibreChat also offers a database that tracks "conversation state" — making it possible to switch to a different AI model in mid-conversation…)

$logfile = "C:\path\to\your\logfile.log"
$searchString = "-1"

# Get the last 140 lines
$lines = Get-Content $logfile -Tail 140

# Filter lines containing the search string
$found = $lines | Where-Object { $_ -match [regex]::Escape($searchString) }

# Output found lines or 0 if none were found
if ($found) {
$found | ForEach-Object { $_ }
} else {
Write-Host 0
}

You can download and the return_zero_or_one-if-string-matches-in-log-powershell.ps1 script here

Tags: data, empty, file, found, generations, inside, Item, last, log, log files, make, Monitoring Health Checks, none, page, possible, requests, search string, simply, switch, tail, tool, zabbix
Posted in Educational, System Administration, Various, Windows | No Comments »

How to split large files in Windows via split command line and File Archive GUI tool easily

Tuesday, October 22nd, 2024

Moving around a very large files especially Virtualbox Virtual Machines or other VM formats between Windows host and OneDrive might be a problem due to either Azure Cloud configured limitations, or other reasons that your company Domain Administrator has configured, thus if you have to migrate your old Hardware Laptop PC Windows 10 to a newer faster better Harware / Better Performance Notebook Computer with Windows 11 and you still want to keep and move your old large files in this short and trivial article, will explain how.

The topic is easily and most of novice sysadmins should have already be faced to bump into something like this but anyways i found useful to mention about Git for Windows, as it is really useful too thus wrote this small article.

The moved huge files, in my case an experimental Virtual Machines Images which I needed to somehow migrate on the new Freshly installed Windows laptop, the Large files were 40 / 80 etc. Gigabytes or whatever large amount of files from your PC to the Cloud Onedrive and of course the most straight forward thing i tried was to simply add the file for inclusion into the Onedrive storage (via OneDrive tool setup interface), however this file, failed due to OneDrive Cloud file format security limitations or Antivirus solutions configured to filter out the large file copying or even a prohibition to be able to include any kind of Virtual Machines ISOs straight into the cloud.

With this big files comes the question:

How to copy the Virtual Machines from your Old Hardware Laptop to the Cloud (without being able to use an external SSD Hard Drive or a USB SSD Flash drive, due to Domain policy configured for your windows to be unable to copy to externally connected Drive but only to read from such.) ?

Here are few sample approaches to do it both from command line (useful if you have to repeat the process or script it and deploy to multiple hosts) or for single hosts via an Archiver tool:

1. Using split command Git for Windows (Bash) MINGW64 shell

Download Git for Windows – https://git-scm.com/download install it and you will get the MINGW64 bash for Windows executable.

Run it either invoke bash command from command line or trigger Windows Run command prompt (Windows button + R) and type full path to executable

C:\Program Files\Git\git-bash.exe

Git-for-Windows-bash-for-windows-MINGW64-windows-11-screenshot

Use the integrated program split and to cut it into pieces use:

# split MyVeryLargeFileVM.vdi -b 800m

To split the .VDI virtualbox file to lets say 5 Gigabite pieces:

# split MyVeryLargeFile.vdi -b 5g

The output files will be named pieces will be named as in a normal UNIX / GNU split command in the format and each piece of 5GB will be named like:

xaa
xab
xac
xad

If you want to get a more meaningful name for the spilitted files you can set a generated split file prefix with suffixes to be 5 digits long:

# split MyVeryLargeFile.vdi MyVeryLargeFileVM-parts_ -b 5g -d -a 5

the -d flag for using numerical suffixes (instead of default aa, ab, ac, etc…),
and the option -a 5 to tell it I want the suffixes to be 5 digits long:

2. Split large files by Archiving them with Winrar (ShareWare) tool

If you have already Winrar installed and you don't want to bother with too much typing from the command line, You can use good old WinRAR as a file splitter/joiner as well.

To split a file into smaller files, select "Store" as the compression method and enter the desired value (bytes) into "Split to volumes" box.
This way you can have split files named as filename.part1.rar, filename.part2.rar, etc.

WinRAR_cut-split-large-files-into-pieces-screenshot-Windows

3. Split files with 7-Zip (FreeWare)

Assuming you have the 7-Zip installed on the PC, you can do the archiving of the Big file to a smaller pieces one, you can create the splitted file from 7Zip interfaces menus:

Or directly cut the single file into multiple volumes, directly from Windows Explorer by Selecting the file and using fall down menus :

Sum it up what learned ?

What we learned is how to cut large files into multiple single consequential ones for easy copy between Network sides, via both Git 4 Windows and manual copy paste of parted multiple files to OneDrive / DropBox / pCloud or Google Drive.
There is a plenty of other approaches to take as there is also file GUI tools, besides using GNU Win / Gnu Tools for Windows or Cygwin / Gsplit GUI tool or some kind of the many Archiver toolsavailable for Windows, another option to split the large files is to use a bunch of PowerShell and Batch scripts written that can help you do the file split for both binaries files or Text files. but i'll stop here as I believe that is pretty much enough for most basic needs.

Tags: AA, about, administrator, amount, and, another, Azure Cloud, case, Cloud Onedrive, configured, copy, due, file, How to, interesting, migrate, simply, thing, unable, Virtualbox Virtual Machines
Posted in Everyday Life, System Administration, Windows | No Comments »

How to Copy / Backup Windows USB drive from one USB to a second

Friday, October 18th, 2024

Did you know that when you copy all the files from a USB Drive you don’t copy all the data?

Did you know that there may be files that are not even visible?

In this tutorial you will discover how to copy all of your USB Drive sector by sector, that is to say, that you will see how to create a copy identical to your USB drive without missing anything!

This can be useful if you have formatted your USB stick in error and want to use it, you can create an image for the USB Drive on your computer and then you can recover the formatted data in the image afterward!

The software used in this tutorial is called ImageUSB, it is free, portable, and easy to use.

Don’t use this method if you want only to copy some files, use this to clone/backup your USB Drive with all its master boot record, partition tables, and data.

Let’s go!

Clone Your USB Drive with ImageUSB on Windows 10

Start by downloading and extracting ImageUSB from this official URL: https://www.osforensics.com/tools/write-usb-images.html

Double-click on imageUSB.exe .

Select your USB Drive from the list, select “Create image from USB drive“. Choose the location for the binary image file (.bin) that will be created from the USB drive.

Click on “Create“.Click “Yes” to confirm your choices.

imageusb clone usb flash drive backup restore 3 create image

Click “Yes” to overwrite the bin file in case it’s already there.

Wait for a couple of minutes…

After the image is created you should see this message. Click “OK“.

Now if you want to restore an image to your USB Drive, just select your USB Drive and choose “Write image to USB drive“. Choose your bin image and click on “Write“.

imageusb clone usb flash drive backup restore 7 write

This program is not recommended on different sizes USB Drives…
Use it mostly for backup/restore on the same USB Drive for your bootable software.

There you have it, the copy of USB to second USB completed !

Enjoy !

Tags: after, ALL, and, are, bin, binary, boot, called, case, choices, Click, clone, com, Computer, copy, couple, create, second, USB, www
Posted in Backups, System Administration, Windows | No Comments »

Recover lost / forgotten root password for CentOS 7 Linux / Boot CentOS 6 into Single User mode to reset admin pass

Friday, September 27th, 2024

If you have some old CentOS 7 Virtual machine hanging for a long time and you don't remember the root password or you don't remember where you have stored it, but you have something important as data left over, you might need to recover root password for your CentOS 7 Virtual Machine.

I recently had to resolve that issue and here is the few easy steps to take to recover the lost root password.

Assuming you have tried to boot the VM and the VM boots fine and your few attempts to input manually some default passwords of yours failed, next

1. Reboot the Virtual Machine to the GRUB boot menu

The GRUB boot screen should appear and be there for few secs

2. Edit the boot loader kernel options ( add add rd.break enforcing=0 )

How to reset root password on CentOS Linux - Clouvider

Press 'e' to Edit the boot loader and modify the boot commands options passed to the linux kernel.

In GRUB edit mode:

add rd.break enforcing=0

to the end of the line starting with linux at the end of passed parameters list as shown in the picture.

When done editing, press Ctrl-x (Control button x key simultaneously) to boot with changed parameters.

ALTERNATIVE WAY TO BOOT THE SYSTEM INTO ROOT WITHOUT PASSWORD PROMPT:

Alternative options to use instead of add rd.break.enforcing=0 are to substitute the rhgb quiet kernel option with init=/bin/bash

As you might wonder for the meaning of the passed 2 parameters:

rd.break breaks the boot process at initramfs while
enforcing=0 disables the SELinux (which often enabled by default on CentOS).

Another way is to

3. Boot in CentOS emergency mode and Reset the root password

When done editing, press Ctrl-x to boot with changed parameters.

As you might wonder for the meaning of the passed parameters:

rd.break breaks the boot process at initramfs while
enforcing=0 disables the SELinux (which often enabled by default on CentOS).

Whence system boots up with the modified kernel options cmd, the switch_root prompt will appear.
As the emerency mode boots the filesystem into read-only mode under /sysroot default directory, in order to be able to
modify the MD5 root password stored hash inside RO mounted /sysroot/etc/shadow you need to remount the Filesystme
in read-write mode.

To Remount the read-only file system /sysroot in write mode:

# mount -o remount,rw /sysroot

As the /sysroot is not the root directory to be able to use a standard passwd command you need to make /sysroot
as the default root folder for the booted linux by chrooting into it.

Generate MD5 password manually (for Hardcore masochistic admins 🙂 )

If you're a hard core linux sysadmin of course, generate your own new md5 password and directly modify /etc/shadow copy pasting the md5 string.

If you want to manually generate the md5 string, you can do it depending on the required encryption algorithm with:

For (md5, sha256, sha512) encrypted pass

# openssl passwd -6 -salt xyz yourpass

For (md5, sha256, sha512) encrypted pwd

# mkpasswd –method=SHA-512 –stdin

For (des, md5, sha256, sha512) encrypted pw

# perl -e 'print crypt("YourPasswd", "salt", "sha512"),"\n"'

Once the string is generated;

# vim /etc/shadow

and exchange the old with new string for MD5

Change password with chroot (the easy common way)

remount read write the filesystem in emergency single user mode CentOS LINUX

# chroot /sysroot

That should drop you into another shell bash-4.x

Reset root user password in CentOS 7

# passwd
Changing password for user root.
New password:
Retype new password:

We need have to sync the entire filesystem we have to use the sync command, for novice sys admins who never heard about this command, below
short description:

The Linux sync command synchronizes cached data to permanent storage.
This data includes modified superblocks, modified inodes, delayed reads and writes, and others. sync uses several system calls:

sync()
syncfs()
fsync()
fdatasync()

For example, the sync command utilizes the sync() system call to write all buffered modifications to file data and metadata to an underlying storage device.

As a Linux systems administrator or developer, understanding the sync command can be crucial for efficient file synchronization. Additionally, sync can be helpful after crashes or when the file system becomes corrupted.

In this tutorial, we’ll explore the various aspects of the sync command. Also, we’ll see how we can use sync in different scenarios.

# sync

# exec /sbin/init

Try out the root password after booting normally into CentOS and the new set administrator pass should work.

Resetting forgotten (lost) root password on CentOS 6

The process is absolutely the same except on the Step 1 (in the modification of GRUB boot menu by pressing e key), add to

rhgb quiet

at the end one 'S'

This S character means 'boot CentOS into Single user mode'

rhgb quiet S

Then, press ENTER key and press b key to boot CentOS 6 into to single user mode.

Tags: about, absolutely, administrator, Admins, after, algorithm, ALL, com, command, default passwords, editing, filesystem, linux kernel, New, password, Reboot, Recover, root password, sync, www
Posted in CentOS, Linux, RedHat, System Administration | No Comments »

Debug and Fix QMAIL Mail Server qmail-inject: fatal: qq temporary problem (#4.3.0) and ‘reformime[1648048]: segfault at 0 ip 00007fea608bef28 sp 00007fff3c8d4bc0 error 4’ errors after update from Debian 11 to Debian 12

Monday, September 2nd, 2024

finding-qmail-install-problems-common-reasons-for-unworking-qmail-debugging-qmail

For a legacy reasons and lack of time and fact once Qmail is run on a server it works almost forever if you don't do very major upgrades and you still to the same version I have few Qmail SMTP servers that are nowadays are there for historical reasons.

After the last major version upgrade from Debian 11 to Debian 12, I've got the qmail smtpd not completely running fine and I have to follow some of my previous blog notes on how to recover in that situations as well as some common logic to resolve it.

After the upgrade I started getting every few minutes a repeating really annoying error due to reformime crashing in /var/log/messages as well as in qmail logs, the exact error was as so

Sep 1 22:15:34 pcfr_hware_local_ip kernel: [366799.585663] Code: eb e1 48 8d 15 31 96 13 00 e9 04 00 00 00 0f 1f 40 00 41 55 49 89 d5 41 54 49 89 f4 55 53 48 89 fb 48 83 ec 08 48 85 ff 74 58 <80> 3b 00 74 46 4c 89 e6 48 89 df e8 58 61 f8 ff 48 8d 2c 03 80 7d
Sep 1 22:17:50 pcfr_hware_local_ip kernel: [366935.524185] reformime[1647438]: segfault at 0 ip 00007f0b9beeff28 sp 00007fff4ffd5850 error 4 in libc.so.6[7f0b9be76000+155000] likely on CPU 1 (core 1, socket 0)
Sep 1 22:17:50 pcfr_hware_local_ip kernel: [366935.524207] Code: eb e1 48 8d 15 31 96 13 00 e9 04 00 00 00 0f 1f 40 00 41 55 49 89 d5 41 54 49 89 f4 55 53 48 89 fb 48 83 ec 08 48 85 ff 74 58 <80> 3b 00 74 46 4c 89 e6 48 89 df e8 58 61 f8 ff 48 8d 2c 03 80 7d
Sep 1 22:18:44 pcfr_hware_local_ip kernel: [366989.796532] reformime[1647577]: segfault at 0 ip 00007fe8e14bef28 sp 00007ffc000e9040 error 4 in libc.so.6[7fe8e1445000+155000] likely on CPU 1 (core 1, socket 0)
Sep 1 22:18:44 pcfr_hware_local_ip kernel: [366989.796554] Code: eb e1 48 8d 15 31 96 13 00 e9 04 00 00 00 0f 1f 40 00 41 55 49 89 d5 41 54 49 89 f4 55 53 48 89 fb 48 83 ec 08 48 85 ff 74 58 <80> 3b 00 74 46 4c 89 e6 48 89 df e8 58 61 f8 ff 48 8d 2c 03 80 7d
Sep 1 22:20:08 pcfr_hware_local_ip kernel: [367072.889786] reformime[1647888]: segfault at 0 ip 00007efcaa6bef28 sp 00007ffdfe793560 error 4 in libc.so.6[7efcaa645000+155000] likely on CPU 1 (core 1, socket 0)
Sep 1 22:20:08 pcfr_hware_local_ip kernel: [367072.889809] Code: eb e1 48 8d 15 31 96 13 00 e9 04 00 00 00 0f 1f 40 00 41 55 49 89 d5 41 54 49 89 f4 55 53 48 89 fb 48 83 ec 08 48 85 ff 74 58 <80> 3b 00 74 46 4c 89 e6 48 89 df e8 58 61 f8 ff 48 8d 2c 03 80 7d
Sep 1 22:21:14 pcfr_hware_local_ip kernel: [367139.010116] reformime[1648048]: segfault at 0 ip 00007fea608bef28 sp 00007fff3c8d4bc0 error 4 in libc.so.6[7fea60845000+155000] likely on CPU 1 (core 1, socket 0)
Sep 1 22:21:14 pcfr_hware_local_ip kernel: [367139.010139] Code: eb e1 48 8d 15 31 96 13 00 e9 04 00 00 00 0f 1f 40 00 41 55 49 89 d5 41 54 49 89 f4 55 53 48 89 fb 48 83 ec 08 48 85 ff 74 58 <80> 3b 00 74 46 4c 89 e6 48 89 df e8 58 61 f8 ff 48 8d 2c 03 80 7d
Sep 1 22:22:43 pcfr_hware_local_ip rsyslogd: — MARK —

To debug more concretely what exactly was happening with reformime and why it was crashing with the libc segfault error, I've used the journalctl log with this cmd:

# journalctl -p 3 -xb

сеп 01 22:10:27 pcfrxen qmail-scanner-queue.pl[2170438]: X-Qmail-Scanner-2.10st:[pcfrxen17252178278122170438] d_m: output spotted from /usr/bin/reformime -x/var/spool/qscan/tmp/pcfrxen17252178278122170438/ (Segmentation fau>
) – that shouldn't happen!
сеп 01 22:11:11 pcfrxen qmail-scanner-queue.pl[2170631]: X-Qmail-Scanner-2.10st:[pcfrxen17252178718122170631] d_m: output spotted from /usr/bin/reformime -x/var/spool/qscan/tmp/pcfrxen17252178718122170631/ (Segmentation fau>
) – that shouldn't happen!
сеп 01 22:15:32 pcfrxen qmail-scanner-queue.pl[2171777]: X-Qmail-Scanner-2.10st:[pcfrxen17252181328122171777] d_m: output spotted from /usr/bin/reformime -x/var/spool/qscan/tmp/pcfrxen17252181328122171777/ (Segmentation fau>
) – that shouldn't happen!
сеп 01 22:15:35 pcfrxen qmail-scanner-queue.pl[2171793]: X-Qmail-Scanner-2.10st:[pcfrxen17252181358122171793] d_m: output spotted from /usr/bin/reformime -x/var/spool/qscan/tmp/pcfrxen17252181358122171793/ (Segmentation fau>
) – that shouldn't happen!
сеп 01 22:21:21 pcfrxen qmail-scanner-queue.pl[2173427]: X-Qmail-Scanner-2.10st:[pcfrxen17252184788122173427] d_m: output spotted from /usr/bin/reformime -x/var/spool/qscan/tmp/pcfrxen17252184788122173427/ (Segmentation fau>
) – that shouldn't happen!

As you can see this showed that the problem is with reformime's passing on -x argument, and some temporary directory, thus to make sure the crash is not a cause of some mixed permissions, I've had to check the /var/spool/qscan permissions, and clamd permissions and few other permissions of the qmail install, and the wrong permissions (perhaps after the update of clamav after the Debian Linux migration was with /var/lib/clamav which was incorrectly owned by user clamav group clamav instead of the qscand / qscand user group, thus to resolve, I've run:

# chown qscand:qscand /var/lib/clamav/ -R

Another thing I've had to correct was the /var/log/qmail permissions which was too permissive (perhaps due to some old install time hurry up stupidity done), so to correct, them:

# chmod 750 /var/log/qmail/

First thing i tried to resolve is of course to reinstall maildrop debian package that provides /usr/bin/reformime binary.

root@pcfreak:/usr/local/bin# dpkg -l |grep -i maildrop
rc courier-maildrop 0.68.2-1 amd64 Courier mail server – mail delivery agent
ii maildrop 2.9.3-2.1 amd64 mail delivery agent with filtering abilities (set-GID=mail)

In an old post of mine on a similar error Fixing Qmail 451 qq temporary problem (#4.3.0) / @4000000050587780174c60dc status: qmail-todo stop processing asap / status: exiting, part of the solution was to reinstall maildrop, so tried this one:

root@pcfreak:/usr/local/bin# apt install –reinstall maildrop

Of course to try it out restarted qmail with the usual

# qmailctl restart

Sadly enough this doesn't solve it, so I had to look up for other solutions and spend about 3 / 4 hours reading online just to convince myself that finding any meaningful in the classical human way, is becoming pretty much impossible task. As the content of information on the Internet has grown tremendously over the last years, it seems the quality of posts and commited data is exponentially detereorating. So the only way to solve crashes of binaries is either to stick to a debugger such as gdb or simply try rebuild the .deb binary from scratch and see whether a recompile from source might makes a difference.

After even more digging up online, found out some Gentoo forums threads, where people described thethe issue was also connected to the failing reformime libc use bug, with an applied C patch, found threads on Ubuntu and Debian users complaining about mysterious errors with libc with maildrop and even a bug report that this is some kind of libc bug, related to the precompiled version of maildrop shipped by default deb based repos.

Hence, My approach to resolve it was to recompile maildrop from source code, which even though looking a tedious task came with plenty of dependencies, I had to install plenty of developlment libraries and tools, compilers etc. as well as the following libs.

# apt install pcre2-utils
# apt install libpcre2-dev
# apt install libidn11
# apt install libidn2-dev
# apt install libcourier-unicode-dev
# apt install libcourier-unicode4

Then had to download and install from source the latest available versions of courier-authlib and its dependencies courier-unicode and once having those two recompiled with

# links https://sourceforge.net/projects/courier/files/courier/1.3.12/courier-1.3.12.tar.bz2/download
# links https://sourceforge.net/projects/courier/files/maildrop/3.1.8/maildrop-3.1.8.tar.bz2/download
# links https://sourceforge.net/projects/courier/files/authlib/0.72.3/courier-authlib-0.72.3.tar.bz2/download
# links https://sourceforge.net/projects/courier/files/courier-unicode/2.3.1/courier-unicode-2.3.1.tar.bz2/download

# tar -jxvf courier-unicode-2.3.1.tar.bz2
# tar -jxvvf courier-authlib-0.72.3.tar.bz2
# tar -jxvvf maildrop-3.1.8.tar.bz2

# cd courier-unicode-2.0/
# ./configure && make && make install
…
# cd ..
# cd courier-authlib-0.72.3
# ./configure && make && make install
# cd ..
…
# cd maildrop-3.1.8/
# ./configure && make && make install
…

I've took the time to also preinstall a bunch of perl modules deb packages which rawly are the ones found in file, i've built with the binaries perl-modules-for-qmail-needed.txt

To reinstall the binaries, run a small shell loop:

# for i in $(cat perl-modules-for-qmail-needed.txt); do apt install –reinstall $i –yes; done

Have to say also identified an issue with /var/qmail/bin/qmail-scanner-queue.pl with qmail-inject failing after testing qmail-scanner-queue installation with:

# /downloads/qmail-scanner-2.11st/contrib/test_installation.sh -doit

# ./test_installation.sh -doit

Sending standard test message – no viruses… 1/4
qmail-inject: fatal: qq temporary problem (#4.3.0)
Bad error. qmail-inject died

Anyone who ever administrated Qmail Mail server knows pretty well, about the Terrible error:

qmail-inject: fatal: qq temporary problem (#4.3.0)

and that it could be mostly anything, thus anyways to find out what might be the cause I've continued to Debug.

# ldd qmail-inject
linux-vdso.so.1 (0x00007ffc43f5a000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fcc2099b000)
/lib64/ld-linux-x86-64.so.2 (0x00007fcc20ba1000)

To succed debug the QMAIL issues with qmail I found as very useful something from another old post of mine on debugging qmail errors – Testing Qmail installation for problems: Common reasons for unworking qmail / How to debug Qmail mail server failing to delivery or send emails, the following log debug loop:

# for i in $(ls -d /var/log/qmail/*qmail*/); do tail -n 10 $i/current|tai64nlocal; sleep 5; done

Finally the last step resolve the qmail-inject error, was to modify /var/qmail/bin/qmail-scanner-queue.pl and exchange PATH of /usr/bin/reformime default shipped debian repository to new custom built /usr/local/bin/reformime.

After retesting the qmail-scanner installation all seemed fine onwards:

# /downloads/qmail-scanner-2.11st/contrib/test_installation.sh -doit

Sending standard test message – no viruses… 1/4
done!

Sending eicar test virus – should be caught by perlscanner module… 2/4
done!

Sending eicar test virus with altered filename – should only be caught by commercial anti-virus modules (if you have any)… 3/4
done!

Sending bad spam message for anti-spam testing – In case you are using SpamAssassin… 4/4

If you have enabled $sa_quarantine, $sa_delete or $sa_reject the
spam-message wont't arrive to the recipients. But if you have enabled
(good idea!) 'debug' you should check
/var/spool/qscan/qmail-queue.log (or where ever you have the log).

Done!

Finished test. Now go and check Email sent to postmaster@mail.pc-freak.net and/or the log..

Thibs Qmail install qmr_inst_check script also reported my server qmail install scripts as in good state:

# /downloads/scripts/qmr_inst_check
! vpopmail database do not exist!

So Hip Hip Hooray my Qmails works again ! Me fixed it again !
if you need help with fixing your company Professional Mail QMAIL server or Postfix, contact me via the contact form. Enjoy

Tags: blog, difference, Enjoy, Fixing Qmail, information, log messages, loop, plenty, qmail-inject:, Sending, state
Posted in Linux, OS Update, Qmail, System Administration | No Comments »

yum search file in all installable RPM, find out which rpm package provides binary file or missing library dependency on CentOS / RHEL / Fedora

Friday, August 23rd, 2024

Sometimes if you have a missing library or a file you know should be available via an rpm but you're not sure which RPM you have to install you have to look up for library or binary file amongs all available installable r[ms on Redhat Linux / CentOS / Fedora or other RPM based distro.

It is really annoying especially, if you try to install an rpm binary and the package does not install due to missing dependency library. Having a missing dependency package could happen, if you use some custom internal prepared repository that is mirroring from original rpm repositories and the RPM Repositories are situated behind a DMZ firewall network (such scenarios are common for corporations and IT companies).

Finding out which file is provided by which package on Debian / Ubuntu and other deb based linux distributions is easy and done via the

# apt-file search filename

Thus if you're a system administrator coming from a Debian GNU / Linux sysadmin realm into the wonderful world of redhats, you will want to have an alternative to apt-file tool. You will be happy to find out that that this tedious task is easily done in RPM based Linux and is integrated straight into yum package manager too.

The command to search which rpm package provides a file is:

# yum whatprovides 'nc'

[root@rhel-linux ~]# yum whatprovides nc
Loaded plugins: fastestmirror, versionlock
Loading mirror speeds from cached hostfile
2:nmap-ncat-6.40-19.el7.x86_64 : Nmap's Netcat replacement
Repo : base
Matched from:
Provides : nc

2:nmap-ncat-6.40-19.el7.x86_64 : Nmap's Netcat replacement
Repo : @base
Matched from:
Provides : nc

yum whatprovides search_file_name can be also invoked with its shortcut yum provides 'search_file_name'

[root@rhel-server ~]# yum provides '/bin/ls'
Loaded plugins: fastestmirror, versionlock
Loading mirror speeds from cached hostfile
coreutils-8.22-24.el7.x86_64 : A set of basic GNU tools commonly used in shell scripts
Repo : base
Matched from:
Filename : /bin/ls

coreutils-8.22-24.el7_9.2.x86_64 : A set of basic GNU tools commonly used in shell scripts
Repo : updates
Matched from:
Filename : /bin/ls

Here is another example:

[root@rhel-server ~]# yum -q provides '*lesspipe.sh*'
less-458-9.el7.x86_64 : A text file browser similar to more, but better
Repo : base
Matched from:
Filename : /usr/bin/lesspipe.sh

source-highlight-3.1.6-6.el7.i686 : Produces a document with syntax highlighting
Repo : base
Matched from:
Filename : /usr/bin/src-hilite-lesspipe.sh

source-highlight-3.1.6-6.el7.x86_64 : Produces a document with syntax highlighting
Repo : base
Matched from:
Filename : /usr/bin/src-hilite-lesspipe.sh

spirv-tools-2019.1-4.el7.x86_64 : API and commands for processing SPIR-V modules
Repo : epel
Matched from:
Filename : /usr/bin/spirv-lesspipe.sh

You can search for any file and if the RPm repository is defined under /etc/yum/repos.d/* and enabled, yum whatprovides command should be able to find it and tell you which RPM package you have to install to have the file installed Redhat way.

You can list all enabled RPM repositories with cmd:

[root@rhel-server ~]# yum repolist enabled
Loaded plugins: fastestmirror, versionlock
Loading mirror speeds from cached hostfile
repo id repo name status
3party Third party packages – x86_64 2,631
base/7/x86_64 CentOS-7 – Base 10,072
cr/7/x86_64 CentOS-7 – CR 0
epel/7/x86_64 EPEL packages for RedCent 7 – x86_64 13,791
extras/7/x86_64 CentOS-7 – Extras 526
updates/7/x86_64 CentOS-7 – Updates 5,802
zabbix-6.0 Zabbix 6.0 repo 429
repolist: 33,251

To list disable RPM repositories:

# yum repolist disabled
…

To list all present available repositories that could be enabled and are set via the /etc/yum.repos.d/* configs

# yum repolist all
…

Tags: API, available, cmd, dependency, distro, document, find, lib, list, loading, mirror, package, rpm, updates, yum
Posted in Linux, Linux Package Management, OS Update, RedHat, System Administration | No Comments »

How to Update / Migrate zabbix-agent 5 to zabbix-agent2 6 on Redhat / CentOS / Fedora Linux

Friday, August 9th, 2024

If you have servers reporting monitoring with Zabbix running still on Zabbix-Agent 1 version 5.0.X but already migrated the Zabbix-server to Zabbix 6, it is a good idea to update the Agent to Zabbix Agent 6 As sson as possible, as you know lacking behind in version makes updating harder and more complicated task.

Mine and I guess most system administrators experience points that Keeping at the same level of versioning on many applications historically has shown to reduce unexpected errors and bugs but nowadays, the rule of keeping local and remote application ( programs ) at the same version level is regularly broken.

Theoretically Zabbix-Agent (Client) and Zabbix (Server) has a compitability for a certain range of versions (Zabbix agents 2 from version 4.4 onwards are compatible with Zabbix 7.0; Zabbix agent 2 must not be newer than 7.0 – for more on zabbix agent – > server version compitability check here ) and having a slight version difference should not be really a problem but often you might have a third party proxies in between such as haproxy or zabbix-proxy or other network oddities and thus my personal opinion is that for interoperability it is better to keep the Zabbix Clients and Zabbix Servers across the DMZ-ed networks running at same version level.

Some would say I have an old fashion thinking as software and technology is moving forward, but as I see how programming code writing and even software is constantly degradating just a reflection of degradation of human element, I prefer to keep my old know how and always stick to same versioning whenever possible.

Some would wonder then why would I upgrade to Zabbix-agent2 ? , if have to keep the same versioning, the reason is zabbix-agent2 is written in GO Language and is much faster and supposably better piece of software than Zabbix Agent1 that is written in Python.

Moreover having Zabbix agent 2 instead of 1 gives also benefits as you can do a bit more with zabbix and on the other hand the machines are more ready for monitoring in terms of future. To know more about the Benefits of Zabbix Agent2 compared to Zabbix Agent 1 read the Agent vs Agent2 comparison on zabbix website.

With this little introduction, lets proceed with the exact steps to take to upgrade zabbix-agent1 to zabbix-agent2.

1. Check the current installed Zabbix-Agent version

[user@monitored-server ~]$ rpm -qa |grep -i zabb
zabbix-get-5.0.42-1.el8.x86_64
zabbix-sender-5.0.42-1.el8.x86_64
zabbix-agent-5.0.42-1.el8.x86_64
[user@server ~]$

2. Create backup copy of current system working zabbix_agentd.conf

Before messing up with the working zabbix-agent as usual create the necessery backup to prevent later suprises

[user@monitored-server ~]$ cp -vrpf /etc/zabbix/zabbix_agentd.conf /etc/zabbix/zabbix_agentd.conf.bak-$(date '+%Y-%m-%d_%H-%M-%S')
…

3. Check current configured Zabbix repos

[user@monitored-server ~]$ vim /etc/yum.repos.d/zabbix.repo

[zabbix-4.0]
name = zabbix-4.0 – 8
baseurl = http://zabbix-repo-server.com/external/zabbix-4.0/8/$basearch
enabled = 0
gpgkey = http://zabbix-repo-server.com/external/zabbix-4.0/zabbix-official-repo.key
gpgcheck = 1

[zabbix-4.4]
name = zabbix-4.4 – 8
baseurl = http://zabbix-repo-server.com/external/zabbix-4.4/8/$basearch
enabled = 0
gpgkey = http://zabbix-repo-server.com/external/zabbix-4.4/zabbix-official-repo.key
gpgcheck = 1

[zabbix-5.0]
name = zabbix-5.0 – 8
baseurl = http://zabbix-repo-server.com/external/zabbix-5.0/8/$basearch
enabled = 1
gpgkey = http://zabbix-repo-server.com/external/zabbix-5.0/zabbix-official-repo.key
gpgcheck = 1

[zabbix-5.4]
name = zabbix-5.4 – 8
baseurl = http://zabbix-repo-server.com/external/zabbix-5.4/8/$basearch
enabled = 0
gpgkey = http://zabbix-repo-server.com/external/zabbix-5.4/zabbix-official-repo.key
gpgcheck = 1

[zabbix-6.0]
name = zabbix-6.0 – 8
baseurl = http://zabbix-repo-server.com/external/zabbix-6.0/8/$basearch
enabled = 0
gpgkey = http://zabbix-repo-server.com/external/zabbix-6.0/zabbix-official-repo.key
gpgcheck = 1

4. Modify repositories and include the Zabbix Agent6 yum repos

[user@monitored-server ~]$ cp -rpf zabbix.repo zabbix.repo.5.0.rpmsave

As we want to keep only the 6.0 version, leave only the zabbix-6.0 section and enable the repo:

[user@monitored-server ~]$ vim /etc/yum.repos.d/zabbix.repo

[zabbix-6.0]
name = zabbix-6.0 – 8
baseurl = http://zabbix-repo-server.com/external/zabbix-6.0/8/$basearch
enabled = 1
gpgkey = http://zabbix-repo-server.com/external/zabbix-6.0/zabbix-official-repo.key
gpgcheck = 1

5. Update zabbix-agent to zabbix-agent2 and update zabbix-get zabbix-sender versions

To not disrupt reported monitoring for zabbix-agent, don't delete zabbix-agent1 but instead in pararallel install and configure
zabbix-agent2 and then once configuration is migrated from Agent 1 to 2, stop the old zabbix-agent and bring up the new one.

[user@monitored-server ~]$ yum check-update

[user@monitored-server ~]$ yum install zabbix-agent2 zabbix-get zabbix-sender -y

Note that if you want to have a precise version number of zabbix-agent that is lets say 6.0.31 to correspond to zabbix-server 6.0.31 (even though in the repositories newer RPM versions are available), run:

[user@monitored-server ~]$ yum upgrade zabbix-agent2-6.0.31-release1.el8

Check new zabbix_agent2 installed version

# zabbix_agent2 -V
zabbix_agent2 (Zabbix) 6.0.31
Revision b6d93755a1b 17 June 2024, compilation time: {undefined} {undefined}, built with: go1.21.3
Plugin communication protocol version is 6.0.13

Copyright (C) 2024 Zabbix SIA
License GPLv2+: GNU GPL version 2 or later <https://www.gnu.org/licenses/>.
This is free software: you are free to change and redistribute it according to
the license. There is NO WARRANTY, to the extent permitted by law.

This product includes software developed by the OpenSSL Project
for use in the OpenSSL Toolkit (http://www.openssl.org/).

Compiled with OpenSSL 1.1.1k FIPS 25 Mar 2021
Running with OpenSSL 1.1.1k FIPS 25 Mar 2021

We use the library Eclipse Paho (eclipse/paho.mqtt.golang), which is
distributed under the terms of the Eclipse Distribution License 1.0 (The 3-Clause BSD License)
available at https://www.eclipse.org/org/documents/edl-v10.php

We use the library go-modbus (goburrow/modbus), which is
distributed under the terms of the 3-Clause BSD License
available at https://github.com/goburrow/modbus/blob/master/LICENSE

6. Migrate old /etc/zabbix/zabbix_agentd.conf to /etc/zabbix/zabbix-agent2.conf

For readability to show the main configured variables for zabbix-agent without the tons of comments, to later include in agent2

[root@monitored-server ~]# cat /etc/zabbix/zabbix_agentd.conf | grep -v '\#' | sed '/^$/d'
PidFile=/var/run/zabbix/zabbix_agentd.pid
LogFile=/var/log/zabbix/zabbix_agentd.log
LogFileSize=0
Server=10.50.37.8,127.0.0.1
ServerActive=10.50.37.8,127.0.0.1
Hostname=fqdn-of-monitored-host.domain.com
Timeout=20
Include=/etc/zabbix/zabbix_agentd.d/*.conf

The default zabbix-agent2 installed config would like similar to:

[root@monitored-server ~]# cat /etc/zabbix/zabbix_agent2.conf | grep -v '\#' | sed '/^$/d'
PidFile=/run/zabbix/zabbix_agent2.pid
LogFile=/var/log/zabbix/zabbix_agent2.log
LogFileSize=0
Server=127.0.0.1
# Specify the location of the Zabbix server host.
ServerActive=127.0.0.1
Hostname=Zabbix server
Include=/etc/zabbix/zabbix_agent2.d/*.conf
PluginSocket=/run/zabbix/agent.plugin.sock
ControlSocket=/run/zabbix/agent.sock
Include=./zabbix_agent2.d/plugins.d/*.conf

The new migrate one, should be like:

[root@monitored-server ~]# vim /etc/zabbix/zabbix_agent2.conf
PidFile=/run/zabbix/zabbix_agent2.pid
LogFile=/var/log/zabbix/zabbix_agent2.log
LogFileSize=10
Server=10.34.89.7,127.0.0.1
ServerActive=10.34.89.7,127.0.0.1
Hostname=lqgblu02f.ffm.de.int.atosorigin.com
Timeout=20
Include=/etc/zabbix/zabbix_agent2.d/*.conf
PluginSocket=/run/zabbix/agent.plugin.sock
ControlSocket=/run/zabbix/agent.sock
Include=/etc/zabbix/zabbix_agent2.d/plugins.d/*.conf

7. Add few Optimization variables for better zabbix-server -> zabbix-proxy -> zabbix-server interactions

If you have sometimes a network delays between zabbix server -> zabbix client and vice versa (depending on whether Zabbix agent is configured as Active or Passive mode), it is often useful
to add those 2 variables:

# How often list of active checks is refreshed, in seconds
RefreshActiveChecks=60
# Refresh the active checks on start.ForceActiveChecksOnStart=1
ForceActiveChecksOnStart=1

Also it might be a good practice to add zabbix_agent2.log monitoring with the agent itself, if the log exceeds certain amount, instead of calling it via logrotate.

# Perform log file rotation at the 1 MB point for the specified filepath
LogFileSize=1

[root@monitored-server ~]# vim /etc/zabbix/zabbix_agent2.conf
PidFile=/run/zabbix/zabbix_agent2.pid
LogFile=/var/log/zabbix/zabbix_agent2.log
LogFileSize=10
Server=10.34.89.7,127.0.0.1
ServerActive=10.34.89.7,127.0.0.1
Hostname=lqgblu02f.ffm.de.int.atosorigin.com
RefreshActiveChecks=60
ForceActiveChecksOnStart=1
Timeout=20
Include=/etc/zabbix/zabbix_agent2.d/*.conf
PluginSocket=/run/zabbix/agent.plugin.sock
ControlSocket=/run/zabbix/agent.sock
Include=/etc/zabbix/zabbix_agent2.d/plugins.d/*.conf

8. Stop the old zabbix agent process and run the new one

# systemctl status –full zabbix-agent2
# systemctl stop zabbix-agent

Assuming that the configuratoin of zabbix-agent is correct, execute zabbix-agent2 via system control.and check its status

# systemctl start zabbix-agent2
# systemctl status –full zabbix-agent2

If no errors in the configuration, the zabbix_agent2 process should be up and running and the status of above systemctl cmd should report fine.
If you need concretics regarding exact Zabbix checks or whther current conigured Userparameter scripts errors, or any other warnings or errors
of zabbix_agent2 interacting to the server, check further the logs

[root@monitored-server ~]# tail -n 10 /var/log/zabbix/zabbix_agent2.log
2024/08/06 17:26:52.998749 using plugin 'WebPage' (built-in) providing following interfaces: exporter, configurator
2024/08/06 17:26:52.998760 using plugin 'ZabbixAsync' (built-in) providing following interfaces: exporter
2024/08/06 17:26:52.998794 using plugin 'ZabbixStats' (built-in) providing following interfaces: exporter, configurator
2024/08/06 17:26:52.998804 lowering the plugin ZabbixSync capacity to 1 as the configured capacity 100 exceeds limits
2024/08/06 17:26:52.998820 using plugin 'ZabbixSync' (built-in) providing following interfaces: exporter
2024/08/06 17:26:52.998993 Plugin communication protocol version is 6.0.13
2024/08/06 17:26:52.999018 Zabbix Agent2 hostname: [lqgblu02f.ffm.de.int.atosorigin.com]
2024/08/06 17:26:54.000667 [102] cannot connect to [127.0.0.1:10051]: dial tcp :0->127.0.0.1:10051: connect: connection refused
2024/08/06 17:26:54.000836 [102] active check configuration update from host [lqgblu02f.ffm.de.int.atosorigin.com] started to fail
2024/08/06 17:26:59.344837 Zabbix Agent 2 stopped. (6.0.31)

Tags: backup copy, checks, Hostname Zabbix, key, Modify, plugin, servers, Stop, version, zabbix-agent
Posted in CentOS, Linux, Monitoring, OS Update, RedHat, System Administration, System Optimization, Zabbix | No Comments »

All Debian Linux package repository apt sources.list file for Debian versions 6, 7, 8, 9, 10, 11 and 12

Friday, May 31st, 2024

debian-package-management-repositories-for-all-distributions

If you have to administrate legacy Debian servers, that keeps hanging either for historical reasons or just because you didn't have time to upgrade it up to latest versions, machines that are hanging in the hangar or a mid office building Old server room, doing nothing but simply NAT (Network Address Translation), Proxying, serving traffic via Squid / Haproxy / Apache / Varnish or Nginx server but you still want to have the possibility to extend the OS even though it is out of date / End of Life reached and out of support as well as perhaps full of security holes, but due to its unvisibility on the Internet hanging in a Demilitarized network the machine stayed on the Local (DMZ)-ed network and still for example you need to install simple things for administration reasons locally on the machine, for example nmap or netcat or some of the network tools for monitoring such as iftop or iptraf etc. you might find out unfortunately that this is not possible anymore, because the configured /etc/apt/sources.list repository mirror is no longer available at its URL. Thus to restore the functioning of apt and apt-get pkg management tools on Debian you need to correct the broken missing package mirrors due to resructurings on the network with a correct ones, originally provided by Debian or eventually if this doesn't work a possible Debian package archive URL.

In this article, I'll simply provide such URLs you might use to correct your no longer functioning package manager due to package repositoriy unavailibility, below are the URLs (most of which that should be working as of year 2024). To resolve the issues edit and place the correct Debian version you're using.

1. Check the version of the Debian Linux

# cat /etc/debian_version
…

or use the universal way to check the linux OS, that should be working on almost all Linux distributions

# cat /etc/issue
Debian GNU/Linux 9 \n \l

2. Modify /etc/apt/sources.list and place URL according to Debian distro version

# vim /etc/apt/sources.list

3. Repositories URL list Original and Archived for .deb packages according to Debian distro release
Debian 6 (Wheezy)

Original repostiroes (Not Available and Not working anymore as of year 2024)

Old Archived .deb repository for 6 Squeeze

deb http://archive.debian.org/debian squeeze main
deb http://archive.debian.org/debian squeeze-lts main

Debian 7 (Wheezy)

Original repostiroes (Not Available and Not working anymore as of year 2024)

Old Archived .deb repository for Jessie (still working as of 2024) :

deb http://archive.debian.org/debian wheezy main contrib non-free
deb http://archive.debian.org/debian-security wheezy/updates main

( Security updates are not provided anymore.)

NOTE: If you get an error about keyrings, just install it

# apt-get install debian-archive-keyring

Debian 8 (Jessie)
Original .deb package repository with non-free included for Debian 8 "Jessie"

deb http://deb.debian.org/debian/ jessie main contrib non-free
deb http://ftp.debian.org/debian/ jessie-updates main contrib
deb http://security.debian.org/ jessie/updates main contrib non-free

Old Archived .deb repository for 8 Jessie (still working as of 2024):

deb http://archive.debian.org/debian/ jessie main non-free contrib
deb-src http://archive.debian.org/debian/ jessie main non-free contrib
deb http://archive.debian.org/debian-security/ jessie/updates main non-free contrib
deb-src http://archive.debian.org/debian-security/ jessie/updates main non-free contrib

# echo "Acquire::Check-Valid-Until false;" | tee -a /etc/apt/apt.conf.d/10-nocheckvalid

# apt-get update

# apt-get update && apt-get upgrade

If you need backports, first be warned that these are archived and no longer being updated; they may have security bugs or other major issues. They are not supported in any way.

deb http://archive.debian.org/debian/ jessie-backports main

Debian 9 (Stretch)
Original .deb package repository with non-free included for Debian 9 "Stretch":

deb http://deb.debian.org/debian/ stretch main contrib non-free
deb http://deb.debian.org/debian/ stretch-updates main contrib non-free
deb http://security.debian.org/ stretch/updates main contrib non-free

Archived old repository .deb for Stretch :

deb http://archive.debian.org/debian/ stretch main contrib non-free
deb http://archive.debian.org/debian/ stretch-proposed-updates main contrib non-free
deb http://archive.debian.org/debian-security stretch/updates main contrib non-free

Debian 10 (Buster)
Origian repository URL:

deb http://deb.debian.org/debian/ buster main non-free contrib
deb http://deb.debian.org/debian/ buster-updates main non-free contrib
deb http://security.debian.org/ buster/updates main non-free contrib

Fixing unworking backports for Debian 10 Buster

Change the /etc/apt/sources.list URL with this one

deb http://archive.debian.org/debian buster-backports main contrib non-free

If you want to list packages installed via the backports repository only, that needs to be replaced with newer versions (if such available from the repository)

# apt list –installed | grep backports
# dpkg –list | grep bpo
# dpkg –list | grep -E '^ii.*bpo.*'

ii libpopt0:amd64 1.18-2 amd64 lib for parsing cmdline parameters
ii libuutil3linux 2.0.3-9~bpo10+1 amd64 Solaris userland utility library for Linux
ii libzfs4linux 2.0.3-9~bpo10+1 amd64 OpenZFS filesystem library for Linux

Debian 11 (Bullseye)
Origianl repository address:

deb http://deb.debian.org/debian bullseye main contrib non-free
deb http://deb.debian.org/debian bullseye-updates main contrib non-free
deb http://security.debian.org/debian-security bullseye-security main contrib non-free

Debian 12 (Bookworm)
Original Repository :

deb http://deb.debian.org/debian bookworm main contrib non-free-firmware non-free
deb http://deb.debian.org/debian bookworm-updates main contrib non-free-firmware non-free
deb http://security.debian.org/debian-security bookworm-security main contrib non-free-firmware non-free

Add Backports to sources.list

deb http://deb.debian.org/debian bookworm-backports main

Thats all, hopefully that would help some sysadmin out there. Enjoy !

Tags: about, administrate, Administration, ALL, amd64, amp, and, ANY, anymore, apache, apt-get, archive, are, article, available, being, Below, broken, bugs, building
Posted in Debian, Linux, Linux Package Management, Migration, OS Update, Remote System Administration, System Administration | No Comments »

How to view WIFI Passwords for Profile from command line with netsh on Windows 10

Wednesday, May 29th, 2024

The common way, if you have connected to a Wireless Network Access Point and saved the password in Windows is to view the password via Windows GUI interface, via menus following few easy steps:

1. Settings -> Network and Internet -> Network and Sharing Center

2. Click on (Wifi Network name) for which you need password and
3. In View your active networks section

4. When the Wi-Fi network status window opens, click Wireless Properties

5. Move to the Security Tab and check the checkbox, next to "Show Characters" to view the network password.

Nevertheless as a system administrator you might have wondered, how you can easily review in plain text Saved Wireless Networks Wi-FI passwords, without using the Graphical Interface via a direct command line cmd.exe?
Such thing is helpful on maintaining multiple Windows 10 hosts, especially if you have a telnet or SSH remote administration enabled or you have a domain of PCs.
To do so open cmd.exe command prompt and run:

C:\Users> netsh

netsh>wlan show profile

Profiles on interface Wi-Fi:

Group policy profiles (read only)
———————————

User profiles
————-
All User Profile : WIFI_Pofile-name
All User Profile: Hotel stage 2
All User Profile: Home Wifi
All User Profile: HP_Custom
…

Now lets review the clear text password of the profile from netsh console:

netsh>wlan show profile "WIFI_Pofile-name" key=clear

Profile WIFI_Pofile-name on interface Wi-Fi:
===================================================

Applied: All User Profile

Profile information
——————-
Version : 1
Type : Wireless LAN
Name : WIFI_Pofile-name
Control options :
Connection mode : Connect automatically
Network broadcast : Connect only if this network is broadcasting
AutoSwitch : Do not switch to other networks
MAC Randomization : Disabled

Connectivity settings
———————
Number of SSIDs : 1
SSID name : "WIFI_Pofile-name"
Network type : Infrastructure
Radio type : [ Any Radio Type ]
Vendor extension : Not present

Security settings
—————–
Authentication : WPA2-Personal
Cipher : CCMP
Authentication : WPA2-Personal
Cipher : GCMP
Security key : Present
Key Content : Very-secret-password-for-WIFI-plain-text

TADADAM !

We see the password key text Saved WIFI Passwords plain text !

Note that sometimes, if you have a Hidden Wifi Network the command to use to reveal the plain text password with netsh would be:

C:\Users> netsh wlan show profile "name=SSID hidden WiFi Net" key=clear
…

This trick is very much used today by "hackers" e.g. script kiddies, who break up into others windows.
It is also useful if you want to have a quick way to review plain text passwords for WIFI accounts with organization, lets say if you're a security expert and doing some kind of periodic Security audits within a corporation on multiple Domain attached computers.

Thanks to Martin Petrov (Amridikon) for his trick as I've learned first time from his blog https://mpetrov.net, which is full of many computer geek goodies stuff.

Of course this approach can be easily scripted with a short PowerShell script:

netsh wlan show profile |
Select-String '(?<=All User Profile\s+:\s).+' |
ForEach-Object {
$wlan = $_.Matches.Value
$passw = netsh wlan show profile $wlan key=clear |
Select-String '(?<=Key Content\s+:\s).+'

[pscustomobject]@{
Name = $wlan
Password = $passw.Matches.Value
}
}

If you need the script View-all-wifi-passwords-plaintext-windows10.ps1 to reuse it download it from here.

There is also some freeware tools online which can help you reveal passwords, saving you any typing, that might be useful if you want to delegate the task to a non-sysadmin user, you can simply point him and ask him to install a GUI Win tool like Wifi Password revealer (that makes showing plain text passwords piece of cake) and let user reveal his passwords for himself, if needs the password to share it to a colleague 🙂
That's all folks, Happy hacking !

Tags: CCMP, Click, GCMP, How to, Key Content, passwords, plain, Profile, text, Wireless Network Access Point
Posted in Remote System Administration, Security, System Administration, Various, Windows | No Comments »

Haproxy Enable / Disable Application backend server configured to roundrobin in emergency case via haproxy socket command

Thursday, May 2nd, 2024

haproxy-stats-socket

Haproxy LB backend BACKEND_ROUNDROBIN are configured to roundrobin with check health check port (check port 33333).
For example letsa say haproxy server is running with a haproxy_roundrobin.cfg like this one.

Under some circumstances however if check port TCP 33333 is UP, but behind 1 or more of Application that is providing the resources to customers misbehaves ,
(app-server1, app-server2, app-server3, app-server4) members , Load Balancer cannot know this, because traffic routing decision is made based on Echo port.

One example scenario when this can happen is if Application server has issue with connectivity towards Database hosts:
(db-host1, db-host2, db-host3, db-host4)

If this happens 25% of traffic might still get balanced to broken Application server. If such scenario happens during OnCall and this is identified as problem,
work around would be to temporary disable the misbehaving App servers member from the 4 configured roundrobin pairs in haproxyproduction.cfg :

For example if app-server3 App node is identified as failing and 25% via LB is lost, to resolve it until broken Application server node is fixed, you will have to temporary exclude it from the ring of roundrobin backend hosts.

1. Check the status of haproxy backends

# echo "show stat" | socat stdio /var/lib/haproxy/stats

As you can see the backend is disabled.

Another way to do it which will make your sessions to the server not directly cut but kept for some time is to put the server you want to exclude from haproxy roundrobin to "maintenace mode".

echo "set server bk_BACKEND_ROUNDROBIN/app-server3 state maint" | socat unix-connect:/var/lib/haproxy/stats stdio

Actually, there is even better and more advanced way to disable backend from a configured rounrobin pair of hosts, with putting the available connections in a long waiting queue in the proxy, and if the App host is inavailable for not too short, haproxy will just ask the remote client to keep the connection for longer and continue the session interaction to remote side and wait for the App server connectivity to go out of maintenance, this is done via "drain" option.

echo "set server bk_BACKEND_ROUNDROBIN/app-server3 state drain" | socat unix-connect:/var/lib/haproxy/stats stdio

This sets the backend in DRAIN mode. No new connections are accepted and existing connections are drained.

To get a better idea on what is drain state, here is excerpt from haproxy official documentation:

Force a server's administrative state to a new state. This can be useful to
disable load balancing and/or any traffic to a server. Setting the state to
"ready" puts the server in normal mode, and the command is the equivalent of
the "enable server" command. Setting the state to "maint" disables any traffic
to the server as well as any health checks. This is the equivalent of the
"disable server" command. Setting the mode to "drain" only removes the server
from load balancing but still allows it to be checked and to accept new
persistent connections. Changes are propagated to tracking servers if any.

2. Disable backend app-server3 from rounrobin

# echo "disable server BACKEND_ROUNDROBIN/app-server3" | socat unix-connect:/var/lib/haproxy/stats stdio

# pxname,svname,qcur,qmax,scur,smax,slim,stot,bin,bout,dreq,dresp,ereq,econ,eresp,wretr,wredis,status,weight,act,bck,chkfail,chkdown,lastchg,downtime,qlimit,pid,iid,sid,throttle,lbtot,tracked,type,rate,rate_lim,rate_max,check_status,check_code,check_duration,hrsp_1xx,hrsp_2xx,hrsp_3xx,hrsp_4xx,hrsp_5xx,hrsp_other,hanafail,req_rate,req_rate_max,req_tot,cli_abrt,srv_abrt,comp_in,comp_out,comp_byp,comp_rsp,lastsess,last_chk,last_agt,qtime,ctime,rtime,ttime,
stats,FRONTEND,,,0,0,3000,0,0,0,0,0,0,,,,,OPEN,,,,,,,,,1,2,0,,,,0,0,0,0,,,,0,0,0,0,0,0,,0,0,0,,,0,0,0,0,,,,,,,,
stats,BACKEND,0,0,0,0,300,0,0,0,0,0,,0,0,0,0,UP,0,0,0,,0,282917,0,,1,2,0,,0,,1,0,,0,,,,0,0,0,0,0,0,,,,,0,0,0,0,0,0,-1,,,0,0,0,0,
Frontend_Name,FRONTEND,,,0,0,3000,0,0,0,0,0,0,,,,,OPEN,,,,,,,,,1,3,0,,,,0,0,0,0,,,,,,,,,,,0,0,0,,,0,0,0,0,,,,,,,,
Backend_Name,app-server4,0,0,0,0,,0,0,0,,0,,0,0,0,0,UP,1,1,0,1,0,282917,0,,1,4,1,,0,,2,0,,0,L4OK,,12,,,,,,,0,,,,0,0,,,,,-1,,,0,0,0,0,
Backend_Name,app-server3,0,0,0,0,,0,0,0,,0,,0,0,0,0,MAINT,1,0,1,1,2,2,23,,1,4,2,,0,,2,0,,0,L4OK,,11,,,,,,,0,,,,0,0,,,,,-1,,,0,0,0,0,
Backend_Name,BACKEND,0,0,0,0,300,0,0,0,0,0,,0,0,0,0,UP,1,1,0,,0,282917,0,,1,4,0,,0,,1,0,,0,,,,,,,,,,,,,,0,0,0,0,0,0,-1,,,0,0,0,0,
…

Once it is confirmed from Application supprt colleagues, that machine is out of maintenance node and working properly again to reenable it:

3. Enable backend app-server3

# echo "enable server bk_BACKEND_ROUNDROBIN/app-server3" | socat unix-connect:/var/lib/haproxy/stats stdio

4. Check backend situation again

# echo "show stat" | socat stdio /var/lib/haproxy/stats
# pxname,svname,qcur,qmax,scur,smax,slim,stot,bin,bout,dreq,dresp,ereq,econ,eresp,wretr,wredis,status,weight,act,bck,chkfail,chkdown,lastchg,downtime,qlimit,pid,iid,sid,throttle,lbtot,tracked,type,rate,rate_lim,rate_max,check_status,check_code,check_duration,hrsp_1xx,hrsp_2xx,hrsp_3xx,hrsp_4xx,hrsp_5xx,hrsp_other,hanafail,req_rate,req_rate_max,req_tot,cli_abrt,srv_abrt,comp_in,comp_out,comp_byp,comp_rsp,lastsess,last_chk,last_agt,qtime,ctime,rtime,ttime,
stats,FRONTEND,,,0,0,3000,0,0,0,0,0,0,,,,,OPEN,,,,,,,,,1,2,0,,,,0,0,0,0,,,,0,0,0,0,0,0,,0,0,0,,,0,0,0,0,,,,,,,,
stats,BACKEND,0,0,0,0,300,0,0,0,0,0,,0,0,0,0,UP,0,0,0,,0,282955,0,,1,2,0,,0,,1,0,,0,,,,0,0,0,0,0,0,,,,,0,0,0,0,0,0,-1,,,0,0,0,0,
Frontend_Name,FRONTEND,,,0,0,3000,0,0,0,0,0,0,,,,,OPEN,,,,,,,,,1,3,0,,,,0,0,0,0,,,,,,,,,,,0,0,0,,,0,0,0,0,,,,,,,,
Backend_Name,app-server4,0,0,0,0,,0,0,0,,0,,0,0,0,0,UP,1,1,0,1,0,282955,0,,1,4,1,,0,,2,0,,0,L4OK,,12,,,,,,,0,,,,0,0,,,,,-1,,,0,0,0,0,
Backend_Name,app-server3,0,0,0,0,,0,0,0,,0,,0,0,0,0,UP,1,0,1,1,2,3,58,,1,4,2,,0,,2,0,,0,L4OK,,11,,,,,,,0,,,,0,0,,,,,-1,,,0,0,0,0,
Backend_Name,BACKEND,0,0,0,0,300,0,0,0,0,0,,0,0,0,0,UP,1,1,1,,0,282955,0,,1,4,0,,0,,1,0,,0,,,,,,,,,,,,,,0,0,0,0,0,0,-1,,,0,0,0,0,
…

You should see the backend enabled again.

NOTE:
If you happen to get some "permission denied" errors when you try to send haproxy commands via the configured haproxy status this might be related to the fact you have enabled the socket in read only mode, if that is so it means the haproxy cannot be written to and therefore you can only read info from it with status commands, but not send any write operations to haproxy via unix socket.

One example haproxy configuration that enables haproxy socket in read only looks like this in haproxy.cfg:

stats socket /var/lib/haproxy/stats

To make the haproxy socket read / write mode, for root superuser and some other users belonging to admin group 'adm', you should set the haproxy.cfg to something like:

stats socket /var/lib/haproxy/stats-qa mode 0660 group adm level admin

or if no special users with a set admin group needed to have access to socket, use instead config like:

stats socket /var/lib/haproxy/stats-qa.sock mode 0600 level admin

Tags: case, checks, command, echo, emergency, example scenario, Haproxy Enable Disable Application, lib, note, server, servers, side, status, var
Posted in Linux, Linux and FreeBSD Desktop, System Administration | No Comments »

☩ Walking in Light with Christ – Faith, Computing, Diary

Archive for the ‘System Administration’ Category

Zabbix Power Shell PS1 script to write zero or one if string is matched inside log file

How to Copy / Backup Windows USB drive from one USB to a second

Clone Your USB Drive with ImageUSB on Windows 10

How to view WIFI Passwords for Profile from command line with netsh on Windows 10

Daily Bible quote

GET ARTICLE UPDATES

Useful blog? Help it:

Links to Other Places

Recent Posts

Ads

Categories

About Myself

Recent Comments

Top Post Views

blogtopsites