Posts Tagged ‘empty’

Zabbix Power Shell PS1 script to write zero or one if string is matched inside log file

Monday, December 2nd, 2024

How to Install and Configure Zabbix Server and Client on Rocky Linux 9 - Cộng Đồng Linux

At work we had setup zabbix log file processing for few servers for a service that is doing a Monitoring Health Checks for a a special application via an encrypted strong encrypted tunnel. The app based on the check reports whether the remote side has processed data or not.
As me and my team are not maintainers of the zabbix-server where the zabbix-agents are sending the data, there is a multiple content of data being sent in simply "" empty strings via a zabbix Item setup. Those empty strings however gets stored in the zabbix-server database and since this check is made frequently about 500 hundred records of empty string lines are being written to the zabbix server, we got complaint by the zabbix adminsitrators, that we have to correct our Monitoring setup to not flood the zabbix-server.

Since zabbix cannot catch up the "" empty string and we cannot supress the string from being written in the Item, we needed a way to change the monitoring so that the configured Application check returns 1 (on error) and 0 (on success).

Zabbix even though advanced has a strange when zabbix log[] function, e.g. 

log[/path/to/log,,,,skip]

log function, used to analyze a log file and cut out last or first lines of a file simmilar to UNIX's  head and tail over log files this is described in the zabbix log file monitoring here . If a string is matched it can return string 1, but if nothing gets matched the result is empty string "" and this empty string cannot be used in a way to analyze the data with Item is used.

There is plenty of discussions online for this weird behavior and many people do offer different approaches to solve the strange situation, but as we have tried with our colleagues sys admins  none of those really worked out.

Thus we decided to use the classical way to work around, e.g. to simply use a powershell script that would check a number of lines inside a provided log file analyze if a string gets found and print out value of "1" if the string is matched or "0" "if not and this PS1 script to be set to run via a standard zabbix userparameter script.

This worked well, as all of us are mainly managing Linux systems, and we don't have enough knowledge on powershell we have used our internal Aartificial Intelligence (AI) clone tool to LibreChat – A free and open source ChatGPT clone.

LibreChat includes OpenAI's models, but also others — both open-source and closed-source — and its website promises "seamless integration" with AI services from OpenAI, Azure, Anthropic, and Google — as well as GPT-4, Gemini Vision, and many others. ("Every AI in one place," explains LibreChat's home page.) Plugins even let you make requests to DALL-E or Stable Diffusion for image generations. (LibreChat also offers a database that tracks "conversation state" — making it possible to switch to a different AI model in mid-conversation…)

$logfile = "C:\path\to\your\logfile.log"
$searchString = "-1"
 
# Get the last 140 lines
$lines = Get-Content $logfile -Tail 140
 
# Filter lines containing the search string
$found = $lines | Where-Object { $_ -match [regex]::Escape($searchString) }
 
# Output found lines or 0 if none were found
if ($found) {
    $found | ForEach-Object { $_ }
} else {
    Write-Host 0
}

You can download and the return_zero_or_one-if-string-matches-in-log-powershell.ps1 script here

Fix Oracle virtualbox Virtual Machine inacessible error in Linux / Windows Host OS

Monday, November 2nd, 2020

Fix Oracle virtualbox Virtual Machine inacessible error in Linux / Windows Host OS

Say you have installed a Virtual Machine on a Host OS be it Windows 7 / 10 or some GNU / Linux / FreeBSD OS and suddenly after a PC shutdown / restart the Virtual machine shows as it couldn't be run by Ora Virtualbox anymore with a message of VM Machine "inaccessible", what is causing this and how to fix it?

Virtualbox-virtual-machine-in-inaccessible-state-screenshot
This error is usually caused by the Guest Operating System's forceful behavior or an OS system crash such as sudden hang due to kernel error or due to sudden electricity drop. Whatever the reason if improper sending of termination process signals to Oracle Vbox are not properly handled this is causing the VBox to not recreate its own files and kill (close) the application of Oracle Virtualbox program in gracious way.

On Windows Virtualization Host OS the common files that needs to be tampered by Virtualbox are found under Virtualbox VMs\Name-of-VM:

virtualbox-vm-inacessible-_error-screenshot.png

For example if you have a Ubuntu installed

Virtualbox-virtual-machine-in-inaccessible-state-screenshot.png
C:\Users\\VirtualBox VMs\Ubuntu

If you have CentOS7 installed instead it will be under

C:\Users\\VirtualBox VMs\CentOS7

Usually under this dir you will find files which are with .vbox extension.

The virtual box files with extension .vbox contain metadata the virtualbox hypervisor requires to resolve the guest virtual OS' configuration.
If the main .vbox file is corrupted (i.e. reporting that it is empty) then use the backup .vbox-prev file to recover the contents of the original file.

How to solve the Virtualbox inacessible weird error:

Usually under the Virtualbox VMs\VM_name you'll find a directory / filestructure like:
 

Logs/
Snapshot/
VM_Name.vbox
VM_Name.vbox-prev
VM_Name.vbox-tmp

 

 

1. Considering .vbox is empty Rename the empty .vbox files a temporary name (e.g. rename VM_Name.vbox to VM_Name-empty.vbox). If you're unsure whether it is empty you can check by opening the file in a text editor and make sure it is empty.

2. Then make a copy of the backup file VM_Name.vbox-prev, where the copy will have the same name as the original but with the word "copy" appended to it (i.e. VM_Name.vbox-prev is renamed to VM_Name_copy.vbox-prev) as well as copy VM_Name.vbox-tmp to VM_Name_copy.vbox-tmp.

! Note that it is important to retain the original backup .vbox-prev file it should not be altered or itself renamed.

3. Now go rename the copy of the newly created .vbox-prev file VM_Name.vbox-prev to the name of the empty .vbox file (VM_Name.vbox).

Now that this is done you may add the .vbox file (guest os) back into the VBOX hypervisor.

If for some reason this did not work and you have a proper working copy of the Virtual Machine under VM_Name.vbox-tmp another approach to try is:
 

1.  Go to your Virtualbox folder i.e. C:\Users\hipo\VirtualBox VMs\Ubuntu

2. Check for file extensions files Ubuntu.vbox-tmp or Ubuntu.vbox-prev needed are there.

3. Overwrite Ubuntu.vbox file with the -tmp one, Just rename file from Ubuntu.vbox-tmp to Ubuntu.vbox

4.  Exit from Virtual Machine and Power it on again.

Hopefully this should recovered the state and snapshot of the "inaccessible" guest VM and
you should now see error gone away and VM will work as before.

 

Delete empty files and directories under directory tree in Linux / UNIX / BSD

Wednesday, October 21st, 2020

delete-empty-directories-and-files-freeup-inodes-by-empty-deleting-directoriers-or-files

Sometimes it happens that you end up on your server with a multiple of empty files. The reason for that could be different for example it could be /tmp is overflown with some session store files on a busy website, or due to some programmers Web executed badly written PHP / Python / Perl / Ruby code bug or lets say Content Management System ( CMS ) based website based on WordPress / Joomla / Drupal / Magento / Shopify etc. due to a broken plugin some specific directory could get filled up with plenty of meaningless empty files, that never gets wiped out if you don't care. This could happen if you offer your users to share files online to a public sharing service as WebFTP and some of the local hacked UNIX user accounts decides to make you look like a fool and run an endless loop to create files in your Hard Drive until your small server HDD filesystem of few terabytes gets filled up with useless empty files and due to full inode count on the filesystem your machine running running services gets disfunctional …

Hence on servers with shared users or simply webservers it is always a good idea to keep an eye on filesystem used nodes count by system are and in case if notices a sudden increase of used FS inodes as part of the investigation process on what caused it to check the amount of empty files on the system attached SCSI / SSD / SAS whatever drive.
 

1. Show a list of free inodes on server


Getting inodes count after logged is done with df command

root@linux-server:~# df -i
Filesystem        Inodes   IUsed     IFree IUse% Mounted on
udev             2041464     516   2040948    1% /dev
tmpfs            2046343    1000   2045343    1% /run
/dev/sdb2       14655488 1794109  12861379   13% /
tmpfs            2046343       4   2046339    1% /dev/shm
tmpfs            2046343       8   2046335    1% /run/lock
tmpfs            2046343      17   2046326    1% /sys/fs/cgroup
/dev/sdc6        6111232  6111232   0   100% /var/www
/dev/sda1       30162944 3734710  26428234   13% /mnt/sda1
/dev/sdd1      122093568 8011342 114082226    7% /backups
tmpfs            2046343      13   2046330    1% /run/user/1000

 

2. Show all empty files and directories count

 

### count empty directories ### root@linux-server:~# find /path/ -empty -type d | wc -l

### count empty files only ### root@linux-server:~# find /path/ -empty -type f | wc -l

 

3. List all empty files in directory or root dir

As you can see on the server in above example the amount of inodes of empty inodes is depleted.
The next step is to anylize what is happening in that web directory and if there is a multitude of empty files taking up all our disk space.
 

root@linux-server:~# find /var/www -type f -empty > /root/empty_files_list.txt


As you can see I'm redirecting output to a file as with the case of many empty files, I'll have to wait for ages and console will get filled up with a data I'll be unable to easily analyze

If the problem is another directory in your case, lets say the root dir.

root@linux-server:~#  DIR='/';
root@linux-server:~# find $DIR -type f -empty > /root/empty_files_list.txt

4. Getting empty directories list


Under some case it might be that the server is overflowed with empty directories. This is also a thing some malicious cracker guy could do to your server if he can't root the server with some exploit but wants to bug you and 'show off his script kiddie 3l337 magic tricks' :). This is easily done with a perl / python or bash shell endless loop inside which a random file named millions of empty directories instead of files is created.

To look up for empty directories hence use:

root@linux-server:~# DIR='/home';
root@linux-server:~# find  $DIR . -type d -empty > /root/empty_directories_list.txt

 

5. Delete all empty files only to clean up inodes

Deletion of empty files will automatically free up the inodes occupied, to delete them.

root@linux-server:~# cd /path/containing/multiple/empty-dirs/
root@linux-server:~# find . -type f -empty -exec rm -fr {} \;

 

6. Delete all empty directories only to clean up inocommanddes

root@linux-server:~# find . -type d -empty -exec rm -fr {} \;

 

7. Delete all empty files and directories to clean up inodes

root@linux-server:~# cd /path/containing/multiple/empty-dirs/
root@linux-server:~# find . -empty -delete

 

8. Use find + xargs to delete if files count to delete is too high

root@linux-server:~# find . -empty | xargs rm -r


That's all folks ! Enjoy now your Filesystem to have retrieved back the lost inodes from the jump empty files or directories.

Happy cleaning  🙂

Howto delete empty directories in GNU /Linux with find linux command

Wednesday, April 7th, 2010

Ever wondered how you can delete all the empty directories in Linux?
I bet you did, there are many ways to achieve that in GNU/Linux, however here is one way you might go:
First it is probably a good idea to list the empty directories and examinethe empty directories before you take the next step and execute a command to delete them:

find . -depth -type d -empty

Now after you take a close look in the directories, next step to partake is delete the directories.

find . -depth -type d empty -exec rmdir {} ;

Be aware that in the above examples, the first one would list all directories in your current directory in which you
execute the command, the second example will delete all the empty directories starting from your current directory unto thedeepest located empty directory in the directory tree.