configured Archives - ☩ Walking in Light with Christ - Faith, Computing, Diary ☩ Walking in Light with Christ

Posts Tagged ‘configured’

Windows 10 install local Proxy server to Save bandwidth on a slow and limited Mobile Phone HotSpot network Shared connections

Wednesday, August 20th, 2025

If you're running on Internet ISP that is providing via a Internet / Wifi Router device with a 3G / 4G / 5G etc. but your receiving point location is situated somewhere very far in a places like High mountains lets say Rila Mountain or Alps on a very distant places where Internet coverate of Inetner Service Provider is low or very low but you need still to Work / Play / Entertain on the Net frequently.
Hence you will cenrtainly be looking for a ways to Speed Up / Optimize the Internet connectivity somehow.
You cannot do miracles but certainly the daily operations and a pack up of repeating traffic can be achieved by using installing and using simple local proxy server.

The advantages of using a proxy are even more besides the speed up of Internet connection lines, here is the Pros you get by using the proxy:

Using Caches frequently accessed content (e.g., images, scripts, web pages).
Blocks ads and trackers (reduces bandwidth).
Compresses data (if needed)
Can serve multiple local devices if needed.

To save bandwidth on a slow and limited connectivity Internet router or mobile phone hotspot using Windows 10, you can install a local proxy server that:

Here’s a step-by-step guide to set this up:

Install a local caching proxy server on Windows 10 to reduce bandwidth usage over a mobile hotspot.

1. Install Squid (Caching Proxy Server)

Squid is a powerful and widely used open-source caching proxy.

Download Squid for Windows

Download Squid for Windows from:

https://squid.acmeconsulting.it/download (Unofficial, stable build)

or compile it manually (if you're having an own Linux or BSD router that is passing on the traffic)

2. Install Squid Proxy sever on Windows

2.1. Extract or install the downloaded Squid package.

…

2.2. Install it as a Windows Service

Open Command Prompt (Admin) and run:

C:\\Users\\hipo\\Downloads> squid -i

Initialize cache directories:

C:\\Users\\hipo\\Downloads> squid -z

3. Configure Squid Proxy via squid.conf

3.1. Open squid.conf

usually in

C:\\Squid\\etc\\squid\\squid.conf

3.2. Edit key lines:

http_port 3128
cache_dir ufs c:/squid/var/cache 100 16 256
access_log c:/squid/var/logs/access.log
cache_log c:/squid/var/logs/cache.log
maximum_object_size 4096 KB
cache_mem 64 MB

3.3. Allow local access:

acl localnet src 192.168.0.0/16
http_access allow localnet

(Adjust IP ranges according to your network.)

Here's a ready-to-use Squid configuration file optimized for Running on Windows 10:

Caching web content to save bandwidth
Blocking ads and trackers
Allowing local device connections

Location for the squid Config File

The Windows squid installer should have setup the Squid proxy by default inside C:\Squid so the full path to squid.conf should be:
Place this as

squid.conf

in:

C:\\Squid\\etc\\squid\\squid.conf

# BASIC CONFIGURATION
http_port 3128
visible_hostname localhost

# CACHE SETTINGS
cache_mem 128 MB
maximum_object_size 4096 KB
maximum_object_size_in_memory 512 KB
cache_dir ufs c:/squid/var/cache 100 16 256
cache_log c:/squid/var/logs/cache.log
access_log c:/squid/var/logs/access.log

# DNS
dns_nameservers 8.8.8.8 1.1.1.1

# ACLs (Access Control Lists)
acl localhost src 127.0.0.1/32
acl localnet src 192.168.0.0/16
acl Safe_ports port 80 # HTTP
acl Safe_ports port 443 # HTTPS
acl Safe_ports port 21 # FTP
acl CONNECT method CONNECT

# BLOCKED DOMAINS (Ad/Tracking)
acl ads dstdomain .doubleclick.net .googlesyndication.com .googleadservices.com
acl ads dstdomain .ads.yahoo.com .adnxs.com .track.adform.net
http_access deny ads

# SECURITY & ACCESS CONTROL
http_access allow localhost
http_access allow localnet
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access deny all

# REFRESH PATTERNS (Cache aggressively)
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i \.jpg$ 10080 90% 43200
refresh_pattern -i \.png$ 10080 90% 43200
refresh_pattern -i \.gif$ 10080 90% 43200
refresh_pattern -i \.css$ 10080 90% 43200
refresh_pattern -i \.js$ 10080 90% 43200
refresh_pattern -i \.html$ 1440 90% 10080
refresh_pattern . 0 20% 4320

# LOGGING
logfile_rotate 10

4. Start the Squid Win Service from Admin command prompt

C:\Users\hipo> net start squid

5. Test the Proxy

Set the proxy server in your Windows proxy settings:

Go to Settings > Network & Internet > Proxy
Enable Manual proxy setup:

Address: 127.0.0.1

Port: 3128

Browse the web — Squid will now cache content locally.

Make sure

C:\Squid\var\cache

and

C:\Squid\var\logs

exist.

You can expand the ad block list by importing public blocklists. Let me know if you want help with that.

To share this proxy with other local devices, ensure they’re on the same network and allowed via ACL.

6. Block Ads and Save More Bandwidth with the Proxy

You can modify Squid to:

Block ad domains (using

acl

rules or a blacklist)

Limit download sizes

Restrict background updates or telemetry

Example rule to block a domain:

acl ads dstdomain .doubleclick.net .ads.google.com http_access deny ads

7. Use Aternative lightweight Proxy Privoxy (Lightweight filtering proxy)

What is Privoxy?

Privoxy is a lightweight, highly customizable proxy server focused on privacy protection, content filtering, and web page optimization.

Unlike caching proxies (like Squid), Privoxy doesn’t store data locally—but it filters and blocks unnecessary traffic before it even reaches your browser.

7.1. Why Use Privoxy to Speed Up Internet?

Here's how Privoxy helps:

Feature	Benefit
Blocks Ads & Banners	*Reduces page load size and clutter*
Stops Trackers	Prevents background data requests
Filters Pop-ups	Improves usability and safety
Speeds Up Web Browsing	By stripping unwanted content
Low Resource Usage	Works on older or low-spec systems

Privoxy is easier to set up than Squid and usually much more simple and fits well if you want something simpler and more light weight and is also great for ad/tracker blocking.
To install and use it it comes to 4 simple steps

Download from: https://www.privoxy.org/
Install and run it.
Configure browser/system to use proxy lets say on:

127.0.0.1:8118
Customize

config.txt

to add block rules.

7.2. Configure Your Web Browser or System Proxy

Set your browser/system to use the local Privoxy proxy:

Proxy address:

127.0.0.1

Port:

8118

On Windows:

Go to Settings > Network & Internet > Proxy

Enable Manual Proxy Setup

Enter:

Address:

127.0.0.1

Port:

8118

Save

7.3: Enable Privoxy Filtering and Blocking Rules

Privoxy comes with built-in rules for:

Ad blocking
Tracker blocking
Cookie management
Script filtering

You can customize filters in the configuration files via following configs:

Main config:

C:\\Program Files (x86)\\Privoxy\\config

Action files:

C:\\Program Files (x86)\\Privoxy\\default.action

Filter files:

C:\\Program Files (x86)\\Privoxy\\default.filter

7.4. Example to Block All Ads with Privoxy

Look in

default.action

and ensure these are uncommented:

{ +block }

Or add specific ad server domains:

{ +block{Ad Servers} }
.com.doubleclick.net
.ads.google.com
.adnxs.com

You can further use community-maintained blocklists for stronger Ads filtering.

Privoxy does not compress traffic, so to speed up even further with privoxy you might Compress traffic to do so use ziproxy (the http traffic compressor).

Now all your HTTP traffic is routed through Privoxy and you will notice search engines and repeatingly accessed websites pictures and Internet resources such as css / javscript / htmls etc. will give a boost !

Tags: Blocks, Blocks Ads Banners, com, configured, connections, domain, Edit, filtering, Install Squid Proxy, logs, miracles, net, Open Command Prompt Admin, Proxy Privoxy Lightweight, proxy server, requests, Restart, Settings Network Internet Proxy, sourceforge, squid, Use Aternative, var, web content, Windows, Windows Service, www, youtube
Posted in Computer Security, Curious Facts, Windows | No Comments »

Deploying a Server and Managing a 10-Node Linux Infrastructure with Ansible

Friday, May 30th, 2025

File:Ansible Logo.png - Wikimedia Commons

As organizations grow, manually configuring and maintaining servers quickly becomes inefficient, error-prone, and unscalable. Ansible, a powerful open-source automation tool, allows sysadmins and DevOps engineers to automate server provisioning, configuration management, and application deployment across multiple nodes—making it ideal for managing a fleet of Linux servers.

In this article, you’ll learn how to deploy a server using Ansible and manage an infrastructure of 10 Linux servers with ease.

What Is Ansible?

Ansible is a suite of software tools that enables infrastructure as code. It is open-source and the suite includes software provisioning, configuration management, and application deployment functionality.Ansible is an agentless IT automation tool. It uses SSH to connect to remote machines and YAML-based playbooks to define automation tasks. With Ansible, you can:

Install and configure software packages
Enforce configuration consistency
Perform updates across nodes
Deploy applications
Orchestrate complex workflows

Pre-requisites

To follow this guide, you need:

A control node (the machine from which you run Ansible)
10 Linux servers (can be VMs or physical machines)
SSH access from the control node to all servers
Python installed on the target machines (most Linux distros have this by default)

1. Install Ansible on the Control Node

On Ubuntu / Debian Linux

# apt update sudo apt install ansible -y

Or for CentOS/RHEL

# yum install epel-release -y
# yum install ansible -y

Verify installation:

# ansible –version

2. Configure Your Inventory File

Ansible uses an inventory file to define which servers to manage. By default, this is located at /etc/ansible/hosts , but you can also create a custom one.

Create an inventory file hosts.ini

[webservers]
server1 ansible_host=192.168.1.101
server2 ansible_host=192.168.1.102
server3 ansible_host=192.168.1.103
[dbservers]
server4 ansible_host=192.168.1.104
[all:vars]
ansible_user=ansible_user
ansible_ssh_private_key_file=~/.ssh/id_rsa

You can also use hostnames if DNS is configured.

3. Test Connectivity

# ansible -i hosts.ini all -m ping

If everything is set up correctly, you should see pong responses from all servers.

4. Write Your First Playbook (Provisioning Example)

Create a file called server_setup.yml:

—
– name: Provision and configure Linux servers
hosts: all
become: yes
tasks:

– name: Update apt cache (Debian/Ubuntu)
apt:
update_cache: yes
when: ansible_os_family == "Debian"

– name: Install common packages
package:
name:
– curl
– git
– htop
state: present

– name: Ensure Nginx is installed on webservers
apt:
name: nginx
state: present
when: "'webservers' in group_names"

– name: Start and enable Nginx
service:
name: nginx
state: started
enabled: yes
when: "'webservers' in group_names"

5. Run the Playbook

# ansible-playbook -i hosts.ini server_setup.yml

Ansible will SSH into each server, execute the tasks, and return status messages.

6. Scaling to 10+ Servers

Managing a growing infrastructure is simple with Ansible:

Add new servers to hosts.ini
Reuse existing playbooks for setup
Use roles to modularize configuration (e.g., webserver, database, monitoring)
Schedule playbooks using cron or CI/CD pipelines (e.g., GitHub Actions, Jenkins)

7. Sample Ansible Project Structure

Organizing your Ansible project effectively is crucial for scalability and maintainability. Here's a sample recommended directory layout:

ansible-infra/
├── hosts.ini
├── server_setup.yml
├── roles/
│ ├── common/
│ │ ├── tasks/
│ │ │ └── main.yml
│ │ └── files/
│ └── webserver/
│ ├── tasks/
│ │ └── main.yml
│ └── templates/
└── group_vars/
└── all.yml

Key Components:

ansible.cfg: Configuration file defining paths and settings.
inventories/: Directory containing environment-specific inventory files.
playbooks/: Directory for playbooks that define automation tasks.
roles/: Directory for reusable roles, each with its own tasks and variables.
requirements.yml: File listing external roles or collections.

For a detailed explanation of this structure, refer to the Ansible Documentation.

Visual Workflow (text) Diagram

To illustrate the workflow, here's a diagram depicting how Ansible interacts with your infrastructure:

Workflow Steps:

Control Node: You initiate commands from your local machine.
Ansible Core: Ansible connects via SSH to each managed node.
Managed Nodes: Ansible applies configurations as defined in your playbooks and roles.

This workflow ensures consistent and automated management of your infrastructure.

Best Practices

Use roles: Organize playbooks into reusable roles
(ansible-galaxy init myrole)
Maintain version control: Keep playbooks and inventory in Git
Encrypt secrets: Use ansible-vault for secure credentials
Test in staging: Always validate changes before pushing to production

Conclusion

Ansible is a game-changer for managing Linux infrastructure. With a few simple playbooks, you can provision, configure, and maintain your servers consistently and securely.
For a 10-node infrastructure, it offers the perfect balance of simplicity and power—letting you scale efficiently without the overhead of agent-based solutions.

Tags: Ansible, bash, configured, Control Node, create, deploying, Ensure Nginx, file, hosts, infrastructure, ini, installed, linux?, machines, Managed Nodes Ansible, managing, Sample Ansible Project Structure, server, software packages, ssh, state, use, webservers
Posted in Debian, Everyday Life, Linux, Linux Package Management, Various | No Comments »

Debugging Jitsi Meet Server Problems: A Practical Guide

Saturday, April 26th, 2025

Jitsi Meet is a powerful open-source video conferencing platform. But like any real-time communication system, it can run into issues—from video/audio glitches to full-blown connection failures. Debugging Jitsi Meet can be tricky due to its multi-component architecture. This guide walks you through a systematic approach to identify and resolve common server-side issues.

1. Understand the Architecture

Before diving into logs, it's important to understand Jitsi Meet's core components:

Jitsi Meet (Web UI) – The front-end interface.
Jicofo (Focus component) – Manages conference sessions.
Prosody (XMPP Server) – Handles user authentication and signaling.
JVB (Jitsi Videobridge) – Routes video/audio streams.
Nginx or Apache – Web server proxy (often with HTTPS and WebSocket forwarding).

Knowing how these interact helps pinpoint the failing layer.

2. Check Logs in the Right Places

Each component has its own logs. Check them in the following order:

✅ Prosody Logs

Location: /var/log/prosody/prosody.log and prosody.err
Look for: Authentication issues, connection denials, or component registration problems.

✅ Jicofo Logs

Location: /var/log/jitsi/jicofo.log
Look for: Room creation errors, XMPP connection failures, conference creation attempts.

✅ JVB Logs

Location: /var/log/jitsi/jvb.log
Look for: ICE failures, STUN/TURN issues, packet loss, and bridge reachability.

✅ Web Server Logs (Nginx/Apache)

Location (Nginx): /var/log/nginx/error.log and access.log
Look for: HTTP errors (404, 502), WebSocket connection problems.

✅ Browser Console Logs

Tools: Press F12 in browser → Console/Network tabs.
Look for: WebSocket failures, CORS issues, or media permission problems.

3. Common Problems & Fixes

"Failed to join conference"

Cause: Prosody may not be running or not configured correctly.

Fix: Restart Prosody and check domain configuration in /etc/prosody/conf.avail/

No Audio or Video

Usual Cause: Media not reaching the bridge or blocked by firewall

Fix:

Verify JVB is listening on correct ports (UDP 10000).
Check firewall/NAT settings (especially on cloud VMs).
Use tcpdump or ss to check traffic flow.

WebSocket Connection Fails

Usual Cause: Web server (Proxy) misconfiguration.

Fix:

Ensure Nginx is forwarding WebSocket requests to /xmpp-websocket/ .
Add proper proxy settings in nginx.conf

Authentication Not Working

Cause: Misconfigured JWT or internal authentication.

Fix:

Check Prosody's config for authentication method.
If using JWT, verify token structure and shared secret.

4. Use Debugging Tools

Jitsi Meet in debug mode:

Add #config.debug=true to your meeting URL.

ICE Debugging:

Check about:webrtc (Firefox) or WebRTC Internals (Chrome).
Look at ICE candidate gathering and connectivity checks.
Test TURN/STUN:
- Use tools like trickle-ice to validate your server's ICE configuration.

5. Networking and Firewall Checks

Make sure these ports are open:

TCP 443 – HTTPS
UDP 10000 – Media (JVB)
TCP 4443 – (Optional, fallback media)
TCP 5222 – XMPP (if not using BOSH/WebSocket)

# ss -tuln ufw status

6. Component Health Checks

Do
# systemctl status for each main jitsi component services:

# systemctl status prosody # systemctl status jicofo # systemctl status jitsi-videobridge2

Check uptime, errors, or failure restarts.

7. Enable More Verbose Logs

Increase logging levels for deeper debugging:

Prosody: Edit /etc/prosody/prosody.cfg.lua → set log = { ... debug = "*" }.
Jicofo/JVB: Edit /etc/jitsi/jicofo/logging.properties and /etc/jitsi/videobridge/logging.properties
→ change log level to FINE or ALL .

8. Update & Restart Services

Sometimes updates or configs don’t apply until services are restarted:

# apt update && apt upgrade systemctl restart prosody jicofo jitsi-videobridge2 nginx

Final Closure Thoughts

Debugging Jitsi Meet requires a structured approach, start from the user-facing symptoms, trace through each service, and verify network and authentication configurations.
Debug the status of prosody, jicofo and jitsi-videobridge2, check the firewall openings are okay to the jitsi server
With some log analysis and a bit of patience, experimentation and the help of forums or Artificial Intelligence tool like ChatGPT, the Jitsi server errors will get solved.

Tags: about, access, ALL, amp, and, ANY, authentication, Cause Prosody, configured, errors, Firewall Checks, HTTPS, Location Nginx, logging, look, Prosody Logs, proxy settings, status, systemctl, traffic flow
Posted in Linux, Linux and FreeBSD Desktop, Linux Audio & Video, System Administration | No Comments »

How to Install Jitsi Meet on Debian Linux to have your own free software video conferencing secure server

Thursday, April 24th, 2025

jitsi-meet-create-new-room-for-video-meetings-linux

Jitsi Meet is a free, open-source video conferencing platform that allows you to host secure and scalable video calls both using a Mobile Phone / Tablet / PC or any other electronic device for which jitsi client has available port. Jitsi meet is the best free alternative one can get to Rakuten Viber / Facebook (Meta) / Zoom / Apples' Facetime etc.
What makes Jitsi really worthy is it can make your Video streaming communication give you flexibility to keep your communication a little bit private and harder to be captured than if you use the general Video streaming platforms.
Jitsi is also a very simple to use and can be used either with a Desktop Client on Windows / Linux and Mac OS or Smart Phone running Android (Samsung / Huawei etc.) or iOS (iPhones) you can configure to use the Jitsi server or directly via a SSL encryption secured web URL address. The only thing i really don't like about Jitsi is it uses Java and its way of work is cryptic just like it is pretty hard to debug or understand exactly how the software works, as when errors came the usual crazzy Java exceptions are filling the jitsi logs.

In below short guide, I'll try provides a simple step-by-step instructions for installing Jitsi Meet on a Debian-based systems, hoping that anyone can benefit from Jitsi by building his own server.

What you should have before you start buillding your new Jitsi meet server

Before you begin, ensure that your system meets the following requirements:

A fresh installation of Debian 10 (Buster) or newer.
A non-root user with sudo privileges.
A fully updated system.
A domain name pointing to your server's IP address.
OpenJDK 11 installed.

To get a better understanding on how Jitsi meet works it is worthy to take a quick look on Jitsi Architectural diagram:

1. Update Your System

Start by updating your system's package list and upgrading existing packages:

# apt update sudo apt upgrade -y

2. Install Required Dependencies

Install the necessary packages for adding repositories and managing keys:

# apt install apt-transport-https curl gnupg2 -y

3 Add Jitsi Repository

Add the Jitsi repository key to your system:

# curl https://download.jitsi.org/jitsi-key.gpg.key | sudo gpg --dearmor -o /usr/share/keyrings/jitsi-keyring.gpg

Then, add the Jitsi repository:

# echo "deb [signed-by=/usr/share/keyrings/jitsi-keyring.gpg] https://download.jitsi.org stable/" | sudo tee /etc/apt/sources.list.d/jitsi-stable.list > /dev/null

Update your package list to include the Jitsi repository into apt database:

# apt update

4. Install Jitsi Meet

Install the Jitsi Meet package:

# apt install jitsi-meet -y

During installation, you'll be prompted to:

Enter the hostname: Provide your domain name (e.g., meet.example.com ).
Choose SSL certificate: Select "Generate a new self-signed certificate" or "Obtain a Let's Encrypt certificate" if you have a valid domain.Jitsi Scaleway

If you opt for Let's Encrypt, ensure that ports 80 and 443 are open on your firewall.

5. Configure Firewall openings

If you have already a configured firewall to filter out traffic, open the necessary ports to allow traffic to your Jitsi Meet server from your router or entry firewall device as well as on the Linux itself:

Allow access to SSH server

# ufw allow 22/tcp

Allow access to HTTP unecrypted to Jitsi meet server

# ufw allow 80/tcp # ufw allow 443/tcp

Allow access necessery for proper operation of Jitsi VideoBridge (port range 10000 to 20000)

# ufw allow 10000:20000/udp # ufw enable

Verify the firewall status is Okay

# ufw status

6. Access Jitsi Meet in a browser

Open a web browser and navigate to your server's domain or IP address:

https://meet.your-custom-domain-or-IP.com

Hopefully all is okay and You should see the Jitsi Meet interface, where you can start or join a meeting.

7. Secure Conference Creation (Optional extra)

By default, anyone can create a conference. To restrict this:

Install and configure Prosody for authentication.
For those who don't know Prosody is a modern XMPP communication server
Set up secure domains and configure authentication settings.

For detailed instructions, refer to the Jitsi DevOps Guide.

Conclusion

Now You should have successfully installed Jitsi Meet on your Debian server.
Installing to Ubuntu and Redhat OSes such as Fedora or Redhat based distros should be not much difrerent from on this guide, except you have to use
the correct RPM repositories.

Now you can further now host secure video conferences using your own infrastructure and have an increased privacy and perhaps be more calm that the CIA or Mussat, MI6 / FSB might be not spying your Video conference talks (except if they don't already do it on an OS level which most likely the case but this doesn't matter. :).

For advanced configurations and features, consult the Jitsi Handbook and the Jitsi DevOps Guide.Jitsi

That's all folks Enjoy !

Tags: about, access, Access Jitsi Meet, bash, Configure Firewall, configured, domain name, firewall, free software, host, Install Jitsi Meet, ip, list, packages, server, Set, sudo, system, video conferences, web browser
Posted in Linux, Linux and FreeBSD Desktop, Linux Audio & Video | No Comments »

How to log multiple haproxy server instance processes on single server in seperate files with rsyslog filters

Monday, February 3rd, 2025

Lets say you want to have 2 separates instances of haproxy and log the output to separate files, how this can be achived?

In this article, i'll tell in few easy steps how to enable multiple haproxy server instances created on the same Linux server / VPS or docker container to run and log its served content in separate log files without using separate file logging handlers "local"s.
The task might be helpful for people who are involved with DevOps and has to route separate proxy traffic on same linux machine.

Lets say you have the following haproxy process instances running with separate haproxy configs:

1. haproxy
2. haproxy_worker2
3. haproxy_worker3

List of processes on the Linux host would looks like that.

[root@linux-server rsyslog.d]# ps -ef|grep -i hap
root 1151275 1147138 0 11:58 pts/2 00:00:00 grep –color=auto -i hap
root 1835200 1 0 Jan30 ? 00:00:00 /usr/sbin/haproxy -Ws -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid
haproxy 1835203 1835200 0 Jan30 ? 00:10:41 /usr/sbin/haproxy -Ws -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid
root 1835216 1 0 Jan30 ? 00:00:00 /usr/sbin/haproxy_worker2 -Ws -f /etc/haproxy/haproxy_worker2.cfg -p /run/haproxy_worker2.pid
haproxy 1835219 1835216 0 Jan30 ? 00:02:46 /usr/sbin/haproxy_worker2 -Ws -f /etc/haproxy/haproxy_worker2.cfg -p /run/haproxy_worker2.pid
root 1835216 1 0 Jan30 ? 00:00:00 /usr/sbin/haproxy_worker3 -Ws -f /etc/haproxy/haproxy_worker3.cfg -p /run/haproxy_worker3.pid
haproxy 1835219 1835216 0 Jan30 ? 00:02:46 /usr/sbin/haproxy_worker3 -Ws -f /etc/haproxy/haproxy_worker3.cfg -p /run/haproxy_worker3.pid

Question is how to log the 3 haproxies passed through configured connection IP and frontend / backend outputs to separate files

/var/log/haproxy.log , /var/log/haproxy_worker2.log and /var/log/haproxy_worker3.log

To achieve the task, you will need to set-up 3 rsyslog config files name it according to your preferences and make sure no other rsyslog
file with haproxy related configuration does not mess up with the configs (e.g. is not having a config start number NUMBER_file.conf prior to the below created files.

Then create lets say 49_haproxy.conf and 50_haproxy_worker2.conf and 51_haproxy_worker3.conf

[root@linux-server rsyslog.d]# cat 48_haproxy.conf
#$ModLoad imudp
$UDPServerAddress 127.0.0.1
$UDPServerRun 514
#2022/02/02: HAProxy logs to local6, save the messages
# Template to include only the timestamp in HAProxy logs
template(name="HaproxyTimestampOnly" type="string" string="%timegenerated% %msg:::drop-last-lf%\n")
local6.* /var/log/haproxy.log;HaproxyTimestampOnly
# Apply the template to HAProxy prod port mapping logs
#if $programname startswith 'haproxy[' then /var/log/haproxy.log;HaproxyTimestampOnly
& stop

[root@linux-server rsyslog.d]# cat 50_haproxy_worker2.conf
$ModLoad imudp
$UDPServerAddress 127.0.0.1
$UDPServerRun 514
# Template to include only the timestamp in HAProxy logs
template(name="HaproxyTimestampOnly" type="string" string="%timegenerated% %msg:::drop-last-lf%\n")

# Apply the template to HAProxy prod port mapping logs
if $programname startswith 'haproxy_worker2' then /var/log/haproxy_worker2.log;HaproxyTimestampOnly

[root@linux-server rsyslog.d]# cat 51_haproxy_worker3.conf
$ModLoad imudp
$UDPServerAddress 127.0.0.1
$UDPServerRun 514
# Template to include only the timestamp in HAProxy logs
template(name="HaproxyTimestampOnly" type="string" string="%timegenerated% %msg:::drop-last-lf%\n")

# Apply the template to HAProxy prod port mapping logs
if $programname startswith 'haproxy_worker3' then /var/log/haproxy_worker3.log;HaproxyTimestampOnly

Those rsyslog configs permissions has to be as follows:

[root@linux-server home]# ls -al /etc/rsyslog.d/48_haproxy.conf
-rw-r–r– 1 root root 488 Jan 30 12:44 /etc/rsyslog.d/48_haproxy.conf
[root@linux-server home]# ls -al /etc/rsyslog.d/50_haproxy_worker2.conf
-rw-r–r– 1 root root 379 Jan 30 12:45 /etc/rsyslog.d/50_haproxy_worker2.conf
[root@linux-server home]# ls -al /etc/rsyslog.d/51_haproxy_worker2.conf
-rw-r–r– 1 root root 379 Jan 30 12:45 /etc/rsyslog.d/51_haproxy_worker2.conf

The permissions for files to log the haproxy has to be as so:

[root@linux-server home]# ls -al /var/log/haproxy.log
-rw-r—– 1 haproxy haproxy 5014349 Feb 3 12:11 /var/log/haproxy.log
[root@linux-server home]# ls -al /var/log/haproxy_worker2.log
-rw-r—– 1 root root 728139 Feb 3 12:11 /var/log/haproxy_worker2.log
[root@linux-server home]# ls -al /var/log/haproxy_worker3.log
-rw-r—– 1 root root 728139 Feb 3 12:11 /var/log/haproxy_worker3.log

To make the changes take affect restart consequentially rsyslog first and then the 3 haproxy instances:

[root@linux-server home]# systemctl restart rsyslog
[root@linux-server home]# systemctl restart haproxy
[root@linux-server home]# systemctl restart haproxy2
[root@linux-server home]# systemctl restart haproxy3

Go on and check the logs that everything comes in from the haproxys running the same server into the separate files:

[root@linux-server home]# tail -f /var/log/haproxy.log /var/log/haproxy_worker2.log /var/log/haproxy_worker3.log
…

Hope this has helped someone out there looking to solve on how to log multiple haproxy instances on the same servers into separate files.

That's all folks. Enjoy!

Tags: configured, haproxy, How to, root root, rsyslog, sbin, separate, server instances, template, var
Posted in Everyday Life, Haproxy, Linux | No Comments »

How to split large files in Windows via split command line and File Archive GUI tool easily

Tuesday, October 22nd, 2024

Moving around a very large files especially Virtualbox Virtual Machines or other VM formats between Windows host and OneDrive might be a problem due to either Azure Cloud configured limitations, or other reasons that your company Domain Administrator has configured, thus if you have to migrate your old Hardware Laptop PC Windows 10 to a newer faster better Harware / Better Performance Notebook Computer with Windows 11 and you still want to keep and move your old large files in this short and trivial article, will explain how.

The topic is easily and most of novice sysadmins should have already be faced to bump into something like this but anyways i found useful to mention about Git for Windows, as it is really useful too thus wrote this small article.

The moved huge files, in my case an experimental Virtual Machines Images which I needed to somehow migrate on the new Freshly installed Windows laptop, the Large files were 40 / 80 etc. Gigabytes or whatever large amount of files from your PC to the Cloud Onedrive and of course the most straight forward thing i tried was to simply add the file for inclusion into the Onedrive storage (via OneDrive tool setup interface), however this file, failed due to OneDrive Cloud file format security limitations or Antivirus solutions configured to filter out the large file copying or even a prohibition to be able to include any kind of Virtual Machines ISOs straight into the cloud.

With this big files comes the question:

How to copy the Virtual Machines from your Old Hardware Laptop to the Cloud (without being able to use an external SSD Hard Drive or a USB SSD Flash drive, due to Domain policy configured for your windows to be unable to copy to externally connected Drive but only to read from such.) ?

Here are few sample approaches to do it both from command line (useful if you have to repeat the process or script it and deploy to multiple hosts) or for single hosts via an Archiver tool:

1. Using split command Git for Windows (Bash) MINGW64 shell

Download Git for Windows – https://git-scm.com/download install it and you will get the MINGW64 bash for Windows executable.

Run it either invoke bash command from command line or trigger Windows Run command prompt (Windows button + R) and type full path to executable

C:\Program Files\Git\git-bash.exe

Git-for-Windows-bash-for-windows-MINGW64-windows-11-screenshot

Use the integrated program split and to cut it into pieces use:

# split MyVeryLargeFileVM.vdi -b 800m

To split the .VDI virtualbox file to lets say 5 Gigabite pieces:

# split MyVeryLargeFile.vdi -b 5g

The output files will be named pieces will be named as in a normal UNIX / GNU split command in the format and each piece of 5GB will be named like:

xaa
xab
xac
xad

If you want to get a more meaningful name for the spilitted files you can set a generated split file prefix with suffixes to be 5 digits long:

# split MyVeryLargeFile.vdi MyVeryLargeFileVM-parts_ -b 5g -d -a 5

the -d flag for using numerical suffixes (instead of default aa, ab, ac, etc…),
and the option -a 5 to tell it I want the suffixes to be 5 digits long:

2. Split large files by Archiving them with Winrar (ShareWare) tool

If you have already Winrar installed and you don't want to bother with too much typing from the command line, You can use good old WinRAR as a file splitter/joiner as well.

To split a file into smaller files, select "Store" as the compression method and enter the desired value (bytes) into "Split to volumes" box.
This way you can have split files named as filename.part1.rar, filename.part2.rar, etc.

WinRAR_cut-split-large-files-into-pieces-screenshot-Windows

3. Split files with 7-Zip (FreeWare)

Assuming you have the 7-Zip installed on the PC, you can do the archiving of the Big file to a smaller pieces one, you can create the splitted file from 7Zip interfaces menus:

Or directly cut the single file into multiple volumes, directly from Windows Explorer by Selecting the file and using fall down menus :

Sum it up what learned ?

What we learned is how to cut large files into multiple single consequential ones for easy copy between Network sides, via both Git 4 Windows and manual copy paste of parted multiple files to OneDrive / DropBox / pCloud or Google Drive.
There is a plenty of other approaches to take as there is also file GUI tools, besides using GNU Win / Gnu Tools for Windows or Cygwin / Gsplit GUI tool or some kind of the many Archiver toolsavailable for Windows, another option to split the large files is to use a bunch of PowerShell and Batch scripts written that can help you do the file split for both binaries files or Text files. but i'll stop here as I believe that is pretty much enough for most basic needs.

Tags: AA, about, administrator, amount, and, another, Azure Cloud, case, Cloud Onedrive, configured, copy, due, file, How to, interesting, migrate, simply, thing, unable, Virtualbox Virtual Machines
Posted in Everyday Life, System Administration, Windows | No Comments »

How to do a port redirect to localhost service with socat or ncat commands to open temporary access to service not seen on the network

Friday, February 23rd, 2024

You know sometimes it is necessery to easily and temporary redirect network TCP ports to be able to be accessible from Internal DMZ-ed Network via some Local Network IP connection or if the computer system is Internet based and has an external "'real" Internet Class A / B address to be reachable directly from the internet via lets say a modern Internet browser such as Mozilla Firefox / Google Chrome Browser etc.

Such things are easy to be done with iptables if you need to do the IP redirect permanent with Firewall rule changes on Linux router with iptables.
One way to create a TCP port redirect using firewall would include few iptable rules like for example:

1. Redirect port traffic from external TCP port source to internal one

# iptables -t nat -I PREROUTING -p tcp –dport 10000 -j REDIRECT –to-ports 80
# iptables -t nat -I OUTPUT -p tcp -o lo –dport 10000 -j REDIRECT –to-ports 80
# iptables -t nat -A OUTPUT -o lo -d 127.0.0.1 -p tcp –dport 80 -j DNAT –to-destination 192.168.0.50:10000
# iptables -t nat -I OUTPUT –source 0/0 –destination 0/0 -p tcp –dport 80 -j REDIRECT –to-ports 10000

Then you will have 192.168.00.50:10000 listener (assuming that the IP is already configured on some of the host network interface, plugged in to the network).

But as messing up with the firewall is not the best thing to do especially, if you need to just temporary redirect external listener port to a service configured on the server to only run on TCP port on loopback address 127.0.0.1, you can do it instead with another script or command for simplicy.

One simple way to do a port redirect on the fly on GNU / Linux or FreeBSD / OpenBSD is with socat command.

Lets say you have a running statistics of a web server Apache / Nginx / Haproxy frontend / backend statistics or whatever kind of web TCP service on port 80 on your server and this interface is on purpose configured to be reachable only on localhost interface port 80, so you can either access it by creating an ssh tunnel towards the service on 127.0.0.1 or by accessing it by redirecting the traffic towards another external TCP port, lets say 10000.

Here is how you can achieve

2. Redirect Local network accessible IP on all configured Server network interfaces port 10000 to 127.0.0.1 TCP 80 with socat

# socat tcp-l:10000,fork,reuseaddr tcp:127.0.0.1:80

If you need to access later the redirected port in a Browser, pick up the machine first configured IP and open it in a browser (assuming there is no firewall filter prohibiting access to redirected port).

root@pcfreak:~# ifconfig eth0
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 109.104.212.130 netmask 255.255.255.0 broadcast 109.104.212.255
ether 91:f8:51:03:75:e5 txqueuelen 1000 (Ethernet)
RX packets 652945510 bytes 598369753019 (557.2 GiB)
RX errors 0 dropped 10541 overruns 0 frame 0
TX packets 619726615 bytes 630209829226 (586.9 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

Then in a browser open http://102.104.212.130 or https://102.104.212.130 (depending on if remote service has SSL encryption enabled or not) and you're done, the configured listener Server service should pop-up on the screen.

3. Redirect IP Traffic from External IP to Localhost loopback interface with netcat ( ncat ) swiss army knife hackers and sysadmins tool

If you need to redirect lets say TCP / IP port 8000 to Port a server local binded service on TCP 80 with ncat, instead of socat (if lets say socat is not pre-installed on the machine), you can do it by simply running those two commands:

[root@server ~]# mkfifo svr1_to_svr2
[root@server ~]# ncat -vk -l 8000 < svr1_to_svr2 | ncat 127.0.0.1 80 > svr1_to_svr2
Ncat: Version 7.92 ( https://nmap.org/ncat )
Ncat: Listening on 0.0.0.0:10000
Ncat: Connection from 10.10.258.39.
Ncat: Connection from 10.10.258.39:51813.
Ncat: Connection from 10.10.258.39.
Ncat: Connection from 10.10.258.39:23179.

I you don't care to log what is going on the background of connection and you simply want to background the process with a one liner command you can achive that with:

[root@server /tmp]# cd tmp; mkfifo svr1_to_svr2; (ncat -vk -l 8000 < svr1_to_svr2 | ncat 127.0.0.1 80 > svr1_to_svr2 &)

Then you can open the Internal Machine Port 80 TCP service on 8000 in a browser as usual.

For those who want a bit of more sophisticated proxy like script I would suggest you take a look at using netcat and a few lines of shell script loop, that can simulate a raw and very primitive proxy with netcat this is exampled in my previous article Create simple proxy server with netcat ( nc ) based utility.

Hope this article is helpful to anyone, there is plenty of other ways to do a port redirect with lets say perl, python and perhaps other micro tools. If you know of one liners or small scripts, that do it please share in comments, so we can learn from each other !

Enjoy ! 🙂

Tags: configured, How to, ifconfig eth0, Internal Machine Port, Mozilla Firefox Google Chrome Browser, Ncat Connection, need, network interface, port, proxy server, say, shell script, swiss army knife, tcp
Posted in Educational, Linux, Linux and FreeBSD Desktop, Networking | No Comments »

Install Zabbix Proxy configure and connect to Zabbix server on CentOS Linux

Thursday, May 4th, 2023

Install Zabbix Proxy configure and connect to Zabbix server on CentOS Linux

1. Why use Zabbix-Proxy hidden advantages of using Zabbix-Proxy ?

Proxy can be used for many purposes and can provide many hidden benefits, just to name few of them:

Offload Zabbix Server when monitoring thousands of devices
Monitor remote locations
Monitor locations having unreliable communications
Simplify maintenance of distributed monitoring
Improved Security (Zabbix server can be restricted to be connectable only by the set of connected Zabbix Proxy / Proxies

advantages-of-using-zabbix-proxy-instead-of-direct-connect-monitored-hosts-to-zabbix-server-diagram

A Zabbix proxy is the ideal solution if you have numerous hosts with multiple slow items that are affecting the performance of the server simply because processes are spending most of the time simply waiting for a response. A proxy can collect information from all hosts using its internal processes and then send raw historical data to the server. The time needed to connect and receive the host response will be on the proxy site, and the server performance will not be affected at all. A proxy just sends raw values to the server, and the server itself does not have to connect to the host to get the data.

2. Install zabbix-proxy-sqlite3 rpm package from Zabbix Official Repositories download page

Zabbix repository provides choice of 3 packages named as follows:

zabbix-proxy-mysql
zabbix-proxy-pgsql
zabbix-proxy-sqlite3

where the last value of the name (after zabbix-proxy) represents database type of the package — MySQL, PostgreSQL and SQLite respectively.

To not bother installing MySQL / PostgreSQL separate database servers, a lightweight choice is to use the sqlite3 db version.
As I prefer zabbix-proxy data to be stored inside a flat database, thus I choose to use zabbix-proxy-sqlite3.

[root@sysadminshelp:/root ]# yum info zabbix-proxy-sqlite3-5.0.31-1.el7.x86_64
Заредени плъгини: fastestmirror
Loading mirror speeds from cached hostfile
* base: mirrors.netix.net
* epel: fedora.ipacct.com
* extras: mirrors.netix.net
* remi: remi.mirror.karneval.cz
* remi-php74: remi.mirror.karneval.cz
* remi-safe: remi.mirror.karneval.cz
* updates: mirrors.netix.net
Инсталирани пакети
Име : zabbix-proxy-sqlite3
Архитект. : x86_64
Версия : 5.0.31
Издание : 1.el7
Обем : 4.4 M
Хранилище : installed
Обобщение : Zabbix proxy for SQLite3 database
URL : http://www.zabbix.com/
Лиценз : GPLv2+
Описание : Zabbix proxy with SQLite3 database support.

My experience to try to install thethe default CentOS RPM package for zabbix-proxy-sqlite3 provided by default
RPM package that came with CentOS did not work as expected and trying to install / configure and use it via

[root@sysadminshelp:/root ]# yum install zabbix-proxy-sqlite3.x86_64 -y

[root@sysadminshelp:/root ]# vi /etc/zabbix/zabbix_proxy.conf

Led me to a nasty errors seen in /var/log/zabbixsrv/zabbix_proxy.log like:

May 1st 2023, 08:42:45.020 zabbix_server cannot set list of PSK ciphersuites: file ssl_lib.c line 1314: error:1410D0B9:SSL routines:SSL_CTX_set_cipher_list:no cipher match
May 1st 2023, 08:42:45.018 zabbix_server cannot set list of PSK ciphersuites: file ssl_lib.c line 1314: error:1410D0B9:SSL routines:SSL_CTX_set_cipher_list:no cipher match
May 1st 2023, 08:42:45.013 zabbix_server cannot set list of PSK ciphersuites: file ssl_lib.c line 1314: error:1410D0B9:SSL routines:SSL_CTX_set_cipher_list:no cipher match
May 1st 2023, 08:42:45.013 zabbix_server cannot set list of PSK ciphersuites: file ssl_lib.c line 1314: error:1410D0B9:SSL routines:SSL_CTX_set_cipher_list:no cipher match
May 1st 2023, 08:42:45.011 zabbix_server cannot set list of PSK ciphersuites: file ssl_lib.c line 1314: error:1410D0B9:SSL routines:SSL_CTX_set_cipher_list:no cipher match

After some googling and reading some threads came upon this one https://support.zabbix.com/browse/ZBXNEXT-3604, there is exmplaed errors preventing the configured zabbix-proxy
to start are caused by the zabbix-proxy-sqlite3 package provided by Redhat (due to openssl incompitability bug or something ).

As one of people in the discussion pointed out the quickest workaround suggested is simply to use the official Zabbix Repository packages for zabbix-proxy-sqlite3, in order to not waste anymore time on this
trivial stuff to install it, simply run:

[root@sysadminshelp:/root ]# rpm -Uvh \
https://repo.zabbix.com/zabbix/5.0/rhel/7/x86_64/zabbix-proxy-sqlite3-5.0.31-1.el7.x86_64.rpm

Alternative way if you seem to not have the machine connected to the internet is simply download the package with wget / lynx / curl / w3m from another machine
that can reach the Internet upload the package via the local LAN or VPN and install it:

# wget https://repo.zabbix.com/zabbix/5.0/rhel/7/x86_64/zabbix-proxy-sqlite3-5.0.31-1.el7.x86_64.rpm …

[root@sysadminshelp:/root ]# rpm -ivh zabbix-proxy-sqlite3-5.0.31-1.el7.x86_64.rpm

NOTE ! Before you install proxy, keep in mind that your proxy version must match the Zabbix server version !

3. Generate a PSK random secret key and set proper permissions for zabbix-proxy directories

[root@sysadminshelp:/root ]# cd /etc/zabbix/

[root@sysadminshelp:/root ]# openssl rand -hex 32 >> /etc/zabbix/zabbix_proxy.psk
[root@sysadminshelp:/root ]# chown root:zabbix zabbix_proxy.psk [root@sysadminshelp:/root ]# vi /etc/zabbix/zabbix_proxy.conf [root@sysadminshelp:/root ]# mkdir -p /var/lib/zabbix-proxy/sqlite3db
[root@sysadminshelp:/root ]# chown -R zabbix:zabbix /var/lib/zabbix-proxy
[root@sysadminshelp:/var/lib/zabbixsrv/sqlite3db]# sqlite3 zabbix_proxy
SQLite version 3.7.17 2013-05-20 00:56:22
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> .databases
seq name file
— ————— ———————————————————-
0 main /var/lib/zabbixsrv/sqlite3db/zabbix_proxy
sqlite> [root@sysadminshelp:/root ]# vi /etc/zabbix_proxy.conf
#DBName=zabbix_proxy
DBName=/var/lib/zabbixsrv/sqlite3db/zabbix_proxy

4. Configure zabbix proxy to be able to connect to Zabbix Server

[root@sysadminshelp:/root ]# vi /etc/zabbix/zabbix_proxy.conf    ############ GENERAL PARAMETERS #################
   ProxyMode=0
   Server=192.168.1.28
   ServerPort=10051
   Hostname=zabbix-proxy
   ListenPort=10051
   SourceIP=10.168.1.55
   LogFile=/var/log/zabbix/zabbix_proxy.log
   LogFileSize=1
   DebugLevel=2
   PidFile=/var/run/zabbix/zabbix_proxy.pid
   DBName=/var/lib/zabbix-proxy/sqlite3db/zabbix_proxy.db
   DBUser=zabbix

   ######### PROXY SPECIFIC PARAMETERS #############
   ProxyOfflineBuffer=24
   HeartbeatFrequency=60
   ConfigFrequency=120

   ############ ADVANCED PARAMETERS ################
   StartPollersUnreachable=3
   StartHTTPPollers=3
   JavaGateway=127.0.0.1
   JavaGatewayPort=10052
   StartJavaPollers=5
   SNMPTrapperFile=/var/log/snmptrap/snmptrap.log
   StartSNMPTrapper=1
   CacheSize=32M
   Timeout=4
   ExternalScripts=/usr/lib/zabbix/externalscripts
   LogSlowQueries=3000

   ####### TLS-RELATED PARAMETERS #######
   TLSConnect=psk
   TLSAccept=psk
   TLSPSKIdentity=PSK zabbix-proxy-fqdn-hostname
   TLSPSKFile=/etc/zabbix/zabbix_proxy.psk

5. Check and make sure the installed zabbix proxy as well as the zabbix_proxy server zabbix_agentd client and zabbix_server are at the same major version release

a) Check zabbix proxy version

[root@sysadminshelp:/etc/zabbix]# zabbix_proxy -V
zabbix_proxy (Zabbix) 5.0.31
Revision f64a07aefca 30 January 2023, compilation time: Jan 30 2023 09:55:10

Copyright (C) 2023 Zabbix SIA
License GPLv2+: GNU GPL version 2 or later <https://www.gnu.org/licenses/>.
This is free software: you are free to change and redistribute it according to
the license. There is NO WARRANTY, to the extent permitted by law.

This product includes software developed by the OpenSSL Project
for use in the OpenSSL Toolkit (http://www.openssl.org/).

Compiled with OpenSSL 1.0.1e-fips 11 Feb 2013
Running with OpenSSL 1.0.1e-fips 11 Feb 2013
[root@sysadminshelp:/etc/zabbix]#

b) check zabbix_agentd version

[root@sysadminshelp:/etc/zabbix]# zabbix_agentd -V
zabbix_agentd (daemon) (Zabbix) 5.0.30
Revision 2c96c38fb4b 28 November 2022, compilation time: Nov 28 2022 11:27:43

Copyright (C) 2022 Zabbix SIA
License GPLv2+: GNU GPL version 2 or later <https://www.gnu.org/licenses/>.
This is free software: you are free to change and redistribute it according to
the license. There is NO WARRANTY, to the extent permitted by law.

This product includes software developed by the OpenSSL Project
for use in the OpenSSL Toolkit (http://www.openssl.org/).

Compiled with OpenSSL 1.0.1e-fips 11 Feb 2013
Running with OpenSSL 1.0.1e-fips 11 Feb 2013

c) Check zabbix server version

[root@zabbix:~]# zabbix_server -V
zabbix_server (Zabbix) 5.0.30
Revision 2c96c38fb4b 28 November 2022, compilation time: Nov 28 2022 09:19:03

Copyright (C) 2022 Zabbix SIA
License GPLv2+: GNU GPL version 2 or later <https://www.gnu.org/licenses/>.
This is free software: you are free to change and redistribute it according to
the license. There is NO WARRANTY, to the extent permitted by law.

This product includes software developed by the OpenSSL Project
for use in the OpenSSL Toolkit (http://www.openssl.org/).

Compiled with OpenSSL 1.1.1d 10 Sep 2019
Running with OpenSSL 1.1.1n 15 Mar 2022

6. Starting the zabbix-proxy for a first time

Before beginning with installation make sure selinux is disabled, as it might cause some issues with Zabbix

[root@sysadminshelp:/etc/zabbix]# sestatus
SELinux status: disabled

If you need to have the selinux enabled you will have to allow the zabbix-proxy into selinux as well:

# cd /tmp
# grep zabbix_proxy /var/log/audit/audit.log | grep denied | audit2allow -m zabbix_proxy > zabbix_proxy.te
# grep zabbix_proxy /var/log/audit/audit.log | grep denied | audit2allow -M zabbix_proxy
# semodule -i zabbix_proxy.pp

[root@sysadminshelp:/etc/zabbix]# systemctl start zabbix-proxy

Also lets enable zabbix-proxy to automatically start it on next server reboot / boot.

root@sysadminshelp:/etc/zabbix]# systemctl enable zabbix-proxy

Normally running zabbix-proxy should provide a status messages like:

[root@sysadminshelp:/etc/zabbix]# systemctl status zabbix-proxy
● zabbix-proxy.service – Zabbix Proxy
Loaded: loaded (/usr/lib/systemd/system/zabbix-proxy.service; disabled; vendor preset: disabled)
Active: active (running) since чт 2023-05-04 14:58:36 CEST; 2h 59min ago
Process: 8500 ExecStop=/bin/kill -SIGTERM $MAINPID (code=exited, status=0/SUCCESS)
Process: 8504 ExecStart=/usr/sbin/zabbix_proxy -c $CONFFILE (code=exited, status=0/SUCCESS)
Main PID: 8506 (zabbix_proxy)
CGroup: /system.slice/zabbix-proxy.service
├─8506 /usr/sbin/zabbix_proxy -c /etc/zabbix/zabbix_proxy.conf
├─8507 /usr/sbin/zabbix_proxy: configuration syncer [synced config 40521 bytes in 0.0…
├─8508 /usr/sbin/zabbix_proxy: trapper #1 [processed data in 0.000808 sec, waiting fo…
├─8509 /usr/sbin/zabbix_proxy: trapper #2 [processed data in 0.005028 sec, waiting fo…
├─8510 /usr/sbin/zabbix_proxy: trapper #3 [processed data in 0.001240 sec, waiting fo…
├─8511 /usr/sbin/zabbix_proxy: trapper #4 [processed data in 0.004378 sec, waiting fo…
├─8512 /usr/sbin/zabbix_proxy: trapper #5 [processed data in 0.004991 sec, waiting fo…
├─8513 /usr/sbin/zabbix_proxy: preprocessing manager #1 [queued 0, processed 3 values…
├─8514 /usr/sbin/zabbix_proxy: preprocessing worker #1 started
├─8515 /usr/sbin/zabbix_proxy: preprocessing worker #2 started
├─8516 /usr/sbin/zabbix_proxy: preprocessing worker #3 started
├─8517 /usr/sbin/zabbix_proxy: heartbeat sender [sending heartbeat message success in…
├─8518 /usr/sbin/zabbix_proxy: data sender [sent 0 values in 0.005241 sec, idle 1 sec…
├─8519 /usr/sbin/zabbix_proxy: housekeeper [deleted 4501 records in 0.011462 sec, idl…
├─8520 /usr/sbin/zabbix_proxy: http poller #1 [got 0 values in 0.000248 sec, idle 5 s…
├─8521 /usr/sbin/zabbix_proxy: http poller #2 [got 0 values in 0.000239 sec, idle 5 s…
├─8522 /usr/sbin/zabbix_proxy: http poller #3 [got 0 values in 0.000328 sec, idle 5 s…
├─8523 /usr/sbin/zabbix_proxy: discoverer #1 [processed 0 rules in 0.000261 sec, idle…
├─8524 /usr/sbin/zabbix_proxy: history syncer #1 [processed 0 values in 0.000009 sec,…
├─8525 /usr/sbin/zabbix_proxy: history syncer #2 [processed 0 values in 0.000007 sec,…
├─8526 /usr/sbin/zabbix_proxy: history syncer #3 [processed 0 values in 0.000014 sec,…
├─8527 /usr/sbin/zabbix_proxy: history syncer #4 [processed 0 values in 0.000021 sec,…
├─8528 /usr/sbin/zabbix_proxy: java poller #1 [got 0 values in 0.000017 sec, idle 5 s…
├─8529 /usr/sbin/zabbix_proxy: java poller #2 [got 0 values in 0.000019 sec, idle 5 s…
├─8530 /usr/sbin/zabbix_proxy: java poller #3 [got 0 values in 0.000019 sec, idle 5 s…
├─8531 /usr/sbin/zabbix_proxy: java poller #4 [got 0 values in 0.000018 sec, idle 5 s…
├─8532 /usr/sbin/zabbix_proxy: java poller #5 [got 0 values in 0.000013 sec, idle 5 s…
├─8533 /usr/sbin/zabbix_proxy: snmp trapper [processed data in 0.000026 sec, idle 1 s…
├─8534 /usr/sbin/zabbix_proxy: self-monitoring [processed data in 0.000034 sec, idle …
├─8535 /usr/sbin/zabbix_proxy: task manager [processed 0 task(s) in 0.000169 sec, idl…
├─8536 /usr/sbin/zabbix_proxy: poller #1 [got 0 values in 0.000012 sec, idle 5 sec]
├─8537 /usr/sbin/zabbix_proxy: poller #2 [got 0 values in 0.000021 sec, idle 5 sec]
├─8538 /usr/sbin/zabbix_proxy: poller #3 [got 0 values in 0.000039 sec, idle 5 sec]
├─8539 /usr/sbin/zabbix_proxy: poller #4 [got 0 values in 0.000024 sec, idle 5 sec]
├─8540 /usr/sbin/zabbix_proxy: poller #5 [got 0 values in 0.000019 sec, idle 5 sec]
├─8541 /usr/sbin/zabbix_proxy: unreachable poller #1 [got 0 values in 0.000011 sec, i…
├─8542 /usr/sbin/zabbix_proxy: unreachable poller #2 [got 0 values in 0.000018 sec, i…
├─8543 /usr/sbin/zabbix_proxy: unreachable poller #3 [got 0 values in 0.000041 sec, i…
└─8544 /usr/sbin/zabbix_proxy: icmp pinger #1 [got 0 values in 0.000022 sec, idle 5 s…

май 04 14:58:36 sysadminshelp systemd[1]: Stopped Zabbix Proxy.
май 04 14:58:36 sysadminshelp systemd[1]: Starting Zabbix Proxy…
май 04 14:58:36 sysadminshelp systemd[1]: Started Zabbix Proxy.

zabbix-server-zabbix-proxy-and-zabbix-clients-overview-diagram

7. Configure zabbix-agentd to use your just new brand new zabbix-proxy

Here is my sample configuration file:

[root@sysadminshelp:/etc/zabbix]# grep -v \# /etc/zabbix/zabbix_agentd.conf | sed '/^$/d'
PidFile=/var/run/zabbix/zabbix_agentd.pid
LogFile=/var/log/zabbix/zabbix_agentd.log
LogFileSize=0
Server=zabbix-proxy
ServerActive=zabbix-proxy:10051
ListenIP
Hostname=sysadminshelp
BufferSend=30
BufferSize=100
Include=/etc/zabbix/zabbix_agentd.d/*.conf

Note that the ServerActive given "zabbix-proxy" should be resolvable from the host, or even better you might want to put the IP of the Proxy if
you don't have at least a pseudo Hostname already configured inside /etc/hosts or actual DNS 'A' Active record configured inside a properly resolving
DNS server configured on the host via /etc/resolv.conf.

8. Create and Configure new proxy into the Zabbix-server host

Go to the zabbix server web interface URL into menus:

zabbix-administration-proxy-config

Administration -> Proxies (Proxy)

Click on ;

Create Proxy button (uppper right corner)

*Proxy name: usually-your-host-pingable-fqdn
Proxy mode: Active
Proxy address: 192.168.1.50
Description: pcfreak zabbix proxy

Administration -> Proxies -> Encryption

From "Connection to proxy"

Untick "No encryption"

and

Tick "PSK"

zabbix-administration-proxy-config-encryption

*PSK Identity: PSK proxy
*PSK: Put the key here (copy from /etc/zabbix/zabbix_proxy.psk generated steps earlier with openssl)

[root@sysadminshelp:/etc/zabbix]# cat zabbix_proxy.psk
faddbd96be00ac42c892fda5201634df25d51f3ndbbbf6cee9d354b2817092a28

Press the "Update" Button

zabbix-administration-proxy-config-encryption1

and go again to Proxies and check the zabbix-proxy is connected to the server and hosts configured to use the zabbix proxy reporting frequently.

To make sure that the configured new hosts to use the Zabbix Proxy instead of direct connection to Zabbix Server, go to Latest Data and check whether the configured Hostnames to connect to the Zabbix-Proxy continues to sent Data still.

9. Debugging problems with zabix-proxy and zabbix-agentd connectivity to proxy

In case of troubles check out what is going on inside the Zabbix Proxy / Agent and Server log files

[root@sysadminshelp:/etc/zabbix]# tail -n 50 /var/log/zabbix/zabbix_proxy.log

6832:20230504:134032.281 Starting Zabbix Proxy (active) [zabbix-proxy]. Zabbix 5.0.31 (revision f
64a07aefca).
6832:20230504:134032.281 **** Enabled features ****
6832:20230504:134032.281 SNMP monitoring: YES
6832:20230504:134032.281 IPMI monitoring: YES
6832:20230504:134032.281 Web monitoring: YES
6832:20230504:134032.281 VMware monitoring: YES
6832:20230504:134032.281 ODBC: YES
6832:20230504:134032.281 SSH support: YES
6832:20230504:134032.281 IPv6 support: YES
6832:20230504:134032.281 TLS support: YES
6832:20230504:134032.281 **************************
6832:20230504:134032.281 using configuration file: /etc/zabbix/zabbix_proxy.conf
6832:20230504:134032.291 current database version (mandatory/optional): 05000000/05000005
6832:20230504:134032.291 required mandatory version: 05000000
6832:20230504:134032.292 proxy #0 started [main process]
6833:20230504:134032.292 proxy #1 started [configuration syncer #1]
6833:20230504:134032.329 received configuration data from server at "192.168.1.28", datalen 40521
6834:20230504:134032.392 proxy #2 started [trapper #1]
6835:20230504:134032.401 proxy #3 started [trapper #2]
6836:20230504:134032.402 proxy #4 started [trapper #3]
6838:20230504:134032.405 proxy #6 started [trapper #5]
6837:20230504:134032.409 proxy #5 started [trapper #4]
6843:20230504:134032.409 proxy #11 started [heartbeat sender #1]
6845:20230504:134032.412 proxy #13 started [housekeeper #1]
6847:20230504:134032.412 proxy #15 started [discoverer #1]
8526:20230504:145836.512 proxy #20 started [history syncer #3]
8517:20230504:145836.512 proxy #11 started [heartbeat sender #1]
8530:20230504:145836.515 proxy #24 started [java poller #3]
8531:20230504:145836.517 proxy #25 started [java poller #4]
8532:20230504:145836.520 proxy #26 started [java poller #5]
8536:20230504:145836.522 proxy #30 started [poller #1]
8527:20230504:145836.525 proxy #21 started [history syncer #4]
8535:20230504:145836.525 proxy #29 started [task manager #1]
8533:20230504:145836.528 proxy #27 started [snmp trapper #1]
8539:20230504:145836.528 proxy #33 started [poller #4]
8538:20230504:145836.529 proxy #32 started [poller #3]
8534:20230504:145836.532 proxy #28 started [self-monitoring #1]
8544:20230504:145836.532 proxy #38 started [icmp pinger #1]
8543:20230504:145836.532 proxy #37 started [unreachable poller #3]
8542:20230504:145836.535 proxy #36 started [unreachable poller #2]
8541:20230504:145836.537 proxy #35 started [unreachable poller #1]
8540:20230504:145836.540 proxy #34 started [poller #5]
8507:20230504:150036.453 received configuration data from server at "192.168.1.28", datalen 40521
8507:20230504:150236.503 received configuration data from server at "192.168.1.28", datalen 40521
8507:20230504:150436.556 received configuration data from server at "192.168.1.28", datalen 40521
8507:20230504:150636.608 received configuration data from server at "192.168.1.28", datalen 40521
8507:20230504:150836.662 received configuration data from server at "192.168.1.28", datalen 40521
…

[root@sysadminshelp:/etc/zabbix]# tail -n 10 /var/log/zabbix-agent/zabbix_agentd.log
3096166:20230504:182840.461 agent #1 started [collector]
3096167:20230504:182840.462 agent #2 started [listener #1]
3096168:20230504:182840.463 agent #3 started [listener #2]
3096169:20230504:182840.464 agent #4 started [listener #3]
3096170:20230504:182840.464 agent #5 started [active checks #1]

If necessery to Debug further and track some strange errors, you might want to increase the DebugLevel to lets say DebugLevel=5

5 – extended debugging (produces even more information)

If checking both zabbix_agentd.log and zabbix_proxy.log cannot give you enough of a hint on what might be the issues you face with your userparameter scripts or missing Monitored data etc. and hopefully you have access to the zabbix-server machine, check out the zabbix server log as well

[root@zabbix:~]# tail -n 100 /var/log/zabbix/zabbix_server.log

3145027:20230504:182641.556 sending configuration data to proxy "zabbix-proxy" at "192.168.1.50", datalen 40521, bytes 6120 with compression ratio 6.6
3145029:20230504:182716.529 cannot send list of active checks to "192.168.1.30": host [pcfrxenweb] not found
3145028:20230504:182731.959 cannot send list of active checks to "192.168.1.30": host [pcfrxenweb] not found
3145029:20230504:182756.634 cannot send list of active checks to "192.168.1.30": host [pcfrxenweb] not found

Wrapping it up

In this article, we have learned how to install and configure a zabbix-proxy server and prepare a PSK encryption secret key for it.
We learned also how to connect this server to the central zabbix monitoring host machine in Active mode, so both Zabbix proxy and server can communicate in a secure crypted form,
as well as how to set zabbix_agentd clients to connect to the zabbix proxy
which will from itself send its data to the Central Zabbix server host as well as how to Debug and hopefully solve issues with communication between Zabbix client -> Zabbix Proxy -> Zabbix server.

I know this article, does not say anything revolutionary and there is plenty of posts online talking about how to run yourself a zabbix proxy and make in your home or corporate network,
but I thought to write it down as by writting it and reading a bit more on the topic of Zabbix Server / Proxy / Agent, that give myself a better overview on how this technologies work and such an article will give myself an easier step by step guide to follow,
in future when I have to configure Zabbix Environments for personal hobby or professionally for customers.
Hope you enjoyed. Cheers ! 🙂

Tags: cause, Click, configured, curl, download, file, history, Hope, howto, incompitability, Install Zabbix Proxy, lib, log, match, menus, messages, network, page, pcfreak, Press, psk, rpm, server version, task, unreachable, YES, zabbix-proxy
Posted in Linux, System Administration, Zabbix | 2 Comments »

How to start Syslog ( Syslog-ng ) on IBM AIX

Thursday, October 20th, 2022

Syslog-ng is a system logging application, which can be a replacement of the default syslog. With syslog-ng, the log messages can be sent in an ecrypted/secure channel to a remote server. If the central log server or the network connection becomes unavailable syslog-ng will store messages on the local hard disk.

The syslog-ng application automatically sends the stored messages to the server when the connection is reestablished, in the same order the messages were received. The disk buffer is persistent – no messages are lost even if syslog-ng is restarted. (Another possibility to send those messages to a secondary server.)

syslog-ng can filter log messages and select only the ones matching certain criteria, but it cannot interpret and analyze the meaning behind the messages. It can receive messages from files, remote hosts, and other sources, and these are sent to one or more destinations (files, remote hosts..),

It has a server – client model, here only syslog-ng client informations will be described (syslog-ng server has not been tested, only client, which were sending messages to a remote server (qradar).)

One missing feature of syslog-ng, that it cannot rotate logs by itself. For log rotation an external tool like logrotate needs to be used.

1. Install / Uninstall syslog-ng on AIX

1.1. Install syslog-ng on AIX

After downloading the syslog-ng installer package, we have 2 options
– run ./syslog-ng-<edition>-<version>-<OS>-<platform>.run script, or
– install as an rpm package: rpm -i syslog-ng-premium-edition-<version>-<OS>-<arch>.rpm

During install, the default syslogd will be replaced automatically by syslog-ng (no parallel operation is possible).

1.2. Uninstall syslog-ng on AIX

If the .run installer has been used: /opt/syslog-ng/bin/uninstall.sh (The uninstall script will automatically restore the syslog daemon used before installing syslog-ng.)
If the .rpm package has been used: rpm -e syslog-ng-premium-edition (with rpm, it does not restore the syslog daemon used before syslog-ng).

AIX is a custom and non-free OS if you had to deal with it you might might ponder how to stop / start syslog-ng.
The paths to binaries on AIX
The configuration is not universal, but represents the GEK server.

2. Set Automatic start of syslog-ng on AIX

Enable Start from /etc/tcpip:

start /opt/freeware/sbin/syslog-ng "$src_running"

3. Check syslog-ng configuration is correct

Configuration could be under separate locations but the most likely ones for syslog-ng.conf and license.txt files are located in the
/opt/syslog-ng/etc/ directory or /etc/syslog-ng depending on how it was configured on install time.

After changing tuning the configuration, it can be checked for errors:

# /opt/freeware/sbin/syslog-ng –syntax-only

Tags: AIX, configured, connection, default, hosts, How to, Install Uninstall, remote server, rpm, syslog
Posted in AIX, System Administration | No Comments »

How to monitor Haproxy Application server backends with Zabbix userparameter autodiscovery scripts

Friday, May 13th, 2022

Haproxy is doing quite a good job in High Availability tasks where traffic towards multiple backend servers has to be redirected based on the available one to sent data from the proxy to.

Lets say haproxy is configured to proxy traffic for App backend machine1 and App backend machine2.

Usually in companies people configure a monitoring like with Icinga or Zabbix / Grafana to keep track on the Application server is always up and running. Sometimes however due to network problems (like burned Network Switch / router or firewall misconfiguration) or even an IP duplicate it might happen that Application server seems to be reporting reachable from some monotoring tool on it but unreachable from Haproxy server -> App backend machine2 but reachable from App backend machine1. And even though haproxy will automatically switch on the traffic from backend machine2 to App machine1. It is a good idea to monitor and be aware that one of the backends is offline from the Haproxy host.
In this article I'll show you how this is possible by using 2 shell scripts and userparameter keys config through the autodiscovery zabbix legacy feature.
Assumably for the setup to work you will need to have as a minimum a Zabbix server installation of version 5.0 or higher.

1. Create the required haproxy_discovery.sh and haproxy_stats.sh scripts

You will have to install the two scripts under some location for example we can put it for more clearness under /etc/zabbix/scripts

[root@haproxy-server1 ]# mkdir /etc/zabbix/scripts

[root@haproxy-server1 scripts]# vim haproxy_discovery.sh
#!/bin/bash
#
# Get list of Frontends and Backends from HAPROXY
# Example: ./haproxy_discovery.sh [/var/lib/haproxy/stats] FRONTEND|BACKEND|SERVERS
# First argument is optional and should be used to set location of your HAPROXY socket
# Second argument is should be either FRONTEND, BACKEND or SERVERS, will default to FRONTEND if not set
#
# !! Make sure the user running this script has Read/Write permissions to that socket !!
#
## haproxy.cfg snippet
# global
# stats socket /var/lib/haproxy/stats mode 666 level admin

HAPROXY_SOCK=""/var/run/haproxy/haproxy.sock
[ -n “$1” ] && echo $1 | grep -q ^/ && HAPROXY_SOCK="$(echo $1 | tr -d '\040\011\012\015')"

if [[ “$1” =~ (25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?):[0-9]{1,5} ]];
then
HAPROXY_STATS_IP="$1"
QUERYING_METHOD="TCP"
fi

QUERYING_METHOD="${QUERYING_METHOD:-SOCKET}"

query_stats() {
if [[ ${QUERYING_METHOD} == “SOCKET” ]]; then
echo "show stat" | socat ${HAPROXY_SOCK} stdio 2>/dev/null
elif [[ ${QUERYING_METHOD} == “TCP” ]]; then
echo "show stat" | nc ${HAPROXY_STATS_IP//:/ } 2>/dev/null
fi
}

get_stats() {
echo "$(query_stats)" | grep -v "^#"
}

[ -n “$2” ] && shift 1
case $1 in
B*) END="BACKEND" ;;
F*) END="FRONTEND" ;;
S*)
for backend in $(get_stats | grep BACKEND | cut -d, -f1 | uniq); do
for server in $(get_stats | grep "^${backend}," | grep -v BACKEND | grep -v FRONTEND | cut -d, -f2); do
serverlist="$serverlist,\n"'\t\t{\n\t\t\t"{#BACKEND_NAME}":"'$backend'",\n\t\t\t"{#SERVER_NAME}":"'$server'"}'
done
done
echo -e '{\n\t"data":[\n’${serverlist#,}’]}'
exit 0
;;
*) END="FRONTEND" ;;
esac

for frontend in $(get_stats | grep "$END" | cut -d, -f1 | uniq); do
felist="$felist,\n"'\t\t{\n\t\t\t"{#'${END}'_NAME}":"'$frontend'"}'
done
echo -e '{\n\t"data":[\n’${felist#,}’]}'

[root@haproxy-server1 scripts]# vim haproxy_stats.sh
#!/bin/bash
set -o pipefail

if [[ “$1” = /* ]]
then
HAPROXY_SOCKET="$1"
shift 0
else
if [[ “$1” =~ (25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?):[0-9]{1,5} ]];
then
HAPROXY_STATS_IP="$1"
QUERYING_METHOD="TCP"
shift 1
fi
fi

pxname="$1"
svname="$2"
stat="$3"

DEBUG=${DEBUG:-0}
HAPROXY_SOCKET="${HAPROXY_SOCKET:-/var/run/haproxy/haproxy.sock}"
QUERYING_METHOD="${QUERYING_METHOD:-SOCKET}"
CACHE_STATS_FILEPATH="${CACHE_STATS_FILEPATH:-/var/tmp/haproxy_stats.cache}"
CACHE_STATS_EXPIRATION="${CACHE_STATS_EXPIRATION:-1}" # in minutes
CACHE_INFO_FILEPATH="${CACHE_INFO_FILEPATH:-/var/tmp/haproxy_info.cache}" ## unused
CACHE_INFO_EXPIRATION="${CACHE_INFO_EXPIRATION:-1}" # in minutes ## unused
GET_STATS=${GET_STATS:-1} # when you update stats cache outsise of the script
SOCAT_BIN="$(which socat)"
NC_BIN="$(which nc)"
FLOCK_BIN="$(which flock)"
FLOCK_WAIT=15 # maximum number of seconds that "flock" waits for acquiring a lock
FLOCK_SUFFIX='.lock'
CUR_TIMESTAMP="$(date '+%s')"

debug() {
[ “${DEBUG}” -eq 1 ] && echo "DEBUG: $@" >&2 || true
}

debug "SOCAT_BIN => $SOCAT_BIN"
debug "NC_BIN => $NC_BIN"
debug "FLOCK_BIN => $FLOCK_BIN"
debug "FLOCK_WAIT => $FLOCK_WAIT seconds"
debug "CACHE_FILEPATH => $CACHE_FILEPATH"
debug "CACHE_EXPIRATION => $CACHE_EXPIRATION minutes"
debug "HAPROXY_SOCKET => $HAPROXY_SOCKET"
debug "pxname => $pxname"
debug "svname => $svname"
debug "stat => $stat"

# check if socat is available in path
if [ “$GET_STATS” -eq 1 ] && [[ $QUERYING_METHOD == “SOCKET” && -z “$SOCAT_BIN” ]] || [[ $QUERYING_METHOD == “TCP” && -z “$NC_BIN” ]]
then
echo 'ERROR: cannot find socat binary'
exit 126
fi

# if we are getting stats:
# check if we can write to stats cache file, if it exists
# or cache file path, if it does not exist
# check if HAPROXY socket is writable
# if we are NOT getting stats:
# check if we can read the stats cache file
if [ “$GET_STATS” -eq 1 ]
then
if [ -e “$CACHE_FILEPATH” ] && [ ! -w “$CACHE_FILEPATH” ]
then
echo 'ERROR: stats cache file exists, but is not writable'
exit 126
elif [ ! -w ${CACHE_FILEPATH%/*} ]
then
echo 'ERROR: stats cache file path is not writable'
exit 126
fi
if [[ $QUERYING_METHOD == “SOCKET” && ! -w $HAPROXY_SOCKET ]]
then
echo "ERROR: haproxy socket is not writable"
exit 126
fi
elif [ ! -r “$CACHE_FILEPATH” ]
then
echo 'ERROR: cannot read stats cache file'
exit 126
fi

# index:name:default
MAP="
1:pxname:@
2:svname:@
3:qcur:9999999999
4:qmax:0
5:scur:9999999999
6:smax:0
7:slim:0
8:stot:@
9:bin:9999999999
10:bout:9999999999
11:dreq:9999999999
12:dresp:9999999999
13:ereq:9999999999
14:econ:9999999999
15:eresp:9999999999
16:wretr:9999999999
17:wredis:9999999999
18:status:UNK
19:weight:9999999999
20:act:9999999999
21:bck:9999999999
22:chkfail:9999999999
23:chkdown:9999999999
24:lastchg:9999999999
25:downtime:0
26:qlimit:0
27:pid:@
28:iid:@
29:sid:@
30:throttle:9999999999
31:lbtot:9999999999
32:tracked:9999999999
33:type:9999999999
34:rate:9999999999
35:rate_lim:@
36:rate_max:@
37:check_status:@
38:check_code:@
39:check_duration:9999999999
40:hrsp_1xx:@
41:hrsp_2xx:@
42:hrsp_3xx:@
43:hrsp_4xx:@
44:hrsp_5xx:@
45:hrsp_other:@
46:hanafail:@
47:req_rate:9999999999
48:req_rate_max:@
49:req_tot:9999999999
50:cli_abrt:9999999999
51:srv_abrt:9999999999
52:comp_in:0
53:comp_out:0
54:comp_byp:0
55:comp_rsp:0
56:lastsess:9999999999
57:last_chk:@
58:last_agt:@
59:qtime:0
60:ctime:0
61:rtime:0
62:ttime:0
"

_STAT=$(echo -e "$MAP" | grep :${stat}:)
_INDEX=${_STAT%%:*}
_DEFAULT=${_STAT##*:}

debug "_STAT => $_STAT"
debug "_INDEX => $_INDEX"
debug "_DEFAULT => $_DEFAULT"

# check if requested stat is supported
if [ -z “${_STAT}” ]
then
echo "ERROR: $stat is unsupported"
exit 127
fi

# method to retrieve data from haproxy stats
# usage:
# query_stats "show stat"
query_stats() {
if [[ ${QUERYING_METHOD} == “SOCKET” ]]; then
echo $1 | socat ${HAPROXY_SOCKET} stdio 2>/dev/null
elif [[ ${QUERYING_METHOD} == “TCP” ]]; then
echo $1 | nc ${HAPROXY_STATS_IP//:/ } 2>/dev/null
fi
}

# a generic cache management function, that relies on 'flock'
check_cache() {
local cache_type="${1}"
local cache_filepath="${2}"
local cache_expiration="${3}"
local cache_filemtime
cache_filemtime=$(stat -c '%Y' "${cache_filepath}" 2> /dev/null)
if [ $((cache_filemtime+60*cache_expiration)) -ge ${CUR_TIMESTAMP} ]
then
debug "${cache_type} file found, results are at most ${cache_expiration} minutes stale.."
elif "${FLOCK_BIN}" –exclusive –wait "${FLOCK_WAIT}" 200
then
cache_filemtime=$(stat -c '%Y' "${cache_filepath}" 2> /dev/null)
if [ $((cache_filemtime+60*cache_expiration)) -ge ${CUR_TIMESTAMP} ]
then
debug "${cache_type} file found, results have just been updated by another process.."
else
debug "no ${cache_type} file found, querying haproxy"
query_stats "show ${cache_type}" > "${cache_filepath}"
fi
fi 200> "${cache_filepath}${FLOCK_SUFFIX}"
}

# generate stats cache file if needed
get_stats() {
check_cache 'stat' "${CACHE_STATS_FILEPATH}" ${CACHE_STATS_EXPIRATION}
}

# generate info cache file
## unused at the moment
get_info() {
check_cache 'info' "${CACHE_INFO_FILEPATH}" ${CACHE_INFO_EXPIRATION}
}

# get requested stat from cache file using INDEX offset defined in MAP
# return default value if stat is ""
get() {
# $1: pxname/svname
local _res="$("${FLOCK_BIN}" –shared –wait "${FLOCK_WAIT}" "${CACHE_STATS_FILEPATH}${FLOCK_SUFFIX}" grep $1 "${CACHE_STATS_FILEPATH}")"
if [ -z “${_res}” ]
then
echo "ERROR: bad $pxname/$svname"
exit 127
fi
_res="$(echo $_res | cut -d, -f ${_INDEX})"
if [ -z “${_res}” ] && [[ “${_DEFAULT}” != “@” ]]
then
echo "${_DEFAULT}"
else
echo "${_res}"
fi
}

# not sure why we'd need to split on backslash
# left commented out as an example to override default get() method
# status() {
# get "^${pxname},${svnamem}," $stat | cut -d\ -f1
# }

# this allows for overriding default method of getting stats
# name a function by stat name for additional processing, custom returns, etc.
if type get_${stat} >/dev/null 2>&1
then
debug "found custom query function"
get_stats && get_${stat}
else
debug "using default get() method"
get_stats && get "^${pxname},${svname}," ${stat}
fi

! NB ! Substitute in the script /var/run/haproxy/haproxy.sock with your haproxy socket location

You can download the haproxy_stats.sh here and haproxy_discovery.sh here

2. Create the userparameter_haproxy_backend.conf

[root@haproxy-server1 zabbix_agentd.d]# cat userparameter_haproxy_backend.conf
#
# Discovery Rule
#

# HAProxy Frontend, Backend and Server Discovery rules
UserParameter=haproxy.list.discovery[*],sudo /etc/zabbix/scripts/haproxy_discovery.sh SERVER
UserParameter=haproxy.stats[*],sudo /etc/zabbix/scripts/haproxy_stats.sh $2 $3 $4

# support legacy way

UserParameter=haproxy.stat.downtime[*],sudo /etc/zabbix/scripts/haproxy_stats.sh $2 $3 downtime

UserParameter=haproxy.stat.status[*],sudo /etc/zabbix/scripts/haproxy_stats.sh $2 $3 status

UserParameter=haproxy.stat.last_chk[*],sudo /etc/zabbix/scripts/haproxy_stats.sh $2 $3 last_chk

3. Create new simple template for the Application backend Monitoring and link it to monitored host

create-configuration-template-backend-monitoring

create-template-backend-monitoring-macros

Go to Configuration -> Hosts (find the host) and Link the template to it

4. Restart Zabbix-agent, in while check autodiscovery data is in Zabbix Server

[root@haproxy-server1 ]# systemctl restart zabbix-agent

Check in zabbix the userparameter data arrives, it should not be required to add any Items or Triggers as autodiscovery zabbix feature should automatically create in the server what is required for the data regarding backends to be in.

To view data arrives go to Zabbix config menus:

Configuration -> Hosts -> Hosts: (lookup for the haproxy-server1 hostname)

The autodiscovery should have automatically created the following prototypes

Now if you look inside Latest Data for the Host you should find some information like:

HAProxy Backend [backend1] (3 Items)

HAProxy Server [backend-name_APP/server1]: Connection Response
2022-05-13 14:15:04           History

HAProxy Server [backend-name/server2]: Downtime (hh:mm:ss)
2022-05-13 14:13:57   20:30:42       History

HAProxy Server [bk_name-APP/server1]: Status
2022-05-13 14:14:25   Up (1)       Graph
       ccnrlb01   HAProxy Backend [bk_CCNR_QA_ZVT] (3 Items)

HAProxy Server [bk_name-APP3/server1]: Connection Response
2022-05-13 14:15:05           History

HAProxy Server [bk_name-APP3/server1]: Downtime (hh:mm:ss)
2022-05-13 14:14:00   20:55:20       History

HAProxy Server [bk_name-APP3/server2]: Status
2022-05-13 14:15:08   Up (1)

To make alerting in case if a backend is down which usually you would like only left thing is to configure an Action to deliver alerts to some email address.

Tags: admin, configured, esac, file, frontend, Haproxy Application, How to, index, information, Network Switch, scripts, servers, shell scripts, sid, Zabbix Server
Posted in Bash Scripting, System Optimization, Zabbix | No Comments »