Start « ☩ Walking in Light with Christ

Posts Tagged ‘Start’

Monitoring a 2-Node HAProxy Cluster with Zabbix and crm_mon

Wednesday, August 19th, 2026

When you run HAProxy in a two-node Pacemaker cluster, monitoring the HAProxy service itself is not always enough.

A node can be reachable while the cluster resource is stopped. One cluster node can also fail without causing an immediate service outage because the second node takes over. The situation that really needs attention is when both HAProxy nodes are down.

In this setup, Zabbix can use

crm_mon

to check the Pacemaker cluster and alert when the cluster loses one or both nodes.

This article shows one way to build that monitoring.

The overall Cluster setup

For this example, we have two HAProxy servers:

haproxy01

haproxy02

Both servers are members of the same Pacemaker/Corosync cluster.

Zabbix is running separately and is responsible for monitoring the environment.

The goal is to have Zabbix report three basic states:

0 = both nodes are online

1 = one node is offline

2 = both nodes are offline

The first two states are useful for operational monitoring, but the important one is

2

. If both nodes are down, the HAProxy service is no longer highly available.

Why check ?

crm_mon

A common approach is to check the HAProxy service directly:

# systemctl is-active haproxy

That is useful, but it only tells us about HAProxy on the machine where the command is executed.

Pacemaker has its own view of the cluster.

crm_mon

shows the state of the cluster nodes and resources.

For example:

# crm_mon -1

On a healthy cluster, you might see something like:

Node List:
Online: [ haproxy01 haproxy02 ]
Full List of Resources:
haproxy (systemd:haproxy): Started haproxy01

haproxy02

goes down, the output could look like:

Node List:
Online: [ haproxy01 ]
OFFLINE: [ haproxy02 ]

If both nodes are unavailable:

Node List:
OFFLINE: [ haproxy01 haproxy02 ]

The exact output depends on the Pacemaker version, so it is worth checking the output on your own servers before writing the monitoring script.

1. Creating a simple check script (approach 1)

Rather than putting a long shell command directly into Zabbix, I prefer using a small script and letting the Zabbix agent return a single value.

For example:

/usr/local/bin/check_haproxy_cluster.sh

A simple version could be:

#!/bin/bash
CRM_MON="/usr/sbin/crm_mon"
if [ ! -x "$CRM_MON" ]; then
echo "-1"
exit 1
fi
OUTPUT=$($CRM_MON -1 2>/dev/null)
if [ $? -ne 0 ]; then
echo "-1"
exit 1
fi
ONLINE=$(echo "$OUTPUT" | grep -A1 "Online:" | grep -o "haproxy[0-9]*" | wc -l)
case "$ONLINE" in
2)
echo "0"
;;
1)
echo "1"
;;
0)
echo "2"
;;
*)
echo "-1"
;;
esac

Then make it executable:

# chmod +x /usr/local/bin/check_haproxy_cluster.sh

Run it manually:

/usr/local/bin/check_haproxy_cluster.sh

On a healthy cluster, the result should be:

If one node is down:

And if both nodes are down:

The

-1

value is used for an error, for example if

# crm_mon

cannot be executed.

One thing to watch out for

The example above assumes the node names follow a pattern such as

haproxy01

and

haproxy02

That is fine for a simple setup, but I would not use the script unchanged in production without checking the actual

crm_mon

output.

You should also make sure that finding a node name in the output really means that the node is online. Depending on the Pacemaker version, a more precise parser may be necessary.

2. Add the check to the Zabbix agent

The next step is to expose the script through a Zabbix

UserParameter

For example:

UserParameter=haproxy.cluster.status,/usr/local/bin/check_haproxy_cluster.sh

This can be added to the Zabbix agent configuration or to a separate file under the agent's configuration directory.

After changing the configuration, restart the agent:

# systemctl restart zabbix-agent

You can test the item before doing anything in the Zabbix frontend:

zabbix_agentd -t haproxy.cluster.status

A healthy result should look similar to:

haproxy.cluster.status [t|0]

3. Create the Zabbix item

In Zabbix, create an item using the following settings:

Name

HAProxy Cluster Status

Type

Zabbix agent

Key

haproxy.cluster.status

Type of information

Numeric (unsigned)

For the update interval, something like 30 seconds is usually reasonable:

30s

There is nothing special about 30 seconds. The interval should depend on how quickly you need to know about a failure.

4. Create the triggers

Now we can turn the returned values into actual alerts.

For both nodes being down:

last(/HAProxy Cluster/haproxy.cluster.status)=2

The trigger name could simply be:

Both HAProxy cluster nodes are DOWN

I would normally give this a high severity because there is no longer a working HAProxy node available.

For a single node failure:

last(/HAProxy Cluster/haproxy.cluster.status)=1

The trigger could be:

One HAProxy cluster node is DOWN

This can have a lower severity.

That distinction is useful. A single node failure means the cluster is degraded, but the service may still be working normally. Two failed nodes are a different situation.

The resulting states

The Zabbix configuration is then roughly:

Cluster state	Value	Alert
Both nodes online	0	OK
One node offline	1	Warning
Both nodes offline	2	High/Critical
Check failed	-1	Monitoring problem

This is much easier to work with than having Zabbix process the complete output of

# crm_mon

There is a problem with checking from the HAProxy node

There is an important point here.

If you run the script only on

haproxy01

, what happens when

haproxy01

itself completely disappears?

The Zabbix agent on that server is gone as well. Zabbix cannot ask it what happened to the cluster.

This can leave a blind spot exactly when you need monitoring the most.

For that reason, I would run the cluster check from an independent monitoring location whenever possible.

For example:

Zabbix Server
|
|
Zabbix Proxy
|
+————+——–+
| |
haproxy01 haproxy02

The monitoring system should not rely on one of the two HAProxy servers to tell you that both HAProxy servers have failed.

5. A more reliable approach

For a production installation, I would also consider using the machine-readable output from

crm_mon

For example:

# crm_mon -1 -X

The

-X

option produces XML output. That makes it possible to parse the cluster state without depending on the formatting of the normal human-readable output.

This matters because a script based on simplicity: e.g. – grep, awk, sed

can easily break after a Pacemaker update if the output format changes.

The general idea remains the same:

Pacemaker
|
crm_mon
|
status parser
|
Zabbix item
|
Zabbix trigger

The only difference is that the parser is working with structured data instead of text intended for a person to read.

6. Don't stop at the cluster state

Checking whether the nodes are online is a good start, but it is not the same as checking whether the application is actually working.

For example, you could have:

haproxy01 Online
haproxy02 Online
HAProxy Running
VIP Available

and still have a problem with the application behind HAProxy.

For that reason, I normally split the monitoring into several checks:

+ Is the server reachable?
+ Is the Pacemaker node online?
+ Is the HAProxy resource running?
+ Is the HAProxy service running?
+ Is the virtual IP available?
+ Can a client actually connect through HAProxy?
+ Is the application behind HAProxy responding?

The last check is often the most useful one because it tests the service from the user's point of view.

7. Testing the setup

Before relying on the Zabbix alert, test the different failure scenarios.

Start with both nodes online:

/usr/local/bin/check_haproxy_cluster.sh

Expected:

0

Next, take one node offline using your normal cluster maintenance procedure.

The check should return:

1

Zabbix should report that one HAProxy node is down, but it should not report a complete HAProxy outage.

Finally, test the situation where neither node is available.

The check should return:

2

At that point Zabbix should generate:

Both HAProxy cluster nodes are DOWN

After testing, verify that the problem clears automatically when the cluster returns to its normal state.

Monitoring a 2-Node HAProxy Cluster with Zabbix with more simplistic crm_mon Userparameter (Approach 2)

When you run a high-availability (HA) cluster with Pacemaker, Corosync, and HAProxy, monitoring can be tricky. If you set up standard alerts, a single node rebooting for maintenance can trigger a false "service down" alarm, even though the other node handled the failover perfectly.

To prevent alert fatigue, you need Zabbix to look at the cluster as a single unit and only fire a critical page when both HAProxy nodes go down at the same time.

Here is a practical guide to setting this up using

crm_mon

1. Collect Cluster Data via Zabbix Agent

Instead of parsing messy text outputs, we can make

crm_mon

output clean XML data, then coun

Run these steps on both cluster nodes:

a. Create a new Zabbix agent configuration file:

 vim /etc/zabbix/zabbix_agentd.d/haproxy_cluster.conf

b. Paste the following

UserParameter

line.

This runs

crm_mon

grabs the XML structure, and returns the total count of active HAProxy resources:

 UserParameter=crm.haproxy.active,sudo crm_mon -1 --as-xml | grep -c 'resource id="haproxy".*active="true"'

(Note: Change

"haproxy"

to match the exact resource ID used in your Pacemaker setup).

c. Give the

zabbix

user permission to run

crm_mon

without a password prompt.

Open the sudoers file:

# visudo

Add this line at the bottom:

zabbix ALL=(ALL) NOPASSWD: /usr/sbin/crm_mon

d. Restart your Zabbix agent:

 # systemctl restart zabbix-agent

2. Set Up the Zabbix Item

Open your Zabbix Web UI. Create this item inside a template and apply it to your cluster nodes:

Name: HAProxy Active Nodes in Cluster
Type: Zabbix agent
Key:

crm.haproxy.active

Type of information: Numeric (unsigned)
Update interval: 1m

What these values mean:

2 = Healthy. Both nodes are up and active.
1 = Degraded. One node died, but traffic is still flowing through the survivor.
0 = Outage. Both nodes are down; the service is dead.

3. Create the Smart Triggers

Now we can create two different alerts so your team knows the difference between a minor issue and an emergency.

The "All Hands on Deck" Alert (Both Nodes Down)

This fires only when the active count drops to zero.

Name: HAProxy Cluster Outage: Both nodes are DOWN!
Severity: Disaster
Expression:

 last(/Template HAProxy Cluster/crm.haproxy.active)=0

The Warning Alert (One Node Down)

This lets you know you lost redundancy, but the site is still up. [1]

Name: HAProxy Cluster Degraded: One node is DOWN
Severity: Average
Expression:

last(/Template HAProxy Cluster/crm.haproxy.active)=1

Alternative: The Quick and Dirty "AND" Trigger

If you do not want to deal with crm_mon or sudoers permissions, you can achieve the same result using Zabbix's built-in network checks and a logical and condition.

If you monitor the HAProxy port (e.g., 8080 or 443) on node1

and node2 individually, you can chain them together in a single trigger expression:

 last(/node1/net.tcp.service[http,,443])=0 and last(/node2/net.tcp.service[http,,8080])=0

With this logic, Zabbix stays quiet during routine single-node reboots and only alerts your team if both ports become unresponsive simultaneously.

Final thoughts

For a two-node HAProxy cluster, checking only

# systemctl status haproxy

does not give you the whole picture.

crm_mon

gives Zabbix access to the Pacemaker view of the cluster, and a small monitoring script can turn that information into a simple status value.

The basic idea is straightforward:

0 = both nodes OK
1 = one node down
2 = both nodes down

From there, Zabbix can handle the alerting.

The most important part is not the Zabbix trigger itself. It is making sure that the monitoring check is independent of the servers it is supposed to monitor. If both HAProxy nodes can fail, the check should ideally run from a Zabbix Server, Zabbix Proxy, or another independent monitoring host.

For a small environment, the simple
crm_mon parser is usually enough.

For a production environment, I would use the XML output from crm_mon

and explicitly monitor the two expected cluster nodes and the HAProxy resource as separate checks.

Tags: configuration file, Creating, down, expression, Final, generate, grep, look, mean, Monitoring, Node List, Offline, Open, Restart, script, servers, Start, status
Posted in Haproxy, Linux, Monitoring | No Comments »

Speed up Linux shell use keyboard command alias shortcuts to effiently work like a hacker

Friday, May 1st, 2026

If you want to get truly fast in the Linux Bash shell, stop thinking in commands alone and start doing trivial command tasks by thinking it in keystrokes !
The biggest productivity gains don’t come only by learning new tools, they come from navigating and reusing what is embedded as default functionality, like editing commands , searching through them and shortcuts to run and reuse instantly without need to type again and again.

At the center of this approach is one habit, to try to never type the same command twice.

1. The Allmighty, Reverse Search ( `Ctrl + R` )

If you learn only one shortcut for a begginning say hello to the King of all bash shortcut commands CTRL + R.

Press:

 
Ctrl + R

Then start typing part of a previous command. Bash will search your history in real time and show the most recent match.

Example:

(reverse-i-search)`ssh': ssh user@server

Press:
To cycle further one command match back:

Ctrl + R

again

Edit before running use:

→ (right arrow)

To run found cmd simply press Enter.

This is dramatically faster than scrolling through history or retyping long commands. Over time, your shell history becomes a searchable command database.

2. Stop annoying re-typing: navigate the Line instantly

When editing a command, don’t hold arrow keys—jump instead:

Go to the beginning of line

Ctrl + A

Move to the end of command string:

Ctrl + E

Jump back one word

Alt + B

Jump forward one word ahead

Alt + F

These shortcuts let you fix mistakes or modify long commands in seconds.

3. Precise Delete strings

Precise deletion is just as important as movement:

Delete everything before cursor position:

Ctrl + U

Delete everything after cursor position:

Ctrl + K

Delete previous word from cmd string:

Ctrl + W

Delete next word in command string

Alt + D

Instead of holding backspace, you surgically remove chunks of text.

4. Reuse arguments without rewriting

Bash has built-in shortcuts for reusing parts of previous commands:

Repeat last command, type in shell

!!

Last argument of previous command

!$

Add all arguments from previous command to a command

!*

For example on use last argument from previous command:

mkdir project
cd !$

This jumps into the directory you just created without retyping its name.

hipo@jeremiah:/usr/local/bin$ find . /usr/local/bin/ /bin/ /usr/bin -iname 'ls'
/bin/ls
/usr/bin/ls
hipo@jeremiah:/usr/local/bin$ echo !*
echo . /usr/local/bin/ /bin/ /usr/bin -iname 'ls'

To only get the file name of

5. Fix Mistakes Instantly hack

Made a typo? You don’t need to retype the whole command.

Use the shortcut:

^old^new

Example:

hipo@jeremiah: ~$ ls -al /bin/sl
ls: cannot access '/bin/sl': No such file or directory
hipo@jeremiah: ~$ ^sl^ls
ls -al /bin/ls
-rwxr-xr-x 1 root root 151344 Sep 20  2022 /bin/ls

Bash reruns the previous command with the correction applied.

6. Use history without running history cmd

The quick access to last and previous commands, is perhaps known by most but for novice people starting will shell it is worthy mention:

Scroll through commands:

Keyboard Arrow Up / Down keys ↑ / ↓

run command number n from history !n :

To re-run cmd from history line 10

$ !10

To lets say you want to get last 10 commands from history:

$ history 10

Instead of getting full comand history with

$ history

Use the Ctrl + R which is faster shortcut to arrow keys and walking through history.

7. Use Auto-Complete

The good old well known Tab key is well known one by almost all sysadmins, but I’ll mention it anyways.

Auto-complete file / command
Single Tab press

Show all matches
Press Tab twice

This reduces typing and prevents errors – especially with long file paths.

8. Edit the previous command straight in editor

For complex commands, use:

Ctrl + X, Ctrl + E

This opens your last command in your default editor. You can comfortably edit multi-line or complicated commands, then save and execute.

9. Clear and Reset Quickly

Clear the screen (same as clear ):

Ctrl + L

Cancel current command:

Ctrl + C

Exit shell:

Ctrl + D

These keep your terminal clean and under control.

10. Background and Foreground Control

You can manage running processes with the keyboard too:

Pause (suspend) active running process on cmd line:

Ctrl + Z

Resume process in background:

$ bg

Bring back to foreground:

$ fg

This is especially useful when you accidentally start something in the foreground.

11. Memorize shortcuts / improve shell habits

When these shortcuts become automatic, habit for you will soon reap the benefits.

You will then no longer need to, constantly retype long command lines, you will not loose time to point with the mouse, you save time on editing your command line:

Of course getting it as habit will take few hours to a day.

Start with just building two habits:

Use Ctrl + R instead of retyping
Use Ctrl + A / Ctrl + E instead of arrow keys

Once those stick, layer in the others.

12. Start using fzf fuzzy finder command utility

To get even better command line search and easier manage things with command line binds use fzf.

# apt install –yes fzf
$ source /usr/share/doc/fzf/examples/key-bindings.bash

The fzf command-line tool enhances Linux terminal productivity by replacing the standard, rigid Ctrl+R history search with interactive, real-time fuzzy matching.
It offers a visual interface for searching command history, file paths via Ctrl+T , and directories using Alt+C [Source]. Installing fzf enables a highly efficient workflow, allowing users to find and execute commands faster.

For a complete use cases check GitHub fzf page.

Final Thought

Efficient command line use in Bash is not only about doing less typing, it is about doing more work with less effort, so you can have more time for the important stuff.
The keyboard shortcuts are already there for long time and computer hackers (i mean old school system programmers) has been using them for ages not only in bash but in ksh, zsh, csh and waiting to remove friction from everything you do.
Master them, and the shell stops being a place where you type in like a secretary, but a enjoyable more fun place to spend time on.

Tags: alias, arrow keys, Cancel, command, computer hackers, Delete, last, Linux Bash, movement, Repeat, Start, Stop
Posted in Linux, Linux and FreeBSD Desktop, Programming, System Administration, System Optimization, Various | 1 Comment »

Building a 10-Server FreeBSD Jail Cluster Running a LAMP (Linux / Apache / MySQL / Perl / PHP / Python) Stack

Wednesday, March 25th, 2026

Virtualization and workload isolation are foundational to modern infrastructure.
While most teams today default to container platforms like Docker and orchestration systems such as Kubernetes, an older and highly capable alternative exists in the form of jails from FreeBSD.

FreeBSD jails provide lightweight OS-level isolation, allowing multiple independent userland environments to run on a single host. Introduced long before containers became mainstream, jails were designed with a strong focus on security, simplicity, and performance.
Despite their maturity and robustness, they are less commonly used today, largely due to the rapid rise of container ecosystems and cloud-native tooling.

Choosing between jails and containers is not simply a matter of “old vs new,” but rather a trade-off between control and simplicity versus portability and ecosystem support.

Short Comparison of FreeBSD jails and Containers ( Pros and Cons )

Advantages of FreeBSD Jails

a. Strong, simple isolation

Jails provide a clear and tightly integrated security boundary within the FreeBSD kernel. Their design is straightforward, reducing the risk of misconfiguration compared to layered container security models.

b. High performance

Because jails operate very close to the base system, they deliver near-native performance with minimal overhead—especially beneficial for networking and I/O-heavy workloads.

c. Operational simplicity

There are fewer component moving parts (easier to maintain and debbug):

No separate container runtime
No image layers
No complex orchestration requirements

This makes jails appealing for stable, long-running systems.

d. Predictability and stability

FreeBSD’s conservative, design philosophy results in systems that are highly stable over long periods, that is ideal for infrastructure roles like: storage or networking.

Disadvantages of FreeBSD Jails

a. Limited portability

Not neceserry a huge disadvantage but still,
Jails are tied to FreeBSD. Unlike containers, they cannot be easily moved across different operating systems or cloud platforms.

b. Smaller ecosystem

FBSD Jails is not full equivallent to:

Container registries (like Docker Hub)
Massive orchestration ecosystems (similar things has to be done with scripts and customizations)
Broad third-party integrations

This can slow down a bit development and deployment workflows. Though for a matured Applications that are once well tuned with jails that can be not a real probblem.

Note that though a con, this can also be a pros, as once you tune up an App for it becomes easier to maintain.

c. Less automation tooling

While tools exist, they are not as standardized or widely adopted as container-based CI/CD pipelines.

d. Harder to find people for it

Most developers and DevOps engineers are trained in container technologies, making hiring and collaboration easier in container-based environments. However for senior hard core sysadmins and system engineers that could be also advantage as not so many people have an indepth insight with both freebsd and fbsd jails.

This guide walks through a practical, production-style setup: 10 FreeBSD servers, each running isolated jails that host a classic LAMP stack (Linux, here replaced by FreeBSD, Apache, MySQL/MariaDB, PHP).
However still the use of companies or individuals who choose freebsd jails aim to better focus is on repeatability, clean architecture, and operational sanity, not just getting it to run once.

Architecture Overview of sample FBSD Cluster

Our Goal:

10 physical or virtual servers
Each server runs multiple jails
Each jail runs a LAMP app instance
Load balancing across nodes (to have a High Availability Cluster like setup)

Host Setup:

2 × load balancer nodes (nginx or HAProxy)
6 × application nodes (Apache + PHP in jails)
2 × database nodes (MariaDB primary/replica)

All systems run FreeBSD, using native jails for isolation.

1. Base FreeBSD Installation (All 10 Servers)

Install FreeBSD on each machine (minimal install is fine).

Update system:

# freebsd-update fetch install
# pkg update && pkg upgrade -y

Install base tools:

# pkg install -y sudo vim bash git

2. Install Jail Management tool (iocage)

We’ll use iocage, a modern jail manager.

# pkg install -y iocage
# sysrc iocage_enable="YES"
# service iocage start

Activate ZFS (recommended):

# zpool create zroot /dev/da0

Initialize iocage:

# iocage activate zroot
# iocage fetch

3. Create a Reusable Jail Template

Instead of building each jail manually, create a golden template.

# iocage create -n lamp-template -r 13.2-RELEASE ip4_addr="vnet0|10.0.0.10/24" boot=off
# iocage start lamp-template
# iocage console lamp-template

4. Install LAMP Stack Inside the Jail

Inside the jail:

4.1. Install Apache

# pkg install -y apache24
# sysrc apache24_enable="YES"

4.2. Install MariaDB

# pkg install -y mariadb106-server
# sysrc mysql_enable="YES"

Initialize DB:

service mysql-server start
mysql_secure_installation

4.3. Install PHP pre-compiled ports

# pkg install -y php82 php82-mysqli php82-mbstring php82-opcache

Configure Apache to use PHP:

# echo 'LoadModule php_module libexec/apache24/libphp.so' >> /usr/local/etc/apache24/httpd.conf
# echo 'AddType application/x-httpd-php .php' >> /usr/local/etc/apache24/httpd.conf

5. Test LAMP Stack works OK

Create a test file:

# echo "<?php phpinfo(); ?>" > /usr/local/www/apache24/data/index.php

Start services:

service apache24 start

Visit the jail IP and confirm PHP (page output) works in Firefox / Chrome Browser.

6. Convert Template into Clones

Stop Jail and snapshot:

iocage stop lamp-template
iocage snapshot lamp-template@base

Clone for production:

iocage clone lamp-template -n app01 ip4_addr="vnet0|10.0.0.21/24"
iocage clone lamp-template -n app02 ip4_addr="vnet0|10.0.0.22/24"

Repeat across servers and once working create a small shell script to run as a cron job to create backups automated.

Each server might run 5 up to 20 jails depending on resources.

7. Networking Between Jails

Use VNET for proper isolation:

Enable bridge on host:

# ifconfig bridge0 create
# ifconfig bridge0 addm em0 up

Assign jail interfaces automatically via iocage.

8. Load Balancing Layer

On 2 dedicated nodes, install nginx:

# pkg install -y nginx
# sysrc nginx_enable="YES"

Example config:

http {
    upstream backend {
        server 10.0.0.21;
        server 10.0.0.22;
        server 10.0.1.21;
        server 10.0.1.22;
    }
    server {
        listen 80;
        location / {
            proxy_pass http://backend;
        }
    }
}

9. Database Strategy

You have few options to choose from:

a. Use Centralized DB

Dedicated DB jails on 2 nodes
Primary + replica

b. Use Per-node DB (simpler)

Each jail has its own MariaDB
Use app-level replication if needed

10. Automation Across 10 Servers

Use tools like:

Ansible
SSH scripts
ZFS replication

Example (simple parallel execution loop) or use a set of scripts to handle updating with some Ansible Playbooks or Puppet:

# for host in server{1..10}; do
ssh $host "pkg update"
done

Few more Operational Tips to consider

a. Tune up setup / Do Resource management

Limit jail CPU/memory using rctl
Avoid overcommitting RAM

b. Use Centralized Logging

Centralize logs via syslog or ELK stack / Elasticsearch (see previous Build Central Logging server to Collect, Stire and Visualize Logs)

c. Do regular jail Backups

Use ZFS snapshots to backup each of the Jails:

# zfs snapshot zroot/iocage/jails/app01@backup
…

d. Tighten Security

Disable root SSH
Use PF firewall on host
Keep jails minimal

e. Do a Further Scaling Strategy

Add more servers -> replicate template
Add more jails -> clone snapshots
Scale horizontally via load balancer

Summary and Last Thoughts

When Choose FBSD Jails and when Containers

Use jails when you control the infrastructure, need maximum efficiency, and value simplicity (e.g., appliances, CDNs, storage systems).
Use containers when portability, scalability, and integration with modern DevOps workflows are critical.

This setup plays to the strengths of FreeBSD jails:

1. Performance: near-native speed
2.Isolation: strong and predictable
3. Simplicity: fewer layers than container stacks

FreeBSD jails remain a powerful and efficient isolation mechanism, particularly well-suited for controlled, performance-sensitive environments. Containers, however, dominate in modern application deployment due to their flexibility and ecosystem. The choice ultimately depends on whether you prioritize system-level control or platform-level convenience.

You won’t get the ecosystem of tools like Docker or Kubernetes, but you gain control, stability, and efficiency, which is exactly why companies like Netflix still rely on this model in critical infrastructure.

Tags: apache, application deployment, backend deployment, backups, Choosing, clone, Cloud Computing, containerization, create, devops, Docker, file, freebsd, FreeBSD jails, host, Host Setup, ifconfig, infrastructure, infrastructure design, Initialize, iocage, jail management, jails vs containers, Kubernetes, LAMP stack, libphp, Load Balancing Layer, mariadb, microservices, network performance, OS-level virtualization, performance optimization, php, pkg, running, scalable architecture, server infrastructure, simplicity, snapshots, ssh, Start, system administration, system isolation, Unix systems, virtualization, VNET, ZFS
Posted in Automation, Cloud services, Clusters, Curious Facts, FreeBSD, Linux, System Administration, System Optimization | No Comments »

How to Easily Integrate AI Into Bash on Linux with ollama

Monday, January 12th, 2026

AI is more and more entering the Computer scene and so is also in the realm of Computer / Network management. Many proficient admins already start to get advantage to it.
AI doesn’t need a GUI, or a special cloud dashboard or fancy IDE so for console geeks Sysadmins / System engineers and Dev Ops it can be straightly integrated into the bash / zsh etc. and used to easify a bit your daily sys admin tasks.

If you live in the terminal, the most powerful place to add AI is Bash itself. With a few tools and a couple of lines of shell code, you can turn your terminal into an AI-powered assistant that writes commands, explains errors, and helps automate everyday Linux tasks.

No magic. No bloat. Just Unix philosophy with a brain.

1. AI as a Command-Line Tool

Instead of treating AI like a chatbot, treat it like any other CLI utility:

stdin → prompt
stdout → response
pipes → integration

Hardware Requirements of Ollama Server

2. Minimum VM (It runs, but you’ll hate it)

Only use this to test that things work.

vCPU: 2
RAM: 4 GB
Disk: 20 GB (SSD mandatory)
Storage type: ( HDD / network storage = bad )
Models: phi3, tinyllama

Expect:

Very slow pulls
Long startup times
Laggy responses

a. Recommended VM (Actually usable)

This is the sweet spot for Bash integration and daily CLI use.

vCPU: 4–6 (modern host CPU)
RAM: 8–12 GB
Disk: 30–50 GB local SSD
CPU type: host-passthrough (important!)
NUMA: off (for small VMs)

Models that feel okay:

phi3
mistral
llama3:8b (slow but tolerable)

b. “Feels Good” VM (CPU-only but not painful)

If you want it to feel responsive.

vCPU: 8
RAM: 16 GB
Disk: NVMe-backed storage
CPU flags: AVX2 enabled
Hugepages: optional but nice

Models:

llama3:8b
codellama:7b
mixtral (slow, but usable)

Hypervisor-Specific Advice (Important)

c. KVM / Proxmox (Best choice)

CPU type: host
Enable AES-NI, AVX, AVX2
Use virtio-scsi
Cache: writeback
IO thread: enabled

d. If running VM on VMware platform

Enable Expose hardware-assisted virtualization
Use paravirtual SCSI
Reserve memory if possible

e. VirtualBox (Not recommended)

Poor CPU feature exposure
Weak IO performance

Avoid if you can

f. Local AI With Ollama (Recommended)

If you want privacy, low latency, and no API keys, Ollama is currently the easiest way to run LLMs locally on Linux.

3. Install Ollama

# curl -fsSL https://ollama.com/install.sh | sh

Start the service:

# ollama serve

Pull a model (lightweight and fast):

# ollama pull llama3

Test it:

ollama run llama3 "Explain what the ls command does"

If that works, you’re ready to integrate.

To check whether all is properly setup after installed llama3:

root@haproxy2:~# ollama list
NAME ID SIZE MODIFIED
llama3:latest 365c0bd3c000 4.7 GB About an hour ago

root@haproxy2:~# systemctl status ollama
● ollama.service – Ollama Service
Loaded: loaded (/etc/systemd/system/ollama.service; enabled; preset: enabled)
Active: active (running) since Mon 2026-01-12 16:43:30 EET; 15min ago
Main PID: 37436 (ollama)
Tasks: 16 (limit: 6999)
Memory: 5.0G
CPU: 13min 5.264s
CGroup: /system.slice/ollama.service
├─37436 /usr/local/bin/ollama serve
└─37472 /usr/local/bin/ollama runner –model /usr/share/ollama/.ollama/models/blobs/sha256-6a0746a1ec1aef3e7ec53>
яну 12 16:45:34 haproxy2 ollama[37436]: llama_context: Flash Attention was auto, set to enabled
яну 12 16:45:34 haproxy2 ollama[37436]: llama_context: CPU compute buffer size = 258.50 MiB
яну 12 16:45:34 haproxy2 ollama[37436]: llama_context: graph nodes = 999
яну 12 16:45:34 haproxy2 ollama[37436]: llama_context: graph splits = 1
яну 12 16:45:34 haproxy2 ollama[37436]: time=2026-01-12T16:45:34.959+02:00 level=INFO source=server.go:1376 msg="llama runner>
яну 12 16:45:34 haproxy2 ollama[37436]: time=2026-01-12T16:45:34.989+02:00 level=INFO source=sched.go:517 msg="loaded runners>
яну 12 16:45:35 haproxy2 ollama[37436]: time=2026-01-12T16:45:35.000+02:00 level=INFO source=server.go:1338 msg="waiting for >
яну 12 16:45:35 haproxy2 ollama[37436]: time=2026-01-12T16:45:35.001+02:00 level=INFO source=server.go:1376 msg="llama runner>
яну 12 16:55:58 haproxy2 ollama[37436]: [GIN] 2026/01/12 – 16:55:58 | 200 | 3.669915ms | 127.0.0.1 | HEAD "/"
яну 12 16:55:58 haproxy2 ollama[37436]: [GIN] 2026/01/12 – 16:55:58 | 200 | 42.244006ms | 127.0.0.1 | GET "/api/>
root@haproxy2:~#

4. Turning AI Into a Bash Command

Let’s create a simple AI helper command called ai.

Add this to your ~/.bashrc or ~/.bash_aliases:

ai() {
ollama run llama3 "$*"
}

Reload your shell:

$ source ~/.bashrc

Now you can do things like:

$ ai "Write a bash command to find large files"

$ ai "Explain this error: permission denied"
$ ai "Convert this sed command to awk"

At this point, AI is already a first-class CLI tool.

5. Using AI to Generate Bash Commands

One of the most useful patterns is asking AI to output only shell commands.

Example:

$ ai "Give me a bash command to recursively find .log files larger than 100MB. Output only the command."

Copy, paste, done.

You can even enforce this behavior with a wrapper:

aicmd() {
ollama run llama3 "Output ONLY a valid bash command. No explanation. Task: $*"
}

Now:

$ aicmd "list running processes using more than 1GB RAM"

Danger note: always read commands before running them. AI is smart, not trustworthy.

6. AI for Explaining Commands and Logs

This is where AI shines.

Pipe output directly into it:

$ dmesg | tail -n 50 | ai "Explain what is happening here"

Or errors:

$ make 2>&1 | ai "Explain this error and suggest a fix"

You’ve just built a terminal-native debugger.

7. Smarter Bash History Search

You can even use AI to interpret your intent instead of remembering exact commands:

aih() {
history | ai "From this bash history, find the best command for: $*"
}

Example:

$ aih "compress a directory into tar.gz"

It’s like Ctrl+R, but semantic.

8. Using AI With Shell Scripts

AI can help generate scripts inline:

$ ai "Write a bash script that monitors disk usage and sends a notification when it exceeds 90%"

You’re not replacing scripting skills – you’re accelerating them.

Think of AI as:

a junior sysadmin
a documentation search engine
a rubber duck that talks back

9. Where Ollama Stores Its Data

Depending on how it runs:

System service (most common)

Models live here:

/usr/share/ollama/.ollama/

Inside:

models/
blobs/

User-only install

~/.ollama/

Since you installed as root + systemd, use the first path.

See What’s Taking Space

# du -sh /usr/share/ollama/.ollama/*

Typical output:

models/ → metadata
blobs/ → the big files (GBs)

10. Remove Unused Models (Safe Way)

List models Ollama knows about:

# ollama list

Remove a model properly:

# ollama rm llama3

This removes metadata and unreferenced blobs.

Always try this first.

11. Full Manual Cleanup (Hard Reset)

If things are broken, stuck, or you want a clean slate:

Stop Ollama

# systemctl stop ollama

Delete all local models and cache

# rm -rf /usr/share/ollama/.ollama/models
# rm -rf /usr/share/ollama/.ollama/blobs

(Optional but safe)

# rm -rf /usr/share/ollama/.ollama/tmp

Start Ollama again

# systemctl start ollama

Ollama will recreate everything automatically.

Verify Cleanup Worked

# du -sh /usr/share/ollama/.ollama
# ollama list

You should see:

Very small disk usage
Empty model list

Prevent Disk Bloat (Highly Recommended)

Only pull small models on VMs

Stick to:

phi3
mistral
tinyllama

Remove models you don’t use

# ollama rm modelname

Set a custom data directory (optional)

If /usr is small, move Ollama data:

# systemctl stop ollama
# mkdir -p /opt/ollama-data
# chown -R ollama:ollama /opt/ollama-data

Edit service:

# systemctl edit ollama

Add:

[Service]
Environment=OLLAMA_HOME=/opt/ollama-data

Then:

# systemctl daemon-reload
# systemctl start ollama

Quick “Nuke It” One-Liner (Use With Care)

Deletes everything Ollama-related:

# systemctl stop ollama && rm -rf /usr/share/ollama/.ollama && systemctl start ollama
API-Based Option (Cloud Models)

If you prefer cloud models (OpenAI, Anthropic, etc.), the pattern is identical:

Use curl
Pass prompt
Parse output

Once AI returns text to stdout, Bash doesn’t care where it came from.

12. Best Practices how to use shell AI to not overload machine

Before you go wild:

Don’t auto-execute AI output
Don’t run AI as root
Treat responses as suggestions
Version-control important scripts
Keep prompts specific

AI is powerful — but Linux still assumes you know what you’re doing.

Sum it up

Adding AI to Bash isn’t about replacing skills.
It’s about removing friction.

When AI gets easy to use from the command line it is a great convenience for those who don't want to switch to browser all the time and copy / paste like crazy.
AI as a command-line tool fits perfectly into the Linux console:

composable
scriptable
optional
powerful

Once you’ve used AI from inside your shell for a few hours, going back to browser-based AI chat stuff like querying ChatGPT feels… slow and inefficient.
However keep in mind that ollama is away from perfect and has a lot of downsides and ChatGPT / Grok / DeepSeek often might give you better results, however as ollama is really isolated and non-depend on external sources your private quries will not get into a public AI historic database and you won't be tracked.
So everything has its Pros and Cons. I'm pretty sure that this tool and free AI tools like those will certainly have a good future and will be heavily used by system admins and
programmers in the coming future.
The terminal just got smarter. And it didn’t need a GUI to do it.

Tags: bashrc, brain, buffer size, copy paste, Delete, Edit, find, Flash Attention, Generate Bash Commands, graph, identical, linux?, Output, running processes, Smarter Bash History Search, Start, terminal, time, times, wrapper
Posted in AI on Linux, Automation, Educational, System Administration, Various | No Comments »

How to Set Up SSH Two-Factor Authentication (2FA) on Linux Without Google Authenticator with OATH Toolkit

Wednesday, November 5th, 2025

Most tutorials online on how to secure your SSH server with a 2 Factor Authentication 2FA will tell you to use Google Authenticator to secure SSH logins.

But what if you don’t want to depend on Google software – maybe for privacy, security, or ideological reasons ?

Luckily, you have a choice thanks to free oath toolkit.
The free and self-hosted alternative: OATH Toolkit has its own PAM module libpam-oath to make the 2FA work the openssh server.

OATH-Toolkit is a free software toolkit for (OTP) One-Time Password authentication using HOTP/TOTP algorithms. The software ships a small set of command line utilities covering most OTP operation related tasks.

In this guide, I’ll show you how to implement 2-Factor Authentication (TOTP) for SSH on any Linux system using OATH Toolkit, compatible with privacy-friendly authenticator apps like FreeOTP, Aegis, or and OTP.

It is worthy to check out OATH Toolkit author original post here, that will give you a bit of more insight on the tool.

1. Install the Required Packages

For Debian / Ubuntu systems:

# apt update
# apt install libpam-oath oathtool qrencode
...

For RHEL / CentOS / AlmaLinux:

# dnf install pam_oath oathtool

The oathtool command lets you test or generate one-time passwords (OTPs) directly from the command line.

2. Create a User Secret File

libpam-oath uses a file to store each user’s secret key (shared between your server and your phone app).

By default, it reads from:

/etc/users.oath

Let’s create it securely and set proper permissions to secure it:

# touch /etc/users.oath
# chmod 600 /etc/users.oath

Now, generate a new secret key for your user (replace hipo with your actual username):

# head -10 /dev/urandom | sha1sum | cut -c1-32

This generates a random 32-character key.
Example:

9b0e4e9fdf33cce9c76431dc8e7369fe

Add this to /etc/users.oath in the following format:

HOTP/T30 hipo - 9b0e4e9fdf33cce9c76431dc8e7369fe

HOTP/T30 means Time-based OTP with 30-second validity (standard TOTP).

Replace hipo with the Linux username you want to protect.

3. Add the Key to Your Authenticator App

Now we need to add that secret to your preferred authenticator app.

You can create a TOTP URI manually (to generate a QR code):

$ echo "otpauth://totp/hipo@jericho?secret=\
$(echo 9b0e4e9fdf33cce9c76431dc8e7369fe \
| xxd -r -p | base32)"

You can paste this URI into a QR code generator (e.g., https://qr-code-generator.com) and scan it using FreeOTP , Aegis, or any open TOTP app.
The FreeOTP Free Ap is my preferred App to use, you can install it via Apple AppStore or Google Play Store.

Alternatively, enter the Base32-encoded secret manually into your app:

# echo 9b0e4e9fdf33cce9c76431dc8e7369fe | xxd -r -p | base32

You can also use qrencode nice nifty tool to generate out of your TOTP code in ASCII mode and scan it with your Phone FreeOTP / Aegis App and add make it ready for use:

# qrencode –type=ANSIUTF8 otpauth://totp/hipo@jericho?secret=$( oathtool –verbose –totp 9b0e4e9fdf33cce9c76431dc8e7369fe –digits=6 -w 1 | grep Base32 | cut -d ' ' -f 3 )\&digits=6\&issuer=pc-freak.net\&period=30

qrencode will generate the code. We set the type to ANSI-UTF8 terminal graphics so you can generate this in an ssh login. It can also generate other formats if you were to incorporate this into a web interface. See the man page for qrencode for more options.
The rest of the line is the being encoded into the QR code, and is a URL of the type otpauth, with time based one-time passwords (totp). The user is “hipo@jericho“, though PAM will ignore the @jericho if you are not joined to a domain (I have not tested this with domains yet).

The parameters follow the ‘?‘, and are separated by ‘&‘.

otpauth uses a base32 hash of the secret password you created earlier. oathtool will generate the appropriate hash inside the block:

$( oathtool –verbose –totp 9b0e4e9fdf33cce9c76431dc8e7369fe | grep Base32 | cut -d ' ' -f 3 )

We put the secret from earlier, and search for “Base32”. This line will contain the Base32 hash that we need from the output:

Hex secret: 9b0e4e9fdf33cce9c76431dc8e7369fe
Base32 secret: E24ABZ2CTW3CH3YIN5HZ2RXP
Digits: 6
Window size: 0
Step size (seconds): 30
Start time: 1970-01-01 00:00:00 UTC (0)
Current time: 2022-03-03 00:09:08 UTC (1646266148)
Counter: 0x3455592 (54875538)

368784
From there we cut out the third field, “E24ABZ2CTW3CH3YIN5HZ2RXP“, and place it in the line.

Next, we set the number of digits for the codes to be 6 digits (valid values are 6, 7, and 8). 6 is sufficient for most people, and easier to remember.

The issuer is optional, but useful to differentiate where the code came from.

We set the time period (in seconds) for how long a code is valid to 30 seconds.

Note that: Google authenticator ignores this and uses 30 seconds whether you like it or not.

4. Configure PAM to Use libpam-oath

Edit the PAM configuration for SSH:

# vim /etc/pam.d/sshd

At the top of the file, add:

auth required pam_oath.so usersfile=/etc/users.oath window=30 digits=6

This tells PAM to check OTP codes against /etc/users.oath.

5. Configure SSH Daemon to Ask for OTP

Edit the SSH daemon configuration file:

# vim /etc/ssh/sshd_config

Ensure these lines are set:

UsePAM yes
challengeresponseauthentication yes
ChallengeResponseAuthentication yes
AuthenticationMethods publickey keyboard-interactive
##KbdInteractiveAuthentication no
KbdInteractiveAuthentication yes

N.B.! The KbdInteractiveAuthentication yes variable is necessery on OpenSSH servers with version > of version 8.2_ .

In short This setup means:
1. The user must first authenticate with their SSH key (or local / LDAP password),
2. Then enter a valid one-time code generated from TOTP App from their phone.

You can also use Match directives to enforce 2FA under certain conditions, but not under others.
For example, if you didn’t want to be bothered with it while you are logging in on your LAN,
but do from any other network, you could add something like:

Match Address 127.0.0.1,10.10.10.0/8,192.168.5.0/24
Authenticationmethods publickey

6. Restart SSH and Test It

Apply your configuration:

# systemctl restart ssh

Now, open a new terminal window and try logging in (don’t close your existing one yet, in case you get locked out):

$ ssh hipo@your-server-ip

You should see something like:

Verification code:

Enter the 6-digit code displayed in your FreeOTP (or similar) app.
If it’s correct, you’re logged in! Hooray ! 🙂

7. Test Locally and Secure the Secrets

If you want to test OTPs manually with a base32 encrypted output of hex string:

# oathtool --totp -b \
9b0e4e9fdf33cce9c76431dc8e7369fe

As above might be a bit confusing for starters, i recommend to use below few lines instead:

$ secret_hex="9b0e4e9fdf33cce9c76431dc8e7369fe"
$ secret_base32=$(echo $secret_hex | xxd -r -p | base32)
$ oathtool –totp -b "$secret_base32"
156874

You’ll get the same 6-digit code your authenticator shows – useful for debugging.

If you rerun the oathtool again you will get a difffefrent TOTP code, e.g. :

$ oathtool –totp -b "$secret_base32"
258158

Use this code as a 2FA TOTP auth code together with local user password (2FA + pass pair), when prompted for a TOTP code, once you entered your user password first.

To not let anyone who has a local account on the system to be able to breach the 2FA additional password protection,
Ensure the secrets file is protected well, i.e.:

# chown root:root /etc/users.oath
# chmod 600 /etc/users.oath

How to Enable 2FA Only for Certain Users

If you want to force OTP only for admins, create a group ssh2fa:

# groupadd ssh2fa
# usermod -aG ssh2fa hipo

Then modify /etc/pam.d/sshd:

auth [success=1 default=ignore] pam_succeed_if.so \
user notingroup ssh2fa
auth required pam_oath.so usersfile=/etc/users.oath \
window=30 digits=6

Only users in ssh2fa will be asked for a one-time code.

Troubleshooting

Problem: SSH rejects OTP
Check /var/log/auth.log or /var/log/secure for more details.
Make sure your phone’s time is in sync (TOTP depends on accurate time).

Problem: Locked out after restart
Always keep one root session open until you confirm login works.

Problem: Everything seems configured fine but still the TOTP is not accepted by remote OpenSSHD.
– Check out the time on the Phone / Device where the TOTP code is generated is properly synched to an Internet Time Server
– Check the computer system clock is properly synchornized to the Internet Time server (via ntpd / chronyd etc.), below is sample:

hipo@jeremiah:~$ timedatectl status
Local time: Wed 2025-11-05 00:39:17 EET
Universal time: Tue 2025-11-04 22:39:17 UTC
RTC time: Tue 2025-11-04 22:39:17
Time zone: Europe/Sofia (EET, +0200)
System clock synchronized: yes
NTP service: n/a
RTC in local TZ: no

Why Choose libpam-oath?

100% Free Software (GPL)
Works completely offline / self-hosted
Compatible with any standard TOTP app (FreeOTP, Aegis, andOTP, etc.)
Doesn’t depend on Google APIs or cloud services
Lightweight (just one PAM module and a text file)

Conclusion

Two-Factor Authentication doesn’t have to rely on Google’s ecosystem.
With OATH Toolkit and libpam-oath, you get a simple, private, and completely open-source way to harden your SSH server against brute-force and stolen-key attacks.

Once configured, even if an attacker somehow steals your SSH key or password, they can’t log in without your phone’s one-time code – making your system dramatically safer.

Tags: Certain Users, configuration file, create, Edit, generate, hipo, Install, online, openssh, PAM, password, phone, Phone Device, sample, security, Start, string, terminal window, Test It, TOTP, User Secret File
Posted in Linux, Linux and FreeBSD Desktop, Security, System Administration | No Comments »

How to Run Your Own Windows Domain Authentication on Linux

Thursday, October 2nd, 2025

Run Your Own Domain Authentication on Linux

Running your own domain authentication system on Linux can significantly enhance security and manageability in your IT environment. Whether you're setting up centralized login for a small network or a more complex domain environment, Linux provides powerful tools to become your own domain controller using open-source software.

In this guide, we’ll walk you through setting up Samba as an Active Directory (AD) Domain Controller on a Linux server.
These tutorial should work fine on Debian 12 (Bookworm), though it should work with minor modifications on pretty much most of recent Debs and deb based distros.

What is Domain Authentication?

Domain authentication allows users to log in to any authorized machine within a network using the same set of credentials. It provides centralized management of:

Users and groups
Computer accounts
Group policies
File and printer sharing
Access control

Microsoft's Active Directory is the most well-known implementation, but you can achieve similar functionality using Samba on Linux.

Pre-requirements

A fresh Linux installation (Ubuntu Server 22.04 LTS or Debian 12 recommended)
Static IP address
Root or sudo access
Domain name (e.g., mydomain.local)

1. Update System and Set proper Hostname

# apt update && sudo apt upgrade -y
# hostnamectl set-hostname dc1.mydomain.local

Add the hostname to /etc/hosts:

# vim /etc/hosts

Add the local network IP the SMB Domain controller will have locally on the machine:

192.168.1.100 dc1.mydomain.local dc1

2. Install Samba and Required Packages

# apt install samba krb5-config krb5-user winbind smbclient dnsutils -y

During the installation, you may be prompted for Kerberos configuration:

Default realm: MYDOMAIN.LOCAL
KDC: dc1.mydomain.local
Admin server: dc1.mydomain.local

3. Provision Samba as a Domain Controller

First, stop any running Samba services:

# systemctl stop smbd nmbd winbind
# systemctl disable smbd nmbd winbind

Move default config:

# mv /etc/samba/smb.conf /etc/samba/smb.conf.bak

Now provision the domain:

# samba-tool domain provision –use-rfc2307 –interactive

Answer prompts:

Realm: MYDOMAIN.LOCAL
Domain: MYDOMAIN
Server role: dc
DNS backend: SAMBA_INTERNAL
Admin password: (choose a strong one)

Once done, configure Kerberos using the samba krb5.conf template file:

# mv /etc/krb5.conf /etc/krb5.conf.bak
# cp /var/lib/samba/private/krb5.conf /etc/

4. Start and Enable Samba AD Services

# systemctl unmask samba-ad-dc
# systemctl enable samba-ad-dc –now

Verify it’s working by running:

# samba-tool domain level show

Check Kerberos authentication is OK:

# kinit administrator
# klist

You should see a valid Kerberos ticket.

5. Configure DNS (Optional but Recommended)

If using SAMBA_INTERNAL DNS backend:

Check DNS resolution is OK:

# host -t A dc1.mydomain.local
# host -t SRV _kerberos._udp.mydomain.local

If you want clients to resolve domain names, configure them to use the Samba DC's IP as their DNS server.

6. Add Users and Join Client Machines

Add a new user:

# samba-tool user add your.samba.user

Join a Windows client:

Go to System Properties → Computer Name → Change settings
Click Domain, enter MYDOMAIN
Authenticate with Administrator and the password you set
Reboot

7. Managing the Domain

You can manage users, groups, and policies simply via commands or GUI interface or LDAP tools:

samba-tool (CLI)
RSAT tools on Windows (for GUI management)
via LDAP tools (if you have to stick to RFC2307)

Example commands:

# samba-tool user list
# samba-tool group list
# samba-tool user setpassword your.samba.user

8. Managing Samba AD Samba Linux Domain easily with UI

You can manage a Samba domain (especially when it's running as an Active Directory Domain Controller) via a web interface — but not directly through Samba itself, since it doesn't come with a built-in web UI.

Instead, you can integrate Samba with third-party web-based tools that provide management interfaces for:

Users and groups
Computer accounts
LDAP directory entries
Domain policies (to a limited extent)

Popular Web Interfaces to Manage a Samba Domain

Here are the most reliable options:

8.1. [Cockpit + 389 Directory Server or FreeIPA (for LDAP-based domains)]

Cockpit is a modern web admin interface for Linux servers.
When paired with FreeIPA, you can manage users, groups, policies, and more.
However, this is more suited for FreeIPA-based domains, not Samba AD.

✅ Great for: Linux-native domains
❌ Not compatible with Windows-style Samba AD

8.2. [LDAP Account Manager (LAM)] – RECOMMENDED FOR SAMBA + AD

Website: https://www.ldap-account-manager.org/

LDAP Account Manager (LAM) is one of the best tools to manage a Samba domain via LDAP, especially when:

You use Samba in AD DC mode with RFC2307 extensions (for Unix attributes)
Or, you're using Samba as a member server with an external LDAP backend

Features:

Web-based GUI to manage:
- Users and groups
- Samba-specific attributes (like SID, RID, home directories)
- POSIX and Windows-compatible accounts
Can bind directly to the Samba LDAP directory

Authentication: Admin binds via LDAP (either over plain or TLS)

✅ Works with Samba AD (with some config)
✅ Handles Samba3/4 user schemas
✅ Active development and documentation

8.3. Samba Web Administration Tool (SWAT) ❌ Deprecated

SWAT was the original web interface for Samba but:

It was deprecated and removed from Samba after version 4.1
It's no longer secure or maintained
Not suitable for Samba AD DC environments

Recommendation: Do not use SWAT

8.4. Webmin (Partial Support)

Webmin is a general Linux web admin tool
It has a Samba module, but:
- Designed for traditional Samba file sharing (not AD/DC mode)
- Cannot manage Samba AD users/groups
- Doesn’t interact with samba-tool or the AD schema

✅ Works for standalone Samba file servers
❌ Not suitable for Samba AD DCs

Can You really Use RSAT Instead ?

If you want full Active Directory-style control (like Group Policy, OU structure, DNS, etc.), the best GUI tool is actually RSAT (Remote Server Administration Tools) on Windows
but for that of course you will have to have an own Windows Server setup especailly for it.

Connects to your Samba AD DC
Fully supports:
- Users and groups
- Group Policy Objects (GPO)
- DNS management (if using internal Samba DNS)

Install RSAT on a Windows machine and run dsa.msc (Active Directory Users and Computers).

✅ Officially supported
✅ Full compatibility with Samba AD
❌ Requires a Windows machine

Summary: Web UI for Samba Domain Management

Tool	Works with Samba AD DC?	Features	Notes
LDAP Account Manager (LAM)	Yes	User/group management	Best web option
Cockpit + FreeIPA	❌ No (not Samba AD)	Excellent for FreeIPA domains	Not compatible with Samba AD
Webmin	❌ Not fully	File shares only	No AD/DC management
RSAT (Windows)	✅ Yes	Full AD management	Not web-based

Recommendation

If you're running a Samba AD DC and want a web-based interface:

Use LAM (LDAP Account Manager) for basic account management
Use RSAT tools on Windows for full domain administration
Avoid SWAT and Webmin for this purpose

Security Considerations

Ensure firewall allows relevant ports (e.g., 53, 88, 389, 445, etc.) with Iptables / firewalld or whatever firewall solution you have present on the server and in the Network in which you hosted the server
Keep the system updated
Use secure passwords and rotate them regularly
Consider setting up replication if high availability is needed

Conclusion

Running your own domain authentication system on Linux using Samba is a powerful way to control user access in a centralized manner. It’s ideal for small to mid-sized networks, homelabs, or even enterprise environments looking for a cost-effective alternative to Windows Server.

With Samba acting as your domain controller, you can enjoy the benefits of centralized authentication, integrated DNS, and a high degree of compatibility with Windows clients — all while staying in the open-source ecosystem.

References

Samba Wiki: Setting up Samba as an AD Domain Controller
man samba-tool
man smb.conf

Notes and things to consider:

/var/lib/samba/private/krb5.conf file is generated only after you provision Samba as an Active Directory (AD) Domain Controller using:

# samba-tool domain provision

After provisioning, Samba creates a custom Kerberos config at:

/var/lib/samba/private/krb5.conf

This is true for both Debian and Ubuntu because it's handled by the Samba package itself, not the distro.

Why use that krb5.conf instead of Debian's default?

Well because:

The default /etc/krb5.conf on Debian isn't tailored for Samba AD.
The one Samba generates includes correct realm, KDC, and admin server settings.
It avoids subtle issues like failed kinit or broken Kerberos trust.

So you copy it over Debian’s default:

Gotchas on Debian to be aware of

Do not install samba via tasksel (like tasksel's “Samba file server” role), as it sets up a traditional SMB server, not AD.

Only use samba-tool domain provision if you're setting up AD DC.

Debian sometimes separates systemd services (e.g., samba-ad-dc might not be enabled by default). So make sure to enable samba-ad-dc instead of smbd/nmbd.

Tags: Active Directory Domain Controller, configure, Debian, DNS, LDAP, linux?, mydomain, OK, options, Start, Users
Posted in Everyday Life, Linux, Samba, System Administration, Various | No Comments »

How to Install and use FreeIPA forcentralized SSO authention on Linux computer domain

Wednesday, October 1st, 2025

FreeIPA is a popular open-source identity management solution that centralizes user, host, and service authentication for Linux environments. It combines LDAP, Kerberos, DNS, and certificate management into a single platform, making it easier to manage large Linux deployments securely.

In this article, we’ll cover how to install FreeIPA on a Linux server, perform initial configuration, and start using it for basic user management.

Prerequisites

A clean Linux server (CentOS, RHEL, Fedora, or similar)
Root or sudo access
A fully qualified domain name (FQDN) for your server (e.g., ipa.example.com)
Proper DNS setup (recommended but can be configured during installation)

1. Update system to the latest

Start by updating your system to ensure all packages are current.

# dnf update -y

2. Install FreeIPA Server Packages

Install the FreeIPA server and its dependencies:

# dnf install -y ipa-server ipa-server-dns

ipa-server-dns is optional but recommended if you want FreeIPA to manage DNS for your domain.

3. Configure FreeIPA server

Run the FreeIPA installation script to configure the server. Replace ipa.example.com with your actual server hostname.

sudo ipa-server-install

You will be prompted for:

Realm name: Usually uppercase of your domain, e.g., EXAMPLE.COM
Directory Manager password: LDAP admin password
IPA admin password: FreeIPA admin user password
DNS configuration: Enable if you want FreeIPA to manage DNS

Sample configuration flow:

Realm name: EXAMPLE.COM

DNS domain name: example.com

Server host name: ipa.example.com

Directory Manager password: [choose a strong password]

IPA admin password: [choose a strong password]

Do you want to configure integrated DNS (BIND)? [yes/no]: yes

The installer will set up Kerberos, LDAP, the CA, DNS (if chosen), and the Web UI.

4. Start and Enable FreeIPA Services

The installer usually starts services automatically, but you can verify with:

# systemctl status ipa

Enable the service to start on boot:

# systemctl enable ipa

5. Access FreeIPA Web Interface

Open your browser and navigate to:

https://ipa.example.com/ipa/ui/

6. Add Users and Groups

You can manage users and groups either via the Web UI or the CLI.

Using the CLI:

Add a new user:

# ipa user-add johndoe –first=John –last=Doe –email=johndoe@example.com

Set a password for the new user:

# ipa passwd johndoe

Add a new group:

# ipa group-add developers –desc="Development Team"

Add user to the group:

# ipa group-add-member developers –users=johndoe

7. Join Client Machines to the FreeIPA Domain

On a client machine, install the client packages:

# dnf install -y ipa-client

Run the client setup:

# ipa-client-install –mkhomedir

Follow the prompts to join the client to the FreeIPA domain.

8. Test Authentication

Try logging into the client machine with the FreeIPA user you created:

# ssh username@client-machine-host.com

You should be able to authenticate using the FreeIPA credentials.

Conclusion

You now have a basic FreeIPA server up and running, managing users and authentication across your Linux network. FreeIPA simplifies identity management by providing a centralized, secure, and integrated solution. From here, you can explore advanced features like role-based access control, host-based access control, and certificate management.

Here's a practical example of how FreeIPA can be used in a real-world Linux environment.

Scenario: Centralized Authentication in a DevOps Environment

Tech Problem

Lets say you are managing a growing team of DevOps engineers and developers across multiple Linux servers (e.g., for CI/CD, staging, and production). Manually creating and maintaining user accounts, SSH keys, and sudo permissions on each server is:

Time-consuming
Error-prone
A security risk (inconsistent policies, orphaned accounts)

Solution: Use FreeIPA to Centralize Identity & Access Management

By deploying FreeIPA, you can:

Create user accounts once and manage them centrally
Enforce SSO across servers using Kerberos
Automatically apply sudo rules, group permissions, and access control policies
Easily revoke access for offboarded employees
Use host-based access control (HBAC) to control who can log in to what

Solution Walkthrough

1. Set up FreeIPA server

Installed on: ipa.internal.example.com
Domain: internal.example.com
Realm: INTERNAL.EXAMPLE.COM

2. Add User Accounts

Let's add two users: alice (developer) and bob (DevOps).

# ipa user-add alice –first=Alice –last=Smith –email=alice@internal.example.com
# ipa user-add bob –first=Bob –last=Jones –email=bob@internal.example.com
# ipa passwd alice
# ipa passwd bob

3. Create Groups and Roles necessery

Create functional groups for managing permissions.

# ipa group-add developers –desc="Developers Team"
# ipa group-add devops –desc="DevOps Team"
# ipa group-add-member developers –users=alice
# ipa group-add-member devops –users=bob

4. Configure Sudo Rules

Let’s allow DevOps team members to use sudo on all servers:

# ipa sudorule-add devops-sudo –cmdcat=all
# ipa sudorule-add-user devops-sudo –groups=devops
# ipa sudorule-add-host devops-sudo –hostgroups=all

5. Control Access with HBAC Rules

Let’s say:

Developers can access dev and staging servers
DevOps can access all servers

# Create host groups

# ipa hostgroup-add dev-servers –desc="Development Servers"
# ipa hostgroup-add staging-servers –desc="Staging Servers"

# Add hosts to groups

# ipa hostgroup-add-member dev-servers –hosts=dev1.internal.example.com
# ipa hostgroup-add-member staging-servers –hosts=staging1.internal.example.com

# HBAC rule for developers

# ipa hbacrule-add allow-developers
# ipa hbacrule-add-user allow-developers –groups=developers
# ipa hbacrule-add-host allow-developers –hostgroups=dev-servers
# ipa hbacrule-add-host allow-developers –hostgroups=staging-servers
# ipa hbacrule-add-service allow-developers –hbacsvcs=sshd

# HBAC rule for DevOps (all access)
# ipa hbacrule-add allow-devops
# ipa hbacrule-add-user allow-devops –groups=devops
# ipa hbacrule-add-host allow-devops –hostgroups=all
# ipa hbacrule-add-service allow-devops –hbacsvcs=sshd

6. Join Client Servers to FreeIPA

On each Linux server (e.g., dev1, staging1, prod1), run:

# ipa-client-install –mkhomedir –server=ipa.internal.example.com –domain=internal.example.com

Now, user alice can log in to dev1 and staging1, but not prod1. bob can log in to all servers and use sudo.

7. What Happens When Alice Leaves the Company?

Just disable the user in FreeIPA:

# ipa user-disable alice

This immediately revokes her access across all servers — no need to touch individual machines.

Benefits in This Example

Feature	Outcome
Centralized user management	No need to manually create accounts on every server
Group-based sudo	DevOps has privileged access, others don’t
Access control	Developers only access dev/staging, not prod
Kerberos SSO	Secure, passwordless SSH with ticketing
Auditing	Central logs of who accessed what and when
Quick offboarding	Instant account disablement from a single location

Summary

FreeIPA is not just a replacement for LDAP — it's a full-blown identity and access management solution tailored for Linux systems. In this practical example, it brings enterprise-grade access control, authentication, and user management to a DevOps workflow with minimal friction.

Tags: Add User Accounts, Configure Sudo Rules, DNS, example, flow, installation, internal, linux?, Open, Quick, recommended, Scenario Centralized Authentication, Set, setup, ssh, SSO, Start, Tech Problem
Posted in Computer Security, Linux, System Administration, Various | No Comments »

How to Install and Set Up an NFS Server network Shares on on Linux to easify data transfer across multiple hosts

Monday, April 7th, 2025

How to Configure NFS Server in Redhat,CentOS,RHEL,Debian,Ubuntu and Oracle Linux

Network File System (NFS) is a protocol that allows one system to share directories and files with others over a network. It's commonly used in Linux environments for file sharing between systems. In this guide, we'll walk you through the steps to install and set up an NFS server on a Linux system.

Prerequisites

Before you start, make sure you have:

A Linux system distros (e.g., Ubuntu, CentOS, Debian, etc.)
Root or sudo privileges on the system.
A network connection between the server (NFS server) and clients (machines that will access the shared directories).

1. Install NFS Server Package

On Ubuntu / Debian based Linux systems:

a. First, update the package list

# apt update

b. Install the NFS server package

# apt install nfs-kernel-server

On CentOS/REL-based systems:

2. Install the NFS server package

      # yum install nfs-utils

Once the package is installed, ensure that the necessary services are enabled.

3. Create Shared Directory for file sharing

Decide which directory you want to share over NFS. If the directory doesn't exist, you can create one. For example:

# mkdir -p /nfs_srv_dir/nfs_share

Make sure the directory has the appropriate permissions so that the nfs clients can access it.

# chown nobody:nogroup /nfs_srv_dir/nfs_share
# chmod 755 /nfs_srv_dir/nfs_share

4. Configure NFS Exports ( /etc/exports file)

The NFS exports file (/etc/exports) is perhaps most important file you will have to create and deal with regularly to define the expored shares, this file contains the configuration settings for directories you want to share with other systems.

a. Open the /etc/exports file for editing:

vi /etc/exports

Add an entry for the directory you want to share. For example, if you're sharing /nfs_srv_dir/nfs_share and allowing access to all systems on the network (192.168.1.0/24), add the following line:

/nfs_srv_dir/nfs_share 192.168.1.0/24(rw,sync,no_subtree_check)

Here’s what each option means:

rw: Read and write access.
sync: Ensures that changes are written to disk before responding to the client.

Here is few lines of example of my working /etc/exports on my home running NFS server

/var/www 192.168.0.209/32(rw,no_root_squash,async,subtree_check)
/home/jordan 192.168.0.209/32(rw,no_root_squash,async,subtree_check)
/mnt/sda1/icons-frescoes/ 192.168.0.209/32(rw,no_root_squash,async,subtree_check)
/home/mobfiles 192.168.0.209/32(rw,no_root_squash,async,subtree_check)
/mnt/sda1/icons-frescoes/ 192.168.0.200/32(rw,no_root_squash,async,subtree_check)
/home/hipo/public_html 192.168.0.209/32(rw,no_root_squash,async,subtree_check)
/home/alex/public_html 192.168.0.209/32(rw,no_root_squash,async,subtree_check)
/home/necroleak/public_html 192.168.0.209/32(rw,no_root_squash,async,subtree_check)
/bashscripts 192.168.0.209/32(rw,no_root_squash,async,subtree_check)
/backups/Family-Videos 192.168.0.200/32(ro,no_root_squash,async,subtree_check)

5. Export the NFS Shares with exportfs command

Once the export file is configured, you need to inform the NFS server to start sharing the directory:

# exportfs -a

The -a flag will make it export all the sharings.

6. Start and Enable NFS Services

You need to start and enable the NFS server so it will run on system boot.

On Ubuntu / Debian Linux run the following commands:

# systemctl start nfs-kernel-server
# systemctl enable nfs-kernel-server

On CentOS / RHEL Linux:

# systemctl start nfs-server
# systemctl enable nfs-server

7. Allow NFS Traffic Through the Firewall

If your server has a firewall configured / enabled, you will need to allow NFS-related ports through the firewall.
These ports include 2049 TCP protocol Ports (NFS) and 111 (RPCbind) UDP and TCP protocol , and some additional ports.

On Ubuntu/Debian (assuming you are using ufw [UNCOMPLICATED FIREWALL]):

# ufw allow from 192.168.1.0/24 to any port nfs sudo ufw reload

On CentOS / RHEL Linux:

# firewall-cmd –permanent –add-service=nfs sudo firewall-cmd –permanent –add-service=mountd sudo firewall-cmd –permanent –add-service=rpc-bind sudo firewall-cmd –reload

8. Verify NFS Server is Running

To ensure the NFS server is running properly, use the following command:

# systemctl status nfs-kernel-server

# systemctl status nfs-server

You should see output indicating that the service is active and running.

9. Test the NFS Share (Client-Side)

To test the NFS share, you will need to mount it on a client machine. Here's how to mount it:

On the client machine, install the NFS client utilities:

Ubuntu / Debian Linux

# apt install nfs-common

For CentOS / RHEL Linux

# yum install nfs-utils

Create a mount point (Nomatter the distro),:

# mkdir -p /mnt/nfs_share

Mount the NFS share:

# mount -t nfs <nfs_server_ip>:/nfs_srv_dir/nfs_share /mnt/nfs_share

Replace <nfs_server_ip> with the IP address of the NFS server or DNS host alias if you have one defined in /etc/hosts file.

Verify that the share is mounted:

# df -h

You should see the NFS share listed under the mounted file systems.

10. Configure Auto-Mount at Boot (Optional)

To have the NFS share automatically mounted at boot, you can add an entry to the /etc/fstab file on the client machine.

Open /etc/fstab for editing:

# vi /etc/fstab

Add the following line:

<server-ip>:/nfs_srv_dir/nfs_share /mnt/nfs_share nfs defaults 0 0

Save and close the file.

The NFS share will now be automatically mounted whenever the system reboots.

Debug NFS configuration issues (basics)

You can continue to modify the /etc/exports file to share more directories or set specific access restrictions depending on your needs.

If you encounter any issues, checking the server logs or using

# exportfs -v
/var/www     192.168.0.209/32(async,wdelay,hide,sec=sys,rw,secure,no_root_squash,no_all_squash)
/home/var_data     192.168.0.205/32(async,wdelay,hide,sec=sys,rw,secure,no_root_squash,no_all_squash)
/mnt/sda1/
       192.168.0.209/32(async,wdelay,hide,sec=sys,rw,secure,no_root_squash,no_all_squash)
/mnt/sda2/info
       192.168.0.200/32(async,wdelay,hide,sec=sys,rw,secure,no_root_squash,no_all_squash)
/home/mobfiles   192.168.0.209/32(async,wdelay,hide,sec=sys,rw,secure,no_root_squash,no_all_squash)
/home/var_data/public_html
       192.168.0.209/32(async,wdelay,hide,sec=sys,rw,secure,no_root_squash,no_all_squash)
/var/public
       192.168.0.209/32(async,wdelay,hide,sec=sys,rw,secure,no_root_squash,no_all_squash)
/neon/data
       192.168.0.209/32(async,wdelay,hide,sec=sys,rw,secure,no_root_squash,no_all_squash)
/scripts     192.168.0.209/32(async,wdelay,hide,sec=sys,rw,secure,no_root_squash,no_all_squash)
/backups/data-limited
       192.168.0.200/32(async,wdelay,hide,sec=sys,ro,secure,no_root_squash,no_all_squash)
/disk/filetransfer
       192.168.0.200/23(async,wdelay,hide,sec=sys,ro,secure,no_root_squash,no_all_squash)
/public_shared/data
       192.168.0.200/23(async,wdelay,hide,sec=sys,ro,secure,no_root_squash,no_all_squash)

Of course there is much more to be said on that you can for example, check /var/log/messages /var/log/syslog and other logs that can give you hints about issues, as well as manually try to mount / unmount a NFS stuck share to know more on what is going on, but for a starter that should be enough.

command can help severely in troubleshooting the NFS configuration.

Sum it up what learned ?

We learned how to set up basic NFS server and mounted its shared directory on a client machine.
This is a great solution for centralized file sharing and collaboration on Linux systems (even though many companies are trying to not use it due to its lack of connection encryption for historical reasons NFS has been widely used over the years and has helped dramatically for the Internet as we know it to become the World Wide Web of today. Thus for a well secured network and perhaps not a critical files infrastructure, still NFS is a key player in file sharing among heterogenous networks for multitudes of Gigabytes or Terra Pentabytes of data you would like to share amoung your Personal Computers / Servers / Phones / Tablets and generally all kind of digital computer equipment devices.

Tags: access, alex, alias, ALL, ALLOW, allows, and, bash, clients, Create Shared Directory, file sharing, installed, list, machines, NFS, ports, Start, sudo, system boot, systemctl
Posted in Everyday Life, Linux, Linux and FreeBSD Desktop, Networking, NFS, System Administration | No Comments »

How to install and configure AIDE ( Advanced Intrusion Detection Environment ) on Debian GNU / Linux 11 to monitor files for changes

Thursday, March 9th, 2023

How to install and configure AIDE ( Advanced Intrusion Detection Environment ) on Debian GNU / Linux 11 to monitor files for changes

Having a intrusion detection system is essential to keeping a server security to good level and being compliant with PCI (Payment Card Industry) DSS Standards. It is a great thing for the sake to protect oneself from hackers assaults.

There is plenty of Intrusion Detection systems available all around since many years, in the past one of main ones for Linux as older system administrators should remember was Tripwire – integrity tool for monitoring and alerting on specific file change(s) on a range of systems.

Tripwire is still used today but many today prefer to use AIDE that is a free software replacement for Tripwire under GPL (General Public License), that is starting to become like a "standard" for many Unix-like systems as an inexpensive baseline control and rootkit detection system.

In this article I'll explain shortly how to Install / Configure and Use AIDE to monitor, changes with files on the system.

But before proceeding it is worthy to mention on some of the alternatives companies and businesses choose to as an IDS (Intrusion Detection Systems), that is useful to give a brief idea of the sysadmins that has to deal with Security, on what is some of the main Intrusion Detection Systems adopted on UNIX OSes today:

Samhain
An integrity checker and host intrusion detection system that can be used on single hosts as well as large, UNIX-based networks. It supports central monitoring as well as powerful (and new) stealth features to run undetected in memory, using steganography. Samhain is an open-source multiplatform application for POSIX systems (Unix, Linux, Cygwin/Windows).
OSSEC
OSSEC uses a centralized, cross-platform architecture allowing multiple systems to be monitored and managed.
Snort
IDS which has the capabilities to prevent attacks. By taking a particular action based on traffic patterns, it can become an intrusion prevention system (IPS). – written in Pure C.
Zeek (Bro)
Zeek helps to perform security monitoring by looking into the network's activity. It can find suspicious data streams. Based on the data, it alert, react, and integrate with other tools – written in C++.
Maltrail (Maltrail monitors for traffic on the network that might indicate system compromise or other bad behavior. It is great for intrusion detection and monitoring. – written in Python).

1. Install aide deb package

# apt -y install aide
…

root@haproxy2:~# aide -v
Aide 0.17.3
Compiled with the following options:
WITH_MMAP
WITH_PCRE
WITH_POSIX_ACL
WITH_SELINUX
WITH_XATTR
WITH_CAPABILITIES
WITH_E2FSATTRS
WITH_ZLIB
WITH_MHASH
WITH_AUDIT
Default config values:
config file: <none>
database_in: <none>
database_out: <none>
Available hashsum groups:
md5: yes
sha1: yes
sha256: yes
sha512: yes
rmd160: yes
tiger: yes
crc32: yes
crc32b: yes
haval: yes
whirlpool: yes
gost: yes
stribog256: no
stribog512: no
Default compound groups:
R: l+p+u+g+s+c+m+i+n+md5+acl+selinux+xattrs+ftype+e2fsattrs+caps
L: l+p+u+g+i+n+acl+selinux+xattrs+ftype+e2fsattrs+caps
>: l+p+u+g+i+n+acl+S+selinux+xattrs+ftype+e2fsattrs+caps
H: md5+sha1+rmd160+tiger+crc32+haval+gost+crc32b+sha256+sha512+whirlpool
X: acl+selinux+xattrs+e2fsattrs+caps

2. Prepare AIDE configuration and geenrate (initialize) database

Either you can use the default AIDE configuration which already has a preset rules for various files and directories to be monitored,
or you might add up additional ones.

For details on configuration of aide.conf accepted options "man aide.conf"

The rules and other configurations resides lays under /etc/aide/ directory

The AIDE database is located under /var/lib/aide

root@server:~# ls -al /var/lib/aide/
общо 33008
drwxr-xr-x 2 root root 4096 9 мар 12:38 ./
drwxr-xr-x 27 root root 4096 9 мар 12:01 ../
-rw——- 1 root root 16895467 9 мар 16:03 aide.db
-rw——- 1 root root 16895467 9 мар 18:49 aide.db.new

Also, details about major setting rules config regarding how AIDE will run via cronjob as with most debian services are into /etc/default/aide

Default aide.conf config is in /etc/aide/aide.conf if you need custom stuff to do with it simply edit it.

Here is an Example:
Lets say you want to omit some directory to not be monitored by aide, which would otherwise do, i.e.
omit /var/log/* from monitoring

# At the end of file /etc/aide/aide.conf

add:

!/var/log
!/home/
!/var/lib
!/proc

Initialize the aide database first time

Run aideinit command, aideinit will create a new baseline database – /var/lib/aide/aide.db.new (a baseline)
Note that, /var/lib/aide/aide.db is the old database that aide uses to check against for any changes of files / directories on the configured monitored filesystem objects.

root@server:~# aideinit
Running aide –init…
debug1: client_input_channel_req: channel 0 rtype keepalive@openssh.com reply 1
debug1: client_input_channel_req: channel 0 rtype keepalive@openssh.com reply 1
debug1: client_input_channel_req: channel 0 rtype keepalive@openssh.com reply 1
debug1: client_input_channel_req: channel 0 rtype keepalive@openssh.com reply 1
debug1: client_input_channel_req: channel 0 rtype keepalive@openssh.com reply 1
Start timestamp: 2023-03-09 12:06:16 +0200 (AIDE 0.17.3)
AIDE initialized database at /var/lib/aide/aide.db.new
Number of entries: 66971
—————————————————
The attributes of the (uncompressed) database(s):
—————————————————
/var/lib/aide/aide.db.new
SHA256 : nVrYljiBFM/KaKCTjbaJtR2w6N8vc8qN
DPObbo2UMVo=
SHA512 : S1ZNB0DCqb4UTmuqaalTgiQ3UAltTOzO
YNfEQJldp32q5ahplBo4/65uwgtGusMy
rJC8nvxvYmh+mq+16kfrKA==
RMD160 : xaUnfW1+/DJV/6FEm/nn1k1UKOU=
TIGER : nGYEbX281tsQ6T21VPx1Hr/FwBdwF4cK
CRC32 : fzf7cg==
HAVAL : yYQw/87KUmRiRLSu5JcEIvBUVfsW/G9H
tVvs6WqL/0I=
WHIRLPOOL : 6b5y42axPjpUxWFipUs1PtbgP2q0KJWK
FwFvAGxHXjZeCBPEYZCNkj8mt8MkXBTJ
g83ZELK9GQBPLea7UF3tng==
GOST : sHAzx7hkr5H3q8TCSGCKjndEiZgcvCEL
E45qcRb25tM=

End timestamp: 2023-03-09 12:38:30 +0200 (run time: 32m 14s)

Be patient now, go grab a coffee / tea or snack as the command might take up to few minutes for the aide to walk through the whole monitored filesystems and built its database.

root@server:~# echo cp /var/lib/aide/aide.db{.new,}
cp /var/lib/aide/aide.db.new /var/lib/aide/aide.db

root@server:~# cp /var/lib/aide/aide.db{.new,}
root@server:~# aide –check –config /etc/aide/aide.conf
Start timestamp: 2023-03-09 13:01:32 +0200 (AIDE 0.17.3)
AIDE found differences between database and filesystem!!
Summary:
Total number of entries: 66972
Added entries: 1
Removed entries: 0
Changed entries: 7
—————————————————
Added entries:
—————————————————
f+++++++++++++++++: /var/lib/aide/aide.db
—————————————————
Changed entries:
—————————————————
d =…. mc.. .. . : /etc/aide
d =…. mc.. .. . : /root
f <…. mci.H.. . : /root/.viminfo
f =…. mc..H.. . : /var/lib/fail2ban/fail2ban.sqlite3
d =…. mc.. .. . : /var/lib/vnstat
f =…. mc..H.. . : /var/lib/vnstat/vnstat.db
f >b… mc..H.. . : /var/log/sysstat/sa09
—————————————————
Detailed information about changes:
—————————————————
Directory: /etc/aide
Mtime : 2023-03-09 12:04:03 +0200 | 2023-03-09 12:51:11 +0200
Ctime : 2023-03-09 12:04:03 +0200 | 2023-03-09 12:51:11 +0200
Directory: /root
Mtime : 2023-03-09 12:06:13 +0200 | 2023-03-09 12:51:11 +0200
Ctime : 2023-03-09 12:06:13 +0200 | 2023-03-09 12:51:11 +0200
File: /root/.viminfo
Size : 18688 | 17764
Mtime : 2023-03-09 12:06:13 +0200 | 2023-03-09 12:51:11 +0200
Ctime : 2023-03-09 12:06:13 +0200 | 2023-03-09 12:51:11 +0200
Inode : 133828 | 133827
SHA256 : aV54gi33aA/z/FuBj2ZioU2cTa9H16TT | dnFdLVQ/kx3UlTah09IgEMrJ/aYgczHe
TzkLSxBDSB4= | DdxDAmPOSAM=
…

3. Test aide detects file changes

Create a new file and append some text and rerun the aide check

root@server:~# touch /root/test.txt
root@server:~# echo aaa > /root/test.txt
root@server:~# aide –check –config /etc/aide/aide.conf

Start timestamp: 2023-03-09 13:07:21 +0200 (AIDE 0.17.3)
AIDE found differences between database and filesystem!!
Summary:
Total number of entries: 66973
Added entries: 2
Removed entries: 0
Changed entries: 7
—————————————————
Added entries:
—————————————————
f+++++++++++++++++: /root/test.txt
f+++++++++++++++++: /var/lib/aide/aide.db
—————————————————
Changed entries:
—————————————————
d =…. mc.. .. . : /etc/aide
d =…. mc.. .. . : /root
f <…. mci.H.. . : /root/.viminfo
f =…. mc..H.. . : /var/lib/fail2ban/fail2ban.sqlite3
d =…. mc.. .. . : /var/lib/vnstat
f =…. mc..H.. . : /var/lib/vnstat/vnstat.db
f >b… mc..H.. . : /var/log/sysstat/sa09
….

The same command can be shortened for the lazy typist:

root@server:~# aide -c /etc/aide/aide.conf -C

The command will basically try to check the deviation between the AIDE database and the filesystem.

4. Limiting AIDES Integrity Checks to Specific Files / Directories

In order to limit the integrity checks to a specific entries for example /etc, pass the –limit REGEX option to AIDE check command where REGEX is the entry to check.

For example, check and update the database entries matching /etc, you would run aide command as shown below;

root@server:~# aide -c /etc/aide/aide.conf –limit /etc –check

AIDE found differences between database and filesystem!!
Limit: /etc
Summary:
Total number of entries: 66791
Added entries: 0
Removed entries: 0
Changed entries: 2
—————————————————
Changed entries:
—————————————————
d =…. mc.. .. . : /etc/aide
d =…. mc.. .. . : /etc/default
—————————————————
Detailed information about changes:
—————————————————
Directory: /etc/aide
Mtime : 2023-03-09 15:59:53 +0200 | 2023-03-09 16:43:03 +0200
Ctime : 2023-03-09 15:59:53 +0200 | 2023-03-09 16:43:03 +0200
Directory: /etc/default
Mtime : 2023-03-09 12:06:13 +0200 | 2023-03-09 18:42:12 +0200
Ctime : 2023-03-09 12:06:13 +0200 | 2023-03-09 18:42:12 +0200

—————————————————
The attributes of the (uncompressed) database(s):
—————————————————
/var/lib/aide/aide.db
SHA256 : sjCxyIkr0nC/gTkNmn7DNqAQWttreDF6
vSUV4jBoFY4=
SHA512 : vNMpb54qxrbOk6S1Z+m9r0UwGvRarkWY
0m50TfMvGElfZWR1I3SSaeTdORAZ4rQe
17Oapo5+Sc0E2E+STO93tA==
RMD160 : anhm5E6UlKmPYYJ4WYnWXk/LT3A=
TIGER : 5e1wycoF35/ABrRf7FNypZ45169VTuV4
CRC32 : EAJlFg==
HAVAL : R5imONWRYgNGEfhBTc096K+ABnMFkMmh
Hsqe9xt20NU=
WHIRLPOOL : c6zySLliXNgnOA2DkHUdLTCG2d/T18gE
4rdAuKaC+s7gqAGyA4p2bnDHhdd0v06I
xEGY7YXCOXiwx8BM8xHAvQ==
GOST : F5zO2Ovtvf+f7Lw0Ef++ign1znZAQMHM
AApQOiB9CqA=

End timestamp: 2023-03-09 20:02:18 +0200 (run time: 1m 32s)

5. Add the modified /root/test.txt to AIDE list of known modified files database

root@server:~# aide –update –config /etc/aide/aide.
ERROR: cannot open config file '/etc/aide/aide.': No such file or directory

root@server:~# aide –update –config /etc/aide/aide.conf

Start timestamp: 2023-03-09 18:45:17 +0200 (AIDE 0.17.3)
AIDE found differences between database and filesystem!!
New AIDE database written to /var/lib/aide/aide.db.new
Summary:
Total number of entries: 66791
Added entries: 0
Removed entries: 0
Changed entries: 8
—————————————————
Changed entries:
—————————————————
d =…. mc.. .. . : /etc/aide
d =…. mc.. .. . : /etc/default
d =…. mc.. .. . : /root
f >…. mci.H.. . : /root/.viminfo
f >…. mci.H.. . : /root/test.txt
f =…. mc..H.. . : /var/lib/fail2ban/fail2ban.sqlite3
d =…. mc.. .. . : /var/lib/vnstat
f =…. mc..H.. . : /var/lib/vnstat/vnstat.db
—————————————————
Detailed information about changes:
—————————————————
Directory: /etc/aide
Mtime : 2023-03-09 15:59:53 +0200 | 2023-03-09 16:43:03 +0200
Ctime : 2023-03-09 15:59:53 +0200 | 2023-03-09 16:43:03 +0200
Directory: /etc/default
Mtime : 2023-03-09 12:06:13 +0200 | 2023-03-09 18:42:12 +0200
Ctime : 2023-03-09 12:06:13 +0200 | 2023-03-09 18:42:12 +0200
Directory: /root
Mtime : 2023-03-09 15:59:53 +0200 | 2023-03-09 18:44:34 +0200
Ctime : 2023-03-09 15:59:53 +0200 | 2023-03-09 18:44:34 +0200
File: /root/.viminfo
Size : 16706 | 16933
Mtime : 2023-03-09 15:59:53 +0200 | 2023-03-09 18:44:34 +0200
Ctime : 2023-03-09 15:59:53 +0200 | 2023-03-09 18:44:34 +0200
Inode : 136749 | 133828
SHA256 : KMHGoMVJo10BtafVrWIOLt3Ht9gK8bc+ | rrp8S3VftzZzvjBP1JC+PBpODv9wPKGw
9uHh/z7iJWA= | TA+hyhTiY+U=
SHA512 : ieDHy7ObSTfYm5d8DtYcHKxHya13CS65 | PDAJjyZ39uU3kKFo2lHBduTqxMDq4i01
ObMYIRAre6IgvLslEs0ZodQFyrczMyRt | 1Kvm/h6xzFhHtFgjidtcemG8wDcjtfNF
+d6SrW0gn3skKn2B7G09eQ== | Z7LO230fgGeO7UepqtxZjQ==
RMD160 : nUgg/G4zsVGKzVmmrqltuYUDvtg= | jj61KAFONK92mj+u66RDJmxFhmI=
TIGER : 3vPSOrla5k+k2br1E2ES4eNiSZ2novFX | mn4kNCzd8SQr2ID2VSe4f4l0ta7pO/xo
CRC32 : NDnMgw== | AyzVUQ==
HAVAL : Q9/KozxRiPbLEkaIfnBUZdEWftaF52Mw | 6jADKV6jg7ZVr/A/oMhR4NXc8TO1AOGW
7tiR7DXhl0o= | NrYe+j6UcO0=
WHIRLPOOL : vB/ZMCul4hN0aYd39gBu+HmZT/peRUI8 | mg6c1lYYVNZcy4mVzGojwraim8e3X2/R
KDkaslNb8+YleoFWx0mbhAbkGurc0+jh | urVvEmbsgTuUCJOuf9+OrEACiF0fbe/x
YPBviZIKcxUbTc2nGthTWw== | t+BXnSQWk08OL9EI6gMGqA==
GOST : owVGTgU9BH3b0If569wQygw3FAbZIZde | ffx29GV2jaCB7XzuNjdiRzziIiZYnbi3
eAfQfzlRPGY= | Ar7jyNMUutk=
File: /root/test.txt
Size : 4 | 8
Mtime : 2023-03-09 13:07:12 +0200 | 2023-03-09 18:44:34 +0200
Ctime : 2023-03-09 13:07:12 +0200 | 2023-03-09 18:44:34 +0200
Inode : 133828 | 136751
SHA256 : F+aC8GC1+OR+oExcSFWQiwpa1hICImD+ | jUIZMGfiMdAlWFHu8mmmlml4qAGNQNL5
UOEeywzAq3Y= | 6NhzJ1sYFZE=
SHA512 : d+UmFKFBzvGadt5hk+nIRbjP//7PSXNl | ixn20lcEMDEtsJo3hO90Ea/wHWLCHcrz
Pl16XRIUUPq2FCiQ4PeUcVciukJX7ijL | seBWunbBysY0z3BWcfgnN2vH05WfRfvA
D045ZvGOEcnmL6a6vwp0jw== | QiNtQS1tStuEdB3Voq54zQ==
RMD160 : I6waxKN3rMx4WTz4VCUQXoNoxUg= | urTh1j1t3UHchnJGnBG4lUZnjI4=
TIGER : cwUYgfKHcJnWXcA0pr/OKuxuoxh+b9lA | prstKqCfMXL39aVGFPA0kX4Q9x7a+hUn
CRC32 : UD78Dw== | zoYiEA==
HAVAL : bdbKR9LvPgsYClViKiHx48fFixfIL/jA | ZdpdeMhw4MvKBgWsM4EeyUgerO86Rt82
F3tjdc2Gm8Y= | W94fJFRWbrM=
WHIRLPOOL : OLP0Y4oKcqW2yEvme8z419N1KE4TB9GJ | Xk8Ujo3IU2SzSqbJFegq7p1ockmrnxJF
biHn/9XgrBz4fQiDJ8eHpx+0exA9hXmY | R3Rfstd1jWSwLFNTEwfbRRw+TARtRK50
EbbakMJJdzLt1ipKWiV9gg== | iWJeHLsD5dZ+CzV0tf4sUg==
GOST : ystISzoeH/ZznYrrXmxe4rwmybWMpGuE | GhMWNxg7Is0svJ+5LP+DVWbgt+CDQO+3
0PzRnVEqnR8= | 08dwBuVAwB8=
File: /var/lib/fail2ban/fail2ban.sqlite3
Mtime : 2023-03-09 15:55:01 +0200 | 2023-03-09 18:45:01 +0200
Ctime : 2023-03-09 15:55:01 +0200 | 2023-03-09 18:45:01 +0200
SHA256 : lLilXNleqSgHIP1y4o7c+oG5XyUPGzgi | NCJJ2H6xgCw/NYys1LMA7hOWwoOoxI8Y
RHYH+zvlAL4= | 4SJygfqEioE=
SHA512 : iQj2pNT4NES4fBcujzdlEEGZhDnkhKgc | ClQZ5HMOSayUNb//++eZc813fiMJcXnj
QDlGFSAn6vi+RXesFCjCABT7/00eEm5/ | vTGs/2tANojoe6cqpsT/LaJ3QZXpmrfh
ILcaqlQtBSLJgHjMQehzdg== | syVak1I4n9yg8cDKEkZUvw==
RMD160 : Xg4YU8YI935L+DLvkRsDanS4DGo= | SYrQ27n+/1fvIZ7v+Sar/wQHulI=
TIGER : 2WhhPq9kuyeNJkOicDTDeOeJB8HR8zZe | o1LDZtRclri2KfZBe5J3D4YhM05UaP4E
CRC32 : NQmi4A== | tzIsqg==
HAVAL : t1ET+84+8WgfwqlLy4R1Qk9qGZQRUbJI | MwVnjtM3dad/RuN2BfgsySX2DpfYq4qi
z2J0ROGduXc= | H1pq6RYsA6o=
WHIRLPOOL : xKSn71gFIVhk5rWJIBaYQASl0V+pGn+3 | m5LEXfhBbhWFg/d8CFJhklOurmRSkDSG
N85R0tiCKsTZ2+LRkxDrzcVQdss2k8+z | LC/vICnbEWzLwrCuMwBi1/e5wDNIY8gK
oqExhoXtPsMaREjpCugd3Q== | mvGn40x+G4cCYNZ6lGT9Zg==
GOST : WptpUlfooIlUjzDHU8XGuOU2waRud5SR | i6K4COXU0nyZ1mL3ZBuGUPz/ZXTj8KKQ
E/tnoBqk+q0= | L6VNyS8/X2Y=
Directory: /var/lib/vnstat
Mtime : 2023-03-09 16:00:00 +0200 | 2023-03-09 18:45:01 +0200
Ctime : 2023-03-09 16:00:00 +0200 | 2023-03-09 18:45:01 +0200
File: /var/lib/vnstat/vnstat.db
Mtime : 2023-03-09 16:00:00 +0200 | 2023-03-09 18:45:00 +0200
Ctime : 2023-03-09 16:00:00 +0200 | 2023-03-09 18:45:00 +0200
SHA256 : X/lnJuuSo4jX4HRzxMBodnKHAjQFvugi | oqtY3HTNds/qDNFCRAEsfN5SuO0U5LRg
2sh2c0u69x8= | otc5z1y+eGY=
SHA512 : U/g8O6G8cuhsqCUCbrElxgiy+naJKPkI | y+sw4LX8mlDWkRJMX38TsYSo1DQzxPOS
hG7vdH9rBINjakL87UWajT0s6WSy0pvt | 068otnzw2FSSlM5X5j5EtyJiY6Hd5P+A
ALaTcDFKHBAmmFrl8df2nQ== | jFiWStMbx+dQidXYZ4XFAw==
RMD160 : F6YEjIIQu2J3ru7IaTvSemA9e34= | bmVSaRKN2qU7qpEWkzfXFoH4ZK4=
TIGER : UEwLoeR6Qlf2oOI58pUCEDaWk0pHDkcY | 0Qb4nUqe3cKh/g5CQUnOXGfjZwJHjeWa
CRC32 : Bv3/6A== | jvW6mg==
HAVAL : VD7tjHb8o8KTUo5xUH7eJEmTWgB9zjft | rumfiWJvy/sTK/09uj7XlmV3f7vj6KBM
kOkzKxFWqqU= | qeOuKvu0Zjc=
WHIRLPOOL : wR0qt8u4N8aQn8VQ+bmfrxB7CyCWVwHi | FVWDRE3uY6qHxLlJQLU9i9QggLW+neMj
ADHpMTUxBEKOpOBlHTWXIk13qYZiD+o/ | Wt+Dj9Rz92BG9EomgLUgUkxfiVFO8cMq
XtzTB4rMbxS4Z5PAdC/07A== | WaR/KKq3Z7R8f/50tc9GMQ==
GOST : l3ibqMkHMSPpQ+9ok51/xBthET9+JQMd | qn0GyyCg67KRGP13At52tnviZfZDgyAm
OZtiFGYXmgU= | c82NXSzeyV0=

—————————————————
The attributes of the (uncompressed) database(s):
—————————————————
/var/lib/aide/aide.db
SHA256 : sjCxyIkr0nC/gTkNmn7DNqAQWttreDF6
vSUV4jBoFY4=
SHA512 : vNMpb54qxrbOk6S1Z+m9r0UwGvRarkWY
0m50TfMvGElfZWR1I3SSaeTdORAZ4rQe
17Oapo5+Sc0E2E+STO93tA==
RMD160 : anhm5E6UlKmPYYJ4WYnWXk/LT3A=
TIGER : 5e1wycoF35/ABrRf7FNypZ45169VTuV4
CRC32 : EAJlFg==
HAVAL : R5imONWRYgNGEfhBTc096K+ABnMFkMmh
Hsqe9xt20NU=
WHIRLPOOL : c6zySLliXNgnOA2DkHUdLTCG2d/T18gE
4rdAuKaC+s7gqAGyA4p2bnDHhdd0v06I
xEGY7YXCOXiwx8BM8xHAvQ==
GOST : F5zO2Ovtvf+f7Lw0Ef++ign1znZAQMHM
AApQOiB9CqA=
/var/lib/aide/aide.db.new
SHA256 : QRwubXnz8md/08n28Ek6DOsSQKGkLvuc
gSZRsw6gRw8=
SHA512 : 238RmI1PHhd9pXhzcHqM4+VjNzR0es+3
6eiGNrXHAdDTz7GlAQQ4WfKeQJH9LdyT
1r5ho/oXRgzfa2BfhKvTHg==
RMD160 : GJWuX/nIPY05gz62YXxk4tWiH5I=
TIGER : l0aOjXlM4/HjyN9bhgBOvvCYeqoQyjpw
CRC32 : KFz6GA==
HAVAL : a//4jwVxF22URf2BRNA612WOOvOrScy7
OmI44KrNbBM=
WHIRLPOOL : MBf+NeXElUvscJ2khIuAp+NDu1dm4h1f
5tBQ0XrQ6dQPNA2HZfOShCBOPzEl/zrl
+Px3QFV4FqD0jggr5sHK2g==
GOST : EQnPh6jQLVUqaAK9B4/U4V89tanTI55N
K7XqZR9eMG4=

End timestamp: 2023-03-09 18:49:51 +0200 (run time: 4m 34s)

6. Substitute old aide database with the new that includes the modified files

As you see AIDE detected the changes in /root/test.txt

To apply the changes be known by AIDE for next time (e.g. this file was authorized and supposed to be written there) simply move the new generated database
to current aide database.

# copy generated DB to master DB
root@dlp:~# cp -p /var/lib/aide/aide.db.new /var/lib/aide/aide.db

7. Check once again to make sure recently modified files are no longer seen as changed by AIDE

Recheck again the database to make sure the files you wanted to omit are no longer mentioned as changed

root@server:~# aide –check –config /etc/aide/aide.conf
Start timestamp: 2023-03-09 16:23:05 +0200 (AIDE 0.17.3)
AIDE found differences between database and filesystem!!
Summary:
Total number of entries: 66791
Added entries: 0
Removed entries: 0
Changed entries: 3
—————————————————
Changed entries:
—————————————————
f =…. mc..H.. . : /var/lib/fail2ban/fail2ban.sqlite3
d =…. mc.. .. . : /var/lib/vnstat
f =…. mc..H.. . : /var/lib/vnstat/vnstat.db
—————————————————
Detailed information about changes:
—————————————————
File: /var/lib/fail2ban/fail2ban.sqlite3
Mtime : 2023-03-09 15:55:01 +0200 | 2023-03-09 16:25:02 +0200
Ctime : 2023-03-09 15:55:01 +0200 | 2023-03-09 16:25:02 +0200
SHA256 : lLilXNleqSgHIP1y4o7c+oG5XyUPGzgi | MnWXC2rBMf7DNJ91kXtHXpM2c2xxF60X
RHYH+zvlAL4= | DfLUQLHiSiY=
SHA512 : iQj2pNT4NES4fBcujzdlEEGZhDnkhKgc | gxHVBxhGTKi0TjRE8/sn6/gtWsRw7Mfy
QDlGFSAn6vi+RXesFCjCABT7/00eEm5/ | /wCfPlDK0dkRZEbr8IE2BNUhBgwwocCq
ILcaqlQtBSLJgHjMQehzdg== | zuazTy4N4x6X8bwOzRmY0w==
RMD160 : Xg4YU8YI935L+DLvkRsDanS4DGo= | +ksl9kjDoSU9aL4tR7FFFOK3mqw=
TIGER : 2WhhPq9kuyeNJkOicDTDeOeJB8HR8zZe | 9cvXZNbU+cp5dA5PLiX6sGncXd1Ff5QO
CRC32 : NQmi4A== | y6Oixg==
HAVAL : t1ET+84+8WgfwqlLy4R1Qk9qGZQRUbJI | aPnCrHfmZAUm7QjROGEl6rd3776wO+Ep
z2J0ROGduXc= | s/TQn7tH1tY=
WHIRLPOOL : xKSn71gFIVhk5rWJIBaYQASl0V+pGn+3 | 9Hu6NBhz+puja7uandb21Nt6cEW6zEpm
N85R0tiCKsTZ2+LRkxDrzcVQdss2k8+z | bTsq4xYA09ekhDHMQJHj2WpKpzZbA+t0
oqExhoXtPsMaREjpCugd3Q== | cttMDX8J8M/UadqfL8KZkQ==
GOST : WptpUlfooIlUjzDHU8XGuOU2waRud5SR | WUQfAMtye4wADUepBvblvgO+vBodS0Ej
E/tnoBqk+q0= | cIbXy4vpPYc=
Directory: /var/lib/vnstat
Mtime : 2023-03-09 16:00:00 +0200 | 2023-03-09 16:25:01 +0200
Ctime : 2023-03-09 16:00:00 +0200 | 2023-03-09 16:25:01 +0200
File: /var/lib/vnstat/vnstat.db
Mtime : 2023-03-09 16:00:00 +0200 | 2023-03-09 16:25:01 +0200
Ctime : 2023-03-09 16:00:00 +0200 | 2023-03-09 16:25:01 +0200
SHA256 : X/lnJuuSo4jX4HRzxMBodnKHAjQFvugi | N1lzhV3+tkDBud3AVlmIpDkU1c3Rqhnt
2sh2c0u69x8= | YqE8naDicoM=
SHA512 : U/g8O6G8cuhsqCUCbrElxgiy+naJKPkI | +8B9HvHhOp1C/XdlOORjyd3J2RtTbRBF
hG7vdH9rBINjakL87UWajT0s6WSy0pvt | b0Moo2Gj+cIxaMCu5wOkgreMp6FloqJR
ALaTcDFKHBAmmFrl8df2nQ== | UH4cNES/bAWtonmbj4W7Vw==
RMD160 : F6YEjIIQu2J3ru7IaTvSemA9e34= | 8M6TIOHt0NWgR5Mo47DxU28cp+4=
TIGER : UEwLoeR6Qlf2oOI58pUCEDaWk0pHDkcY | Du9Ue0JA2URO2tiij31B/+663OaWKefR
CRC32 : Bv3/6A== | v0Ai4w==
HAVAL : VD7tjHb8o8KTUo5xUH7eJEmTWgB9zjft | XA+vRnMNdVGFrO+IZtEA0icunWqBGaCf
kOkzKxFWqqU= | leR27LN4ejc=
WHIRLPOOL : wR0qt8u4N8aQn8VQ+bmfrxB7CyCWVwHi | HG31dNEEcak2zZGR24W7FDJx8mh24MaJ
ADHpMTUxBEKOpOBlHTWXIk13qYZiD+o/ | BQNhqkuS6R/bmlhx+P+eQ/JimwPAPOaM
XtzTB4rMbxS4Z5PAdC/07A== | xWG7cMETIXdT9sUOUal8Sw==
GOST : l3ibqMkHMSPpQ+9ok51/xBthET9+JQMd | y6Ek/TyAMGV5egkfCu92Y4qqk1Xge8c0
OZtiFGYXmgU= | 3ONXRveOlr0=

—————————————————
The attributes of the (uncompressed) database(s):
—————————————————
/var/lib/aide/aide.db
SHA256 : sjCxyIkr0nC/gTkNmn7DNqAQWttreDF6
vSUV4jBoFY4=
SHA512 : vNMpb54qxrbOk6S1Z+m9r0UwGvRarkWY
0m50TfMvGElfZWR1I3SSaeTdORAZ4rQe
17Oapo5+Sc0E2E+STO93tA==
RMD160 : anhm5E6UlKmPYYJ4WYnWXk/LT3A=
TIGER : 5e1wycoF35/ABrRf7FNypZ45169VTuV4
CRC32 : EAJlFg==
HAVAL : R5imONWRYgNGEfhBTc096K+ABnMFkMmh
Hsqe9xt20NU=
WHIRLPOOL : c6zySLliXNgnOA2DkHUdLTCG2d/T18gE
4rdAuKaC+s7gqAGyA4p2bnDHhdd0v06I
xEGY7YXCOXiwx8BM8xHAvQ==
GOST : F5zO2Ovtvf+f7Lw0Ef++ign1znZAQMHM
AApQOiB9CqA=

End timestamp: 2023-03-09 16:27:33 +0200 (run time: 4m 28s)

As you can see there are no new added entries for /root/test.txt and some other changed records for vnstat service as well as fail2ban ones, so the Intrusion detection system works just as we expected it.

8. Configure Email AIDE changed files alerting Email recipient address

From here on aide package has set its own cron job which is automatically doing the check operation every day and any new file modifications will be captured and alerts sent to local root@localhost mailbox account, so you can check it out later with mail command.

If you want to sent the Email alert for any files modifications occured to another email, assuming that you have a locally running SMTP server with a mail relay to send to external mails, you can do it via /etc/default/aide via:

MAILTO=root

For example change it to a FQDN email address

MAILTO=external_mail@your-mail.com

9.Force AIDE to run AIDE at specitic more frequent time intervals

You can as well install a cron job to execute AIDE at specific time intervals, as of your choice

Lets say you want to run a custom prepared set of files to monitor in /etc/aide/aide_custom_config.conf configure a new cronjob like below:

root@server:~# crontab -u root -e
*/5 * * * * aide -c /etc/aide/aide_custom_config.conf -u && cp /var/lib/custom-aide/aide.db{.new,}

This will execute AIDE system check every 5 minutse and email the report to ealier configured email username@whatever-your-smtp.com via /etc/default/aide

10. Check the output of AIDE for changes – useful for getting a files changes from aide from scripts

Check the command exit status.

root@server:~# echo $?

According to AIDE man pages, the AIDE’s exit status is normally 0 if no errors occurred. Except when the –check, –compare or –update command was requested, in which case the exit status is defined as:

1 * (new files detected?) +
2 * (removed files detected?) +
4 * (changed files detected?)

Since those three cases can occur together, the respective error codes are added. For example, if there are new files and removed files detected, the exit status will be 1 + 2 = 3.

Additionally, the following exit codes are defined for generic error conditions in aide help manual:

14 Error writing error
15 Invalid argument error
16 Unimplemented function error
17 Invalid configureline error
18 IO error
19 Version mismatch error

PLEASE CONSIDER

That AIDE checks might be resource intensive
and could cause a peak in CPU use and have a negative effect on lets very loaded application server machines,
thus causing a performance issuea during integrity checks !
If you are scanning file system wide and you do it frequent, be sure to provide “enough” resources or schedule the scan at a times that the Linux host will be less used !
Whenever you made any AIDE configuration changes, remember to initialize the database to create a baseline !

Tags: address, AIDE, checker, conf, deb package, How to, Initialize, Install, ips, lib, mail command, mc, monitor, root server, scripts, Start, timestamp, various, vnstat
Posted in Computer Security, Debian, Linux | 1 Comment »

Howto convert KVM QCOW2 format Virtual Machine to Vmdk to migrate to VMware ESXi

Thursday, November 17th, 2022

Why you would want to convert qcow2 to vmdk?

When managing the heterogeneous virtual environment or changing the virtualization solutions that become so common nowadays, you might need to migrate qcow2 from a Linux based KVM virtualization solution to VMWare's proprietary vmdk – the file format in which a VMWare does keep stored it's VMs, especially if you have a small business or work in a small start-up company where you cannot afford to buy something professional as VMware vCenter Converter Standalone or Microsoft virtual machine converter (MVMC)- usually used to to migrate VMware hosts to Hyper-V hosts, but also capable to migrate .qcow2 to .vmdk. The reason is that your old datacenter based on Linux OS custom KVM virtual machines might be moved to VMWare ESX to guarantee better and more systemized management (which though is very questionable, since most of my experiences with VMWare was that though the software was a great one, the people who manage it was not very much specialists in managing it).

Another common reason is that running a separate Linux virtual machine, costs you more than a well organized VMWare farm because you need more qualified Linux specialists to manage the KVMs thus KVM to VMWare management as in most big corporations nowadays’s main target is to cut the costs.
Even with successful migrations like that, though you might often expect a drop in the quality of the service when your VM ends in the VMWare farm.

Nomatter what’s the reason to migrate qcow2 to VMDK So lets proceed with how the .QCOW2 to .VMDK can be easily done.

1. Get information about the VM you would like to migrate to VMDK

In QEMU-KVM environment, the popular image format is qcow2, which outperforms the first generation of qcow format and raw format. You can find the files of virtual disks by checking the information of virtual machine by virsh command:

[root@hypervisor-machine ~]# virsh dominfo virtual-machine-name
INFO
ID: {e59ae416-9314-4e4b-af07-21c31d91b3fb}
EnvID: 1704649750
Name: CentOS7minimal
Description:
Type: VM
State: stopped
OS: centos7
Template: no
Uptime: 00:00:00 (since 2019-04-25 13:04:11)
Home: /vz/vmprivate/e39ae416-9314-4e4b-af05-21c31d91b3fb/
Owner: root@.
GuestTools: state=not_installed
GuestTools autoupdate: on
Autostart: off
Autostop: shutdown
Autocompact: off
Boot order: hdd0 cdrom0
EFI boot: off
Allow select boot device: off
External boot device:
On guest crash: restart
Remote display: mode=manual port=6903 address=0.0.0.0
Remote display state: stopped
Hardware:
cpu sockets=1 cpus=2 cores=2 VT-x accl=high mode=64 ioprio=4 iolimit='0'
memory 2048Mb
video 32Mb 3d acceleration=off vertical sync=yes
memory_guarantee auto
hdd0 (+) scsi:0 image='/vz/vmprivate/e59ae415-9314-4e4b-af05-21c31d91b3fb/harddisk.hdd' type='expanded' 5120Mb subtype=virtio-scsi
cdrom0 (+) scsi:1 image='/home/CentOS-7-x86_64-Minimal-1611.iso' state=disconnected subtype=virtio-scsi
usb (+)
net0 (+) dev='vme42bef5f3' network='Bridged' mac=001C42BEF5F3 card=virtio ips='10.50.50.27/255.255.255.192 ' gw='10.50.50.1'
SmartMount: (-)
Disabled Windows logo: on
Nested virtualization: off
Offline management: (-)
Hostname: kvmhost.fqdn.com

2. Convert the harddrive to VMDK

[root@hypervisor-machine e59ae415-9314-4e4b-af05-21c31d91b3fb]# ls -lsah
1.3G -rw-r—– 1 root root 1.3G Apr 25 14:43 harddisk.hdd

a. Converstion with qemu:

You can use qemu-img tool that is installable via cmds:

yum install quemu-img / apt install qemu-img / zipper install qemu-img (depending on the distribution RedHat / Debian / SuSE Linux)

-f: format of the source image

-O: format of the target image

[root@hypervisor-machine ~]# qemu-img convert -f qcow2 -O vmdk \-o adapter_type=lsilogic,subformat=streamOptimized,compat6 harddisk.hdd harddisklsilogic.vmdk

[root@ hypervisor-machine e59ae415-9314-4e4b-af05-21c31d91b3fb]# ls -lsah
1.3G -rw-r—– 1 root root 1.3G Apr 25 14:43 harddisk.hdd
536M -rw-r–r– 1 root root 536M Apr 26 14:52 harddisklsilogic.vmdk

3. Upload the new harddrive to the ESXi Hypervisor and adapt it to ESX

This vmdk might not be able to used on ESXi, but you can use it on VMware Workstation. To let it work on ESXi, you need to use vmkfstools to convert it again.

a. Adapt the filesystem to ESXi

[root@hypervisor-machine ~]# vmkfstools -i harddisklsilogic.vmdk -d thin harddisk.vmdk

4. Create a VM and add the converted harddrive to the machine.

Futher

Recreate the initramfs

But of course this won’t work directly as it often happens with Linux 🙂 !!.
We need to make adjustments to the virtual machine as well with few manual interventions:

1. Start the machine from the VMWare interface

2. Grub CentOS Linux rescue will appear from the prompt

3. Run command

dracut –regenerate-all –force

to Recreate the initramfs.

Note that You might also have to edit your network configuration since your network device usually get’s a different name.

Finally reboot the host:

[root@hypervisor-machine ~]# reboot

And voila you’re ready to play the VM inside the ESX after some testing, you might switch off the KVM Hypervisor hosted VM and reroute the network to point to the ESX Cluster.

Tags: command, Convert, External, MVMC, Offline, qemu, Remote, shutdown, Start, video
Posted in Linux, Migration, System Administration, Virtual Machines | No Comments »

Posts Tagged ‘Start’

The overall Cluster setup

Why check ?

1. Creating a simple check script (approach 1)

One thing to watch out for

2. Add the check to the Zabbix agent

3. Create the Zabbix item

4. Create the triggers

The resulting states

There is a problem with checking from the HAProxy node

5. A more reliable approach

6. Don't stop at the cluster state

7. Testing the setup

Final thoughts

1. The Allmighty, Reverse Search ( Ctrl + R )

2. Stop annoying re-typing: navigate the Line instantly

3. Precise Delete strings

4. Reuse arguments without rewriting

5. Fix Mistakes Instantly hack

6. Use history without running history cmd

7. Use Auto-Complete

8. Edit the previous command straight in editor

9. Clear and Reset Quickly

10. Background and Foreground Control

11. Memorize shortcuts / improve shell habits

Once those stick, layer in the others.

Final Thought

Advantages of FreeBSD Jails

a. Strong, simple isolation

b. High performance

c. Operational simplicity

d. Predictability and stability

Disadvantages of FreeBSD Jails

a. Limited portability

b. Smaller ecosystem

c. Less automation tooling

d. Harder to find people for it

Architecture Overview of sample FBSD Cluster

1. Base FreeBSD Installation (All 10 Servers)

2. Install Jail Management tool (iocage)

3. Create a Reusable Jail Template

4. Install LAMP Stack Inside the Jail

4.1. Install Apache

4.2. Install MariaDB

4.3. Install PHP pre-compiled ports

5. Test LAMP Stack works OK

6. Convert Template into Clones

7. Networking Between Jails

8. Load Balancing Layer

9. Database Strategy

a. Use Centralized DB

b. Use Per-node DB (simpler)

10. Automation Across 10 Servers

a. Tune up setup / Do Resource management

b. Use Centralized Logging

c. Do regular jail Backups

d. Tighten Security

e. Do a Further Scaling Strategy

Summary and Last Thoughts

1. AI as a Command-Line Tool

2. Minimum VM (It runs, but you’ll hate it)

a. Recommended VM (Actually usable)

b. “Feels Good” VM (CPU-only but not painful)

c. KVM / Proxmox (Best choice)

d. If running VM on VMware platform

e. VirtualBox (Not recommended)

f. Local AI With Ollama (Recommended)

3. Install Ollama

To check whether all is properly setup after installed llama3:

4. Turning AI Into a Bash Command

5. Using AI to Generate Bash Commands

6. AI for Explaining Commands and Logs

7. Smarter Bash History Search

8. Using AI With Shell Scripts

9. Where Ollama Stores Its Data

System service (most common)

User-only install

See What’s Taking Space

10. Remove Unused Models (Safe Way)

11. Full Manual Cleanup (Hard Reset)

1. The Allmighty, Reverse Search ( `Ctrl + R` )