data « ☩ Walking in Light with Christ

Posts Tagged ‘data’

How to Install and Use Kibana for Log Visualization

Wednesday, February 18th, 2026

I saw Kibana in my professional career and I find it a very interesting tool for sysadmins, so I thought it might be helpful to someone out there to write a small article on how to install and use to to visualize data inside some elasticsearch software.

Kibana is an open-source data visualization and exploration tool used to analyze large volumes of data, especially logs. It is part of the ELK Stack (Elasticsearch, Logstash, Kibana), and is commonly used for centralized log management, security monitoring, and observability.

Kibana is often used in the so-called ELK pipeline for log file collection, analysis and visualization:

Elasticsearch is for searching, analyzing, and storing your data
Logstash (and Beats) is for collecting and transforming data, from any source, in any format
Kibana is a portal for visualizing the data and to navigate within the elastic stack

In this article, you'll learn how to:

Install Kibana
Connect it to Elasticsearch
Visualize log data
Use its basic features

Prerequisites

Before installing Kibana, make sure you have the following:

A Linux server running (Ubuntu / Debian / CentOS / RHEL)
Elasticsearch installed and running
Root or sudo access

Install Kibana

I. On Debian/Ubuntu

Import the Elastic GPG key:

# wget -qO – https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add –

Add the repository:

# echo "deb https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-8.x.list

Update and install:

# apt update
# apt install kibana

II. On RHEL/CentOS Linux

Create repo file:

# tee /etc/yum.repos.d/elastic.repo <<EOF
[elastic-8.x]
name=Elastic repository for 8.x packages
baseurl=https://artifacts.elastic.co/packages/8.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md
EOF

Install Kibana:

# yum install kibana

2. Configure Kibana

The configuration file is located at:

/etc/kibana/kibana.yml

Edit the file:

# vim /etc/kibana/kibana.yml

Update or add the following:

# Server settings
server.port: 5601
server.host: "0.0.0.0"
# Elasticsearch connection
elasticsearch.hosts: [“http://localhost:9200”]
# Logging
logging.level: info
# Security (only if Elasticsearch security is enabled)
# elasticsearch.username: "kibana_system"
# elasticsearch.password: "your_password_here"

Optional: Set basic auth or SSL settings if needed.

3. Start and Enable Kibana

# systemctl enable kibana
# systemctl start kibana

Check status:

# systemctl status kibana
…

4. Access Kibana Web Interface

Open your browser and go to:

http://<your-server-ip>:5601

You’ll be welcomed with the Kibana dashboard.

5. Import and Visualize Logs

Option A: Use Filebeat to Send Logs

Install Filebeat on the server with logs and configure it to send data to Elasticsearch. Kibana will then be able to visualize it.

# apt install filebeat
# filebeat modules enable system
# filebeat setup
# systemctl start filebeat

Option B: Ingest Logs via Logstash or Elasticsearch API

If you already have data in Elasticsearch, Kibana will automatically detect indices.

6. Create Index Pattern

In Kibana, go to Stack Management -> Index Patterns
Click Create Index Pattern
Enter the name (e.g., filebeat-*)
Select the timestamp field (usually @timestamp)
Save

Now Kibana knows how to query and visualize your data.

7. Create Visualizations and Dashboards

Go to Visualize -> Create visualization
Choose a type (bar, pie, line, etc.)
Select an index pattern
Configure metrics and buckets

You can then save visualizations and add them to dashboards.

8. Secure Kibana

Configure TLS/SSL for Kibana / ElasticSearch (such as Logstash)
Use additional Elastic Security features like RBAC (Role Based Access Control, SSO (Single Sign On)
Secure Kibana with a reverse proxy (e.g., Nginx + Basic Auth or Apache / Haproxy infront)

Example Nginx config simple snippet:

location / {
proxy_pass http://localhost:5601;
auth_basic "Restricted";
auth_basic_user_file /etc/nginx/.htpasswd;
}

What is Kibana used for and what it can do for you?

Use Case	Description
Log Monitoring	Visualize system and application logs in real time
Security Analytics	Detect anomalies, failed logins, suspicious activity
DevOps Dashboards	Track uptime, error rates, and system performance
SIEM	Use Elastic Security for threat detection

Once Kibana is installed on a server, you typically use it to visualize and explore data stored in Elasticsearch. Here’s a practical guide with sample usage scenarios:

Access Kibana

After installation, Kibana usually runs on port 5601 by default.

http://<your-server-ip>:5601

Open this URL in a browser.
You should see the Kibana dashboard.

Connect to Elasticsearch

Kibana automatically connects to your Elasticsearch instance if installed locally.
You can verify the connection:

GET /_cluster/health

Go to Dev Tools → Console in Kibana.
Run the above query to check cluster status.

Visualize Data

Kibana allows multiple types of visualizations:

Bar/line chart: trends over time.
Pie chart: distribution of values.
Data table: top IP addresses or most visited URLs.
Maps: geolocation of IP addresses.

Create Dashboards

Combine multiple visualizations in a Dashboard.
Useful for monitoring logs, metrics, or application performance.
Example: Create a dashboard with:
- Requests per URL (bar chart)
- Requests over time (line chart)
- Top client IPs (data table)
- Errors by type (pie chart)

Search & Query Logs

Use Discover to search logs interactively.
Example KQL query:

status:500 AND url:"/login"

This finds all failed login requests.

Set Alerts (Optional)

Kibana’s Alerts and Actions can trigger notifications (email, Slack, etc.) when certain thresholds are crossed.
Example: alert if error responses exceed 100 in 5 minutes.

Once Kibana is installed on a server, you typically use it to visualize and explore data stored in Elasticsearch. Here’s a practical guide with sample usage scenarios:

Access Kibana

After installation, Kibana usually runs on port 5601 by default.

http://<your-server-ip>:5601

Open this URL in a browser.
You should see the Kibana dashboard.

Connect to Elasticsearch

Kibana automatically connects to your Elasticsearch instance if installed locally.
You can verify the connection:

GET /_cluster/health

Go to Dev Tools → Console in Kibana.
Run the above query to check cluster status.

Visualize Data

Kibana allows multiple types of visualizations:

Bar/line chart: trends over time.
Pie chart: distribution of values.
Data table: top IP addresses or most visited URLs.
Maps: geolocation of IP addresses.

Create Dashboards

Combine multiple visualizations in a Dashboard.
Useful for monitoring logs, metrics, or application performance.
Example: Create a dashboard with:
- Requests per URL (bar chart)
- Requests over time (line chart)
- Top client IPs (data table)
- Errors by type (pie chart)

Search & Query Logs

Use Discover to search logs interactively.
Example KQL query:

status:500 AND url:"/login"

This finds all failed login requests.

Set Alerts (Optional)

Kibana’s Alerts and Actions can trigger notifications (email, Slack, etc.) when certain thresholds are crossed.
Example: alert if error responses exceed 100 in 5 minutes.

Once Kibana is installed on a server, you typically use it to visualize and explore data stored in Elasticsearch. Here’s a practical guide with sample usage scenarios:

Access Kibana

After installation, Kibana usually runs on port 5601 by default.

http://your-server-ip:5601

Open this URL in a browser.
You should see the Kibana dashboard.

Connect to Elasticsearch

Kibana automatically connects to your Elasticsearch instance if installed locally.
You can verify the connection:

GET /_cluster/health

Go to Dev Tools → Console in Kibana.
Run the above query to check cluster status.

Visualize Data

Kibana allows multiple types of visualizations:

Bar/line chart: trends over time.
Pie chart: distribution of values.
Data table: top IP addresses or most visited URLs.
Maps: geolocation of IP addresses.

Create Dashboards

Combine multiple visualizations in a Dashboard.
Useful for monitoring logs, metrics, or application performance.
Example: Create a dashboard with:
- Requests per URL (bar chart)
- Requests over time (line chart)
- Top client IPs (data table)
- Errors by type (pie chart)

Search & Query Logs

Use Discover to search logs interactively.
Example KQL query:

status:500 AND url:"/login"

This finds all failed login requests.

Set Alerts (Optional)

Kibana’s Alerts and Actions can trigger notifications (email, Slack, etc.) when certain thresholds are crossed.
Example: alert if error responses exceed 100 in 5 minutes.

Sample Kibana dashboard

Kibana with connected servers to find out Geo Location

Summary closing words (what we did)

Step	Action
1	Install Kibana from Elastic repo
2	Configure to connect to Elasticsearch
3	Start and enable the service
4	Access it via http://<ip>:5601
5	Ingest log data
6	Define index pattern
7	Create dashboards and visualizations

The idea of this article was just to introduce you to the existence of Elasticsearch / kibana and filebeat and logstack and not to give you a fully fine tuned install guide. The usual way to deploy Kibana on multiple servers of course is using a dockerized container version of it. There is plenty to learned on how to use kibana to do a monitoring of your machines. But most simple use is to directly access the locally visible kibana on a server and check the status of processes on the host instead of logging via SSH. Kibana can do pretty much

Some further useful Reading Resources

Kibana Docs: https://www.elastic.co/guide/en/kibana/index.html
Filebeat Docs: https://www.elastic.co/guide/en/beats/filebeat/index.html
Logstash Docs: https://www.elastic.co/guide/en/logstash/index.html

Tags: Access Kibana Web Interface, alert, Common Kibana Use Cases, data, Detect, Edit, ELK, hosts, htpasswd, info, Ingest Logs, linux?, Open, repository, scenarios, server, SSO, sudo, systemctl, update, Useful, www
Posted in Linux, Monitoring, Various | No Comments »

How to Make Easy Backups on Linux Using a GUI tools Deja Dup, TimeShift, BackinTime, Grsync, Vorta

Monday, February 2nd, 2026

Backing up your data on Linux doesn’t have to involve complex terminal commands or custom scripts. While the command line is powerful, many users prefer a simple graphical interface (GUI) that just works.

Luckily, Linux offers several excellent GUI-based backup tools that are easy, reliable, and beginner-friendly.

In this article, we’ll look at why backups matter, and then walk through some of the best GUI backup tools for Linux, along with basic setup tips.

Why Backups Are Important (Even on Linux)

Linux systems are known for stability, but unfortunately, no system is immune to:

Hard drive failures
Accidental file deletion
System updates gone wrong
Malware or ransomware
Laptop theft or damage

A proper backup ensures you can restore your files or even your entire system in minutes instead of losing everything.

What Makes a Good GUI Backup Tool?

For most desktop users, a good backup tool should :

Be easy to use (no terminal required)
Supports automatic scheduled backups
Allow restoring individual files
Work with different types of external drives or network storage
Be relatively actively maintained

Let’s look at the few tools to create backups with lesser effort.

1. Déjà Dup – The Simplest Backup Tool

Best for: Beginners and home users
Available on: Ubuntu, Linux Mint, Fedora, and others

Déjà Dup is one of the most user-friendly backup tools on Linux. It comes preinstalled on Ubuntu and integrates perfectly with the GNOME desktop.

Key Features

Very simple interface
Automatic scheduled backups
Supports local drives, external USB disks, and network locations
Optional encryption for security

# apt info deja-dup
Package: deja-dup
Version: 44.0-2
Priority: optional
Section: utils
Maintainer: Debian GNOME Maintainers <pkg-gnome-maintainers@lists.alioth.debian.org>
Installed-Size: 4,851 kB
Depends: duplicity (>= 0.7.14), dconf-gsettings-backend | gsettings-backend, libadwaita-1-0 (>= 1.2), libc6 (>= 2.34), libglib2.0-0 (>= 2.70.0), libgpg-error0 (>= 1.14), libgtk-4-1 (>= 4.0.0), libjson-glib-1.0-0 (>= 1.5.2), libpackagekit-glib2-18 (>= 1.1.0), libpango-1.0-0 (>= 1.18.0), libsecret-1-0 (>= 0.18.6), libsoup-3.0-0 (>= 3.0.3)
Recommends: gvfs-backends, packagekit, policykit-1
Suggests: python3-pydrive2
Homepage: https://launchpad.net/deja-dup
Tag: admin::backup, implemented-in::c, interface::graphical, interface::x11,
role::program, scope::application, suite::gnome, uitoolkit::gtk,
x11::application
Download-Size: 693 kB
APT-Sources: http://ftp.debian.org/debian bookworm/main amd64 Packages
Description: Backup utility
Déjà Dup is a simple backup tool. It hides the complexity of backing up the
Right Way (encrypted, off-site, and regular) and uses duplicity as the
backend.
.
Features:
* Support for local, remote, or cloud backup locations such as Nextcloud
* Securely encrypts and compresses your data
* Incrementally backs up, letting you restore from any particular backup
* Schedules regular backups
* Integrates well into your GNOME desktop

How to Use Déjà Dup

Using it is generally simplistic, you select the data folders to be backupped and then the media where to backup it. The program supports also encryption with a password which is nice if you want to keep the backed-up data secret (especially if you want to store the backup on Google Cloud or Microsoft Azure)

Open “Backups” from your application menu

Choose folders to back up (e.g., Home folder)
Select a backup location (external drive recommended)
Enable automatic backups

Click on Back Up Now button

That’s it. Déjà Dup runs quietly in the background after setup.

Note ! that it is not a good idea to try to backup the whole Linux installation ! with deja-dup, as you will get a lot of issues with improper permissions errors and stuff and the OS backup won't get consistent, however for a basic backups of User Homes, Cictures and some Personal data situated within a single directory it is simple as it is easy to initially setup and run.

# apt install deja-dup
$ sudo deja-dup

2. Timeshift – System Snapshots Made Easy

Best for: System recovery
Available on: Most Linux distributions

Timeshift focuses on system backups, not personal files. It creates restore points similar to Windows System Restore.

Key Features

Snapshot-based backups
Perfect for rolling back failed updates
Supports RSYNC and BTRFS
Clean and simple GUI

When to Use Timeshift

Before major system updates
After fresh OS installation
To recover from broken packages or configs

# apt info timeshift
Package: timeshift
Version: 22.11.2-1+deb12u1
Priority: optional
Section: utils
Maintainer: Yanhao Mo <yanhaocs@gmail.com>
Installed-Size: 3,231 kB
Depends: cron-daemon | cron, pkexec, psmisc, rsync, libc6 (>= 2.34), libcairo2 (>= 1.2.4), libgdk-pixbuf-2.0-0 (>= 2.22.0), libgee-0.8-2 (>= 0.8.3), libglib2.0-0 (>= 2.39.4), libgtk-3-0 (>= 3.16.2), libjson-glib-1.0-0 (>= 1.5.2), libvte-2.91-0, libxapp1 (>= 1.0.4)
Breaks: util-linux (<< 2.37.2~)
Replaces: timeshift-btrfs
Homepage: https://github.com/linuxmint/timeshift
Tag: uitoolkit::gtk
Download-Size: 617 kB
APT-Manual-Installed: yes
APT-Sources: http://ftp.debian.org/debian bookworm/main amd64 Packages
Description: System restore utility
Timeshift is a system restore utility which takes snapshots
of the system at regular intervals. These snapshots can be restored
at a later date to undo system changes. Creates incremental snapshots
using rsync or BTRFS snapshots using BTRFS tools.

# apt install timeshift
$ sudo timeshift-gtk

3. Use Timeshift alongside a file backup tool like Déjà Dup as a backup solution for OS and data

a. Set up Timeshift (system snapshots)

What to include

Snapshot type:

RSYNC → works on any filesystem (recommended)
BTRFS → if your root is BTRFS

Include:

/ (root filesystem)

Exclude home directories (important!)

In Timeshift settings:

Keep /root excluded
Do NOT include /home/youruser

Timeshift is not meant to back up your personal files.

Schedule (typical)

Daily: 3–5 snapshots
Weekly: 2–3 snapshots
Monthly: optional

Store snapshots on:

A separate drive or partition if possible

b. Set up Deja Dup (personal backups)

Deja Dup is perfect for:

Home directory backups
Encryption
External drives, NAS, cloud (Google Drive, SFTP, etc.)

Folders to back up

Usually:

~/Documents
~/Pictures (or similar)
Optional: ~/.config (only if you know why)
~/Videos
~/Projects

In Deja Dup:

Folders to back up → select what you actually care about

Folders to ignore → add

~/.cache
~/.local/share/Trash
~/Downloads (optional)

Schedule

Daily or weekly backup is usually fine

Keep backups for “forever” or at least several months

c. Prevent overlap (this matters)

To avoid wasting space and time:

Tool	Should back up	Should NOT back up
Timeshift	/, system configs	/home
Deja Dup	/home/youruser	/, system files

Never:

Use Deja Dup to back up /
Use Timeshift to back up /home

That’s the #1 mistake you could do

d. Real-world recovery scenarios

Scenario 1: Bad update / system won’t boot

Boot from live USB
Restore with Timeshift
System is back exactly as before
Files untouched

Scenario 2: Deleted or corrupted files

Open Deja Dup
Restore specific files/folders
Done

Scenario 3: New machine / fresh install

Install OS
Restore system apps/settings manually or via Timeshift (if compatible)
Restore home data with Deja Dup

e. Optional pro tips (to avoid data loss)

Test restores once (seriously)
Label backup drives clearly
Keep Deja Dup backups offsite if possible
After major distro upgrades:
Make a Timeshift snapshot
Don’t restore old Timeshift snapshots across major versions unless you know it’s safe

4. Back In Time – More Control features tool to create GUI-Based backups on Linux

Best for: Advanced users who want flexibility

Available on: Most Linux distributions

Back In Time uses RSYNC but wraps it in a friendly GUI.

Key Features

Scheduled snapshots
Exclude files and folders easily
Restore files from any snapshot
Supports local and remote backups

# apt-cache search backintime

backintime-common – simple backup/snapshot system (common files)
# apt info backintime-qt
Package: backintime-qt
Version: 1.3.3-4
Priority: optional
Section: utils
Source: backintime
Maintainer: Jonathan Wiltshire <jmw@debian.org>
Installed-Size: 416 kB
Depends: backintime-common (= 1.3.3-4), libnotify-bin, pkexec, polkitd, python3-dbus.mainloop.pyqt5, python3-pyqt5, x11-utils, python3:any
Recommends: python3-secretstorage
Suggests: meld | kompare
Conflicts: backintime-kde4
Breaks: backintime-qt4 (<< 1.2.1-0.1~)
Replaces: backintime-kde4, backintime-qt4 (<< 1.2.1-0.1~)
Homepage: https://github.com/bit-team/backintime
Download-Size: 73.8 kB
APT-Sources: http://ftp.debian.org/debian bookworm/main amd64 Packages
Description: simple backup/snapshot system (graphical interface)
Back In Time is a framework for rsync and cron for the purpose of
taking snapshots and backups of specified folders. It minimizes disk space use
by taking a snapshot only if the directory has been changed, and hard links
for unmodified files if it has. The user can schedule regular backups using
cron.
.
This is the graphical interface for Back In Time.
backintime-qt – simple backup/snapshot system (graphical interface)

# apt install backintime-qt
$ sudo backintime-qt

It’s slightly more complex than Déjà Dup, but still very manageable.

5. Backing Up your Data on Linux with Grsync (rsync GUI frontend backup tool interface)

Grsync is a simple yet powerful graphical tool for backing up data on Linux. It acts as a front-end for rsync, one of the most trusted file synchronization utilities in the Linux world, but removes the need to remember long command-line options. This makes Grsync ideal for users who want reliable backups without extra complexity.

With Grsync, you can easily select a source and destination folder, such as backing up your home directory to an external drive or a network location. It supports incremental backups, meaning only changed files are copied after the first run, which saves both time and disk space. Useful options like preserving file permissions, deleting obsolete files, and excluding specific directories (for example, cache or temporary files) can be enabled with simple checkboxes.

Another advantage of Grsync is its safety features. You can perform a dry run to preview what will be copied or deleted before actually starting the backup. This reduces the risk of accidental data loss and makes it easier to fine-tune your backup settings. For Linux users looking for a practical and dependable backup solution, Grsync offers a great balance between power and ease of use.

Best Backup Strategy for Desktop Linux Users

For most users, Deja Dup + TimeShift combo should works perfectly:

Déjà Dup → Personal files (documents, photos, videos)
Timeshift → System snapshots

This way, you’re protected from both data loss and system failure.

Final Thoughts

Linux gives you freedom – and that includes freedom to choose how you protect your data.

With modern GUI backup tools, there’s no excuse not to back up regularly. Whether you’re a casual user or a hardcore PC freak, setting up backups takes just a few minutes and can save you hours (or days) of frustration later.

If you’re serious about your Linux system data,
backup early, backup often and you this

will pay you back.

Tags: admin, application menu, Back In Time, backup, Backup Guide, backup tool, Backup Tools, Click, data, Data Protection, Deja Dup, Easy Backups, External, Fedora Backup, File Backup, file deletion, filesystem, Final Thoughts, Grsync, GUI, GUI Backup Tools, home directory, Home Directory Backup, Incremental Backups, Key Features, Linux Backup, Linux Beginners, linux desktop, Linux GUI, Linux Tutorial, linux?, loss, Malware, Open Backups, Perfect, rsync, scenarios, Select, separate, simple, snapshots, Supports, System Recovery, System Snapshots, Timeshift, Ubuntu Backup, Useful, Vorta, Windows System Restore
Posted in Backups, Linux | No Comments »

Installing Freedoom, Freedoom Phase 2, and Other DOOMs on GNU / Linux

Thursday, January 29th, 2026

freedoom-game-on-linux-shot-killed-monster
You think good old DOOM game is Dead ? Think again.

From graphing calculators and printers to ATMs, traffic signs, refrigerators, and even experimental brain-computer interfaces, DOOM has proven itself less a game and more a force of nature. For decades, developers and hackers have ported it onto devices never meant for fast-paced demon slaying, simply because they could. This strange tradition says as much about DOOM’s elegant design as it does about the creativity – and stubbornness – of the open-source community. Perhaps as a mean of extension people from the free realm decide to create a a playable and free remake of it.

If you want classic DOOM action game to have fun on Linux without proprietary game data, Freedoom is the answer.
It’s free, open source, and designed to be fully compatible with the original DOOM engine ,while still delivering you a monsters to kill, shotguns, and extremely questionable life choices.

This article walks you through installing Freedoom, Phase1 Freedoom Phase 2, and running them with popular DOOM source ports on Linux such as gzdoom and chocolate-doom and freedm.

What Is Freedoom (and Why Should You Care)?

Freedoom is a free replacement for the original DOOM game data (WAD files).

Freedoom comes in two main flavors:

Freedoom (Phase 1) – Doom-compatible (monsters, maps, weapons)
Freedoom Phase 2 – Doom II – compatible (bigger maps, more enemies, more chaos)

Think of them as DOOM and DOOM II, but liberated from copyright hell.

1. Install Freedoom on Linux

On most Linux distributions, Freedoom is available directly from the package manager.

Debian / Ubuntu / Linux Mint

# apt install freedoom

This installs:

freedoom1.wad (Phase 1)
freedoom2.wad (Phase 2)

You now officially own free DOOM. Congratulations.

$ freedoom1

$ freedoom2

2. Choose a DOOM Engine (a.k.a. Source Port)

Freedoom doesn’t run by itself- you need a DOOM engine to play it. Luckily, Linux has lots of excellent options.

2.1. Chocolate Doom (Classic Experience)

Chocolate Doom aims to recreate exactly how DOOM felt in the 1990s. Same physics, same limits, same pixelated glory.

Doom engines closely-compatible with Vanilla Doom

Chocolate Doom aims to accurately reproduce the original DOS version of

Doom and other games based on the Doom engine in a form that can be

run on modern computers. Unlike most modern Doom engines, Chocolate Doom

is not derived from the Boom source port and does not inherit its

features (or bugs).

chocolate-doom package contains:

* Chocolate Doom, a port of Id Software's "Doom" (1993)

* Chocolate Heretic, a port of Raven Software's "Heretic" (1994)

* Chocolate Hexen, a port of Raven Software's "Hexen" (1995)

* Chocolate Strife, a recreation of Rogue Entertainment's "Strife" (1996)

These games are designed to behave as similar to the original DOS version as

is possible.

Chocolate Doom supports all flavors of Doom, including The Ultimate Doom, Doom

2 and Final Doom as well as Chex(R) Quest, HacX, Freedoom: Phase 1 and Phase 2

and FreeDM.

All Chocolate game engines require game data to play. For Chocolate Doom,

free game data is available in the freedoom package. Commercial game data for

all four engines can be packaged using "game-data-packager".

To install it simply run:

# apt install chocolate-doom

Run Freedoom:

$ chocolate-doom -iwad /usr/share/games/doom/freedoom1.wad

Or Doom II–style:

$ chocolate-doom -iwad /usr/share/games/doom/freedoom2.wad

2.2. PrBoom+ (Classic but Smoother)

PrBoom+ keeps the old-school feel but adds:

Better performance
Higher resolutions
Fewer headaches

To install it:

# apt install dsda-doom

Run dsda-doom:

$ dsda-doom -iwad freedoom2.wad

Best for:

Vanilla gameplay with modern comfort
Speedrunners
People who want fewer crashes than 1993 allowed

2.3. GZDoom (Modern, Fancy, Wild)

GZDoom is DOOM after drinking several energy drinks:

Hardware acceleration
Mouse look
Mods, lighting, shaders, chaos

Add necessery repository:

$ rm –f /etc/apt/trusted.gpg.d/drdteam.gpg $ rm –f /etc/apt/sources.list.d/drdteam.list
# wget -P /etc/apt/sources.list.d https://debian.drdteam.org/drdteam-$(dpkg –print-architecture).sources
# apt update # apt install gzdoom

Run Original Doom WAD:

$ gzdoom -iwad /var/www/files/doom2.wad

If you need the original doom wads check out my earlier article DOOM 1, DOOM 2, DOOM 3 game wad files for download / Playing Doom on Debian Linux via FreeDoom open source doom engine
Run Freedoom WAD:

$ gzdoom -iwad freedoom2.wad

Best for:

Mods
Modern controls
Turning DOOM into something unrecognizable (but fun)

**3. Installing Other DOOM Engines**

Once you have an engine, you can play nearly any DOOM-compatible game by pointing it at a WAD file.

Popular options include:

FreeDM – A free deathmatch-focused DOOM experience
Heretic / Hexen – Fantasy DOOM (if you own the WADs)
User-made megawads – Thousands exist, ranging from brilliant to unhinged

Most engines let you load extra content like this:

$ gzdoom -iwad freedoom2.wad -file coolmap.wad

freedoom-exit-screen-cool-doom-text-ascii 3.1. Installing FreeDM Port

Install it with:

# apt install freedm
$ freedm

freedm-doom-linux-shot

3.2. Where Are the WAD Files located ?

Typically they are installed to:

/usr/share/games/doom/

Common files:

freedoom1.wad
freedoom2.wad
freedm.wad

You can copy them anywhere if you prefer a custom setup.

3.3. Keyboard, Mouse, and Sanity Settings

After launching:

Open Options → Controls
Enable mouse look (if using GZDoom)
Rebind keys (your pinky deserves mercy)
Increase resolution unless you’re feeling retro

Final Thoughts, DOOM A Forever Classic

Freedoom proves that DOOM isn’t just a game—it’s a platform, a culture, and for those born in the mid and beginning of 90s a travel back machine.

On Linux, you get:

Free game data
Multiple engines
Infinite mods
Zero DRM
Maximum demon destruction 🙂

Whether you want authentic 1993 vibes or modernized mayhem, Freedoom and Linux are a perfect match.
For old school fans and maniacs as well as IT tech guys with nostalgia or hard core angry system administrators this game is a real bliss
as it can be a true anti-stressor and a perfect for as a mean of anger management 🙂

Enjoy the Blast ! Happy Doom-Ing!

Tags: about, action, after, again, aims, ALL, allowed, and, angry, Chocolate Doom, data, Debian Ubuntu Linux Mint, Freedoom Phase, list, Open Options Controls, package, ports, running, these games, without
Posted in Entertainment, Games, Games Linux, Various | No Comments »

How to boost Linux Server Speed with tmpfs and few smart Optimization tweaks

Monday, October 20th, 2025

speed-up-accelerate-wordpress-joomla-drupal-cms-and-mysql-server-with-tmpfs_ramfs_decrease-pageload-times-with-ram-caching.png

If you’ve ever managed a busy Linux server, you’ve probably noticed how I/O bottlenecks can make your system feel sluggish – even when CPU and memory usage look fine. I recently faced this problem on a Debian box running Nginx, PHP-FPM, and MySQL / MariaDB. The fix turned out to be surprisingly simple: using tmpfs for temporary directories and tweaking a few kernel parameters.

1. Identify the resource issue Bottleneck

Running iostat -x 1 showed my /var/lib/mysql drive constantly pegged at 100% utilization, while CPU load stayed low. Classic disk-bound performance issue.

I checked swap usage:

# swapon –show
# free -h

The system was swapping occasionally — not good for performance-critical workloads.

2. Use tmpfs for Cache and Temporary Files

Linux allows you to use part of your RAM as a fast, volatile filesystem. Perfect for things like cache and sessions.

I edited /etc/fstab and added:

tmpfs /tmp tmpfs defaults,noatime,mode=1777,size=1G 0 0
tmpfs /var/cache/nginx tmpfs defaults,noatime,mode=0755,size=512M 0 0

Then:

# mount -a

Immediately, I noticed fewer disk I/O spikes. Nginx’s cache hits were lightning-fast, and PHP temporary files were written in memory instead of SSD.

3. Tune Kernel Parameters

Adding these lines to /etc/sysctl.conf helped reduce swapping and improve responsiveness:

vm.swappiness=10
vm.vfs_cache_pressure=50

Then apply:

# sysctl -p

4. Use tmpfs for MySQL / MariaDB tmpdir (Optional)

If you have enough RAM, move MySQL’s temporary directory to tmpfs:

# mkdir -p /mnt/mysqltmp
# mount -t tmpfs -o size=512M tmpfs /mnt/mysqltmp
# chown mysql:mysql /mnt/mysqltmp

Then set in /etc/mysql/my.cnf: / /etc/mysql/mariadb.cnf

tmpdir = /mnt/mysqltmp

This dramatically speeds up large sorts and temporary table creation.

5. Monitor the Effects and tune up if necessery

After applying these changes, iostat showed disk utilization dropping from 95% to under 20% under the same workload.
Average response times from Nginx dropped by around 30–40%, and the server felt much more responsive.
However other cases might be different so it is a good idea to play around with tmpfs side according to your CPU / Memory system parameters etc. and find out the best values that would fit your Linux setup best.

Short rephrasal

tmpfs is one of those underused Linux features that can make a real-world difference for sysadmins and self-hosters. Just remember that data in tmpfs disappears after reboot, so only use it for volatile data (cache, temp files, etc.).

If you’re running a VPS or small dedicated box and looking for a quick, low-cost performance boost — give tmpfs a try. You might be surprised how much smoother your system feels.

Tags: cnf, data, filesystem, How to, Linux Server Speed, mnt, response times, Short, tmpfs, use
Posted in Linux, Linux and FreeBSD Desktop, MySQL, Performance Tuning, Various | No Comments »

Monitoring Linux server activity with psacct, accton, lastconn commands. Understanding /var/account/pacct and /var/account/acct Acct File Format

Friday, September 5th, 2025

Intro: Understanding Process Accounting in Linux/Unix Systems

In multi-user Unix and Linux environments, keeping track of what’s happening under the hood is essential—for security, system performance, compliance, and resource management. One powerful but often overlooked feature that provides deep visibility into system activity is process accounting.

Process accounting records every command executed on a system, along with key details such as the user who ran it, the time it was executed, how long it ran, how much CPU and memory it consumed, and more. This information is stored in binary log files such as /var/account/pacct or /var/account/acct, which can then be analyzed using specialized tools.

System administrators, security analysts, and auditors use this data for a wide range of purposes: tracking user activity, analyzing system resource usage, investigating security incidents, or even supporting cost allocation in shared computing environments. In regulated industries, process accounting can also serve as a reliable audit trail, helping organizations meet compliance requirements.

While it doesn’t capture full command-line arguments or real-time process behavior, process accounting provides a concise, system-wide history of what has been run and by whom—making it a valuable component of any robust monitoring or auditing strategy.

In this article, we'll explore what process accounting is, how it works, what kind of data it records, and how it can be used to improve system transparency, security, and efficiency.

Understanding the /var/account/pacct or /var/account/acct files involves diving into process accounting on Unix and Linux systems. These files log detailed information about every process that has run on the system. They are binary files and not human-readable so to parse them you the sa / lastcomm tools.

**What are /var/account/pacct and /var/account/acct dirs and common locations across *nix OS-es?**

These are process accounting files used to log per-process execution details.
Created and maintained by the acct package (acctd or psacct, depending on your distro).
Location may vary slightly by distro:
- /var/account/pacct – common on Debian/Ubuntu as well as on FreeBSD / OpenBSD
- /var/account/acct – common on Red Hat/CentOS.
- /var/adm/pact – is location for Sun Solaris UNIX (if you face some legacy systems somewhere and you wonder what it is)

What Do process account data dirs contain ?

These binary log files record information like:

Field	Description
ac_comm	Command name (truncated to 16 characters)
ac_utime	User CPU time used
ac_stime	System CPU time used
ac_etime	Elapsed time
ac_btime	Process start time (in epoch seconds)
ac_uid	User ID of the process owner
ac_gid	Group ID of the process
ac_mem	Average memory usage (in pages)
ac_io	I/O operations (usually blocks read/written)
ac_tty	Controlling terminal
ac_exitcode	Exit status
ac_flag	Flags (forked, exec'd, etc.)

Note: The exact fields and formats may vary based on your OS and kernel version.

How to View or Parse the acct File

Since the file is binary, use tools to interpret it:

1. lastcomm command – Shows last executed commands

# lastcomm

Output includes user, command, CPU time, and date/time.

Note ! that if lastcomm returns an empty output that means the process accounting (if installed on the system) is not yet enabled. To have it return its purposed data enable it.

2. accton – Enables or disables accounting

To enable proc accounting:

# accton on /var/account/pact

To disable proc accounting:

# accton off /var/account/pact

3. Enable disable process accounting on Linux boot

On Redhat based Linux distributions accton can be switch on / off on boot up with the good old chkconfig command

On Debian based Ubuntu / Mint etc. distros

To enable process accounging service on boot:

# update-rc.d acct defaults
# /etc/init.d/acct start

Alternatively you can do it on Debian Linux via its config /etc/default/acct:

# cat /etc/default/acct
# Defaults for acct
# If you want to keep acct installed, but not started automatically, set this
# variable to 0. Because /etc/cron.daily/acct calls the initscript daily, it is
# not sufficient to stop acct once after booting if your machine remains up.
ACCT_ENABLE="1"
# Amount of days that the logs are kept.
ACCT_LOGGING="30"

On Redhat RPM distros good old chkconfig is used

# chkconfig psacct on

4. sa command (summary accounting)

# sa
    3414   54989.71re      64.59cp         0avio     16834k
      34   54603.57re      61.21cp         0avio     38640k   ***other*
      14       1.33re       1.29cp         0avio     74725k   ls
      39      11.88re       1.16cp         0avio     56746k   find
     149      25.51re       0.83cp         0avio     64912k   spamd
      16     293.95re       0.06cp         0avio         0k   kworker/dying*
      22       5.12re       0.01cp         0avio      3951k   sshd
       3       0.01re       0.01cp         0avio     61611k   awstats.pl
       2      11.93re       0.00cp         0avio     58288k   ssh
       3       0.00re       0.00cp         0avio     76224k   php5.6
       2       0.00re       0.00cp         0avio     75824k   php7.3
       7       0.65re       0.00cp         0avio     56846k   bash
       2       0.00re       0.00cp         0avio     75360k   php7.0
       2       0.00re       0.00cp         0avio     69920k   php8.3
       4       0.00re       0.00cp         0avio     58032k   mkvalidrcptto
       2       0.00re       0.00cp         0avio     75408k   php7.4
       2       0.00re       0.00cp         0avio     69920k   php8.2
      60       2.02re       0.00cp         0avio     56882k   cron*
       3       1.24re       0.00cp         0avio     56144k   less
       2       0.00re       0.00cp         0avio     57272k   dpkg
       5       0.01re       0.00cp         0avio     23664k   dig
       3       0.00re       0.00cp         0avio      1443k   lastcomm
       2       0.31re       0.00cp         0avio       940k   dump-acct
       2       0.00re       0.00cp         0avio     13702k   php8.0
       2       0.02re       0.00cp         0avio      1339k   sa
    1226       1.27re       0.00cp         0avio       631k   watchdog*
     560       0.00re       0.00cp         0avio      5316k   zabbix_agentd*
     153       0.00re       0.00cp         0avio     56612k   bash*
     149       0.12re       0.00cp         0avio     56016k   pwd
     139       2.04re       0.00cp         0avio       644k   sh
     128       0.04re       0.00cp         0avio       644k   sessionclean*
     115       0.00re       0.00cp         0avio       644k   which
      71       0.00re       0.00cp         0avio     56278k   sed
      56       0.00re       0.00cp         0avio     56528k   chown
      39       0.19re       0.00cp         0avio       618k   qmail-queue
      39       0.00re       0.00cp         0avio       705k   vdelivermail
      39       0.00re       0.00cp         0avio       590k   qmail-local
      29       0.53re       0.00cp         0avio     56144k   wc
      28       0.24re       0.00cp         0avio     56032k   who
      28       0.00re       0.00cp         0avio       701k   valias
      28       0.00re       0.00cp         0avio       702k   vuserinfo
      24      17.13re       0.00cp         0avio      2282k   qmail-smtpd
      21       5.04re       0.00cp         0avio      3924k   sshd*
      17       1.16re       0.00cp         0avio     56544k   ping
      12       0.00re       0.00cp         0avio     56032k   date
      10       0.00re       0.00cp         0avio     56512k   whoamiSummarizes command usage statistics.

For example to pick up percentage information based on process accounting regarding which process took over most of server’s CPU time

# sa -c
    3414 100.00%   54989.71re 100.00%      64.59cp 100.00%         0avio     16834k
      34    1.00%   54603.57re   99.30%      61.21cp   94.77%         0avio     38640k   ***other*
      14    0.41%       1.33re    0.00%       1.29cp    1.99%         0avio     74725k   ls
      39    1.14%      11.88re    0.02%       1.16cp    1.80%         0avio     56746k   find
     149    4.36%      25.51re    0.05%       0.83cp    1.28%         0avio     64912k   spamd
      16    0.47%     293.95re    0.53%       0.06cp    0.09%         0avio         0k   kworker/dying*
      22    0.64%       5.12re    0.01%       0.01cp    0.02%         0avio      3951k   sshd
       3    0.09%       0.01re    0.00%       0.01cp    0.02%         0avio     61611k   awstats.pl
       2    0.06%      11.93re    0.02%       0.00cp    0.01%         0avio     58288k   ssh
       3    0.09%       0.00re    0.00%       0.00cp    0.00%         0avio     76224k   php5.6
       2    0.06%       0.00re    0.00%       0.00cp    0.00%         0avio     75824k   php7.3
       7    0.21%       0.65re    0.00%       0.00cp    0.00%         0avio     56846k   bash
       2    0.06%       0.00re    0.00%       0.00cp    0.00%         0avio     75360k   php7.0
       2    0.06%       0.00re    0.00%       0.00cp    0.00%         0avio     69920k   php8.3
       4    0.12%       0.00re    0.00%       0.00cp    0.00%         0avio     58032k   mkvalidrcptto
       2    0.06%       0.00re    0.00%       0.00cp    0.00%         0avio     75408k   php7.4
       2    0.06%       0.00re    0.00%       0.00cp    0.00%         0avio     69920k   php8.2
      60    1.76%       2.02re    0.00%       0.00cp    0.00%         0avio     56882k   cron*
       3    0.09%       1.24re    0.00%       0.00cp    0.00%         0avio     56144k   less
       2    0.06%       0.00re    0.00%       0.00cp    0.00%         0avio     57272k   dpkg
       5    0.15%       0.01re    0.00%       0.00cp    0.00%         0avio     23664k   dig
       3    0.09%       0.00re    0.00%       0.00cp    0.00%         0avio      1443k   lastcomm
       2    0.06%       0.31re    0.00%       0.00cp    0.00%         0avio       940k   dump-acct
       2    0.06%       0.00re    0.00%       0.00cp    0.00%         0avio     13702k   php8.0
       2    0.06%       0.02re    0.00%       0.00cp    0.00%         0avio      1339k   sa
    1226   35.91%       1.27re    0.00%       0.00cp    0.00%         0avio       631k   watchdog*
     560   16.40%       0.00re    0.00%       0.00cp    0.00%         0avio      5316k   zabbix_agentd*
     153    4.48%       0.00re    0.00%       0.00cp    0.00%         0avio     56612k   bash*

5. dump-acct (from GNU acct) – Parses the binary files stored on machine

# dump-acct /var/account/pacct

Outputs detailed info per process in a readable format.

6. Installing the process accounting Tools if not already on the system

On Debian/Ubuntu to install:

# apt install acct

On Red Hat/CentOS to install:

# yum install psacct

File Format Details (Technical details)

The pacct file format is defined in C struct form in kernel headers:

struct acct {
    char ac_flag;
    char ac_version;
    uint16_t ac_tty;
    uint32_t ac_exitcode;
    uint32_t ac_uid;
    uint32_t ac_gid;
    uint32_t ac_pid;
    uint32_t ac_ppid;
    uint32_t ac_btime;
    uint32_t ac_etime;
    uint32_t ac_utime;
    uint32_t ac_stime;
    uint32_t ac_mem;
    uint32_t ac_io;
    char ac_comm[16];
};

(Fields may be compressed using "comp_t" – a 16-bit pseudo-floating point format used to save space.)

7. How to Decode comp_t (Optional)

Fields like ac_utime, ac_stime, ac_etime, ac_mem, and ac_io are often stored in comp_t format:

#define MANTSIZE 13
#define EXPSIZE 3

unsigned int expand(comp_t c) {
return ((c & 0x1FFF) << ((c >> 13) & 0x7));
}
So a comp_t value can be interpreted as:
value = mantissa << (exponent)

8. File Growth and Rotation manage

These files grow over time and can become large.
Best practice is to rotate and archive them regularly using logrotate or cron jobs.
Example rotation script (manual):

# mv /var/account/pacct /var/account/pacct.1
# touch /var/account/pacct
# accton /var/account/pacct

Summary what learned

We have learned on what is Process accounting on Linux. The purpose of process accounting is to keep an Audit of what has happened on the Linux server recently thus it is a must to be present on Linux machines, that needs to keep a good history record for potential Security audits like PCI-DSS etc. Here is a short table to recall once again what learned.

Topic	Summary
What it is	Binary log of all executed processes
Location	/var/account/pacct or /var/account/acct
Tools to read	lastcomm, sa, dump-acct
Format	Binary; contains CPU, memory, I/O, UID info
Use case	Auditing, usage tracking, forensics

Tags: about, Account, Accounting, adm, command, CPU, cron jobs, data, debian linux, distros, enable, kernel version, Output, process, Redhat, sshd, time, Understanding, var, www
Posted in Linux, Various | No Comments »

How to convert .p12 ssl certificate to .pem with openssl command

Friday, March 21st, 2025

In cryptography, PKCS #12 defines an archive file format for storing many cryptography objects as a single file. It is commonly used to bundle a private key with its X.509 certificate or to bundle all the members of a chain of trust.

A PKCS #12 file may be encrypted and signed. The internal storage containers, called "SafeBags", may also be encrypted and signed. A few SafeBags are predefined to store certificates, private keys and CRLs. Another SafeBag is provided to store any other data at individual implementer's choice.

PKCS #12 is one of the family of standards called Public-Key Cryptography Standards (PKCS) published by RSA Laboratories.

Privacy-Enhanced Mail (PEM) is a de facto file format for storing and sending cryptographic keys, certificates, and other data, based on a set of 1993 IETF standards defining "privacy-enhanced mail." While the original standards were never broadly adopted and were supplanted by PGP and S/MIME, the textual encoding they defined became very popular. The PEM format was eventually formalized by the IETF in RFC 7468.

If you already have a .P12 certificate password signed provided by someone and you need to convert it a .PEM, this can be done like so:

To convert .p12 certificate :

# Initialize variable
cert_p12_in=your-domain-name-cert.p12
cert_p12_pass='XXXZZZYYYPPPQQQ'
cert_pem_out=your-domain-name-cert.pem

# Extract the private key
openssl pkcs12 -in $cert_p12_in -nocerts -nodes -passin "pass:$cert_p12_pass" 2>/dev/null | sed -ne '/-BEGIN PRIVATE KEY-/,/-END PRIVATE KEY-/p' > $cert_pem_out

# Extract the certificate
openssl pkcs12 -in $cert_p12_in -clcerts -nokeys -passin "pass:$cert_p12_pass" 2>/dev/null | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' >> $cert_pem_out

# Extract the Chain certificate, potentially nothing
openssl pkcs12 -in $cert_p12_in -cacerts -nokeys -chain -passin "pass:$cert_p12_pass" 2>/dev/null | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' >> $cert_pem_out

# Display the result
cat $cert_pem_out

That's all you should have the .p12 to .pem successfully converted.
Cheers ! 🙂

Tags: bundle, command, data, dev, file format, How to, null, openssl, pem, signed
Posted in Everyday Life, Web and CMS | No Comments »

Zabbix Power Shell PS1 script to write zero or one if string is matched inside log file

Monday, December 2nd, 2024

At work we had setup zabbix log file processing for few servers for a service that is doing a Monitoring Health Checks for a a special application via an encrypted strong encrypted tunnel. The app based on the check reports whether the remote side has processed data or not.
As me and my team are not maintainers of the zabbix-server where the zabbix-agents are sending the data, there is a multiple content of data being sent in simply "" empty strings via a zabbix Item setup. Those empty strings however gets stored in the zabbix-server database and since this check is made frequently about 500 hundred records of empty string lines are being written to the zabbix server, we got complaint by the zabbix adminsitrators, that we have to correct our Monitoring setup to not flood the zabbix-server.

Since zabbix cannot catch up the "" empty string and we cannot supress the string from being written in the Item, we needed a way to change the monitoring so that the configured Application check returns 1 (on error) and 0 (on success).

Zabbix even though advanced has a strange when zabbix log[] function, e.g.

log[/path/to/log,,,,skip]

log function, used to analyze a log file and cut out last or first lines of a file simmilar to UNIX's head and tail over log files this is described in the zabbix log file monitoring here . If a string is matched it can return string 1, but if nothing gets matched the result is empty string "" and this empty string cannot be used in a way to analyze the data with Item is used.

There is plenty of discussions online for this weird behavior and many people do offer different approaches to solve the strange situation, but as we have tried with our colleagues sys admins none of those really worked out.

Thus we decided to use the classical way to work around, e.g. to simply use a powershell script that would check a number of lines inside a provided log file analyze if a string gets found and print out value of "1" if the string is matched or "0" "if not and this PS1 script to be set to run via a standard zabbix userparameter script.

This worked well, as all of us are mainly managing Linux systems, and we don't have enough knowledge on powershell we have used our internal Aartificial Intelligence (AI) clone tool to LibreChat – A free and open source ChatGPT clone.

LibreChat includes OpenAI's models, but also others — both open-source and closed-source — and its website promises "seamless integration" with AI services from OpenAI, Azure, Anthropic, and Google — as well as GPT-4, Gemini Vision, and many others. ("Every AI in one place," explains LibreChat's home page.) Plugins even let you make requests to DALL-E or Stable Diffusion for image generations. (LibreChat also offers a database that tracks "conversation state" — making it possible to switch to a different AI model in mid-conversation…)

$logfile = "C:\path\to\your\logfile.log"
$searchString = "-1"

# Get the last 140 lines
$lines = Get-Content $logfile -Tail 140

# Filter lines containing the search string
$found = $lines | Where-Object { $_ -match [regex]::Escape($searchString) }

# Output found lines or 0 if none were found
if ($found) {
$found | ForEach-Object { $_ }
} else {
Write-Host 0
}

You can download and the return_zero_or_one-if-string-matches-in-log-powershell.ps1 script here

Tags: data, empty, file, found, generations, inside, Item, last, log, log files, make, Monitoring Health Checks, none, page, possible, requests, search string, simply, switch, tail, tool, zabbix
Posted in Educational, System Administration, Various, Windows | No Comments »

Improve haproxy logging with custom log-format for better readiability

Friday, April 12th, 2024

Haproxy logging is a very big topic, worthy of many articles, but unfortunately not enough is written on the topic, perhaps for the reason haproxy is free software and most people who use it doesn't follow the philosophy of free software sharing but want to keep, the acquired knowledge on the topic for their own and if possible in the capitalist world most of us live to use it for a Load Balancer haproxy consultancy, consultancy fee or in their daily job as system administrators (web and middleware) or cloud specialist etc. 🙂

Having a good haproxy logging is very important as you need to debug issues with backend machines or some other devices throwing traffic to the HA Proxy.
Thus it is important to build a haproxy logging in a way that it provides most important information and the information is as simple as possible, so everyone can understand what is in without much effort and same time it contains enough debug information, to help you if you want to use the output logs with Graylog filters or process data with some monitoring advanced tool as Prometheus etc.

In our effort to optimize the way haproxy logs via a configured handler that sends the haproxy output to logging handler configured to log through rsyslog, we have done some experiments with logging arguments and came up with few variants, that we liked. In that article the idea is I share this set of logging parameters with hope to help some other guy that starts with haproxy to build a good logging readable and easy to process with scripts log output from haproxy.

The criterias for a decent haproxy logging used are:

1. Log should be simple but not dumb
2. Should be concrete (and not too much complicated)
3. Should be easy to read for the novice and advanced sysadmin

Before starting, have to say that building the logging format seems tedious task but to make it fit your preference could take a lot of time, especially as logging parameters naming is hard to remember, thus the haproxy logging documentation log-format description table comes really handy:

Haproxy log-format paremeters ASCII table

Please refer to the table for log-format defined variables :

+---+------+-----------------------------------------------+-------------+
| R | var  | field name (8.2.2 and 8.2.3 for description)  | type        |
+---+------+-----------------------------------------------+-------------+
|   | %o   | special variable, apply flags on all next var |             |
+---+------+-----------------------------------------------+-------------+
|   | %B   | bytes_read           (from server to client)  | numeric     |
| H | %CC  | captured_request_cookie                       | string      |
| H | %CS  | captured_response_cookie                      | string      |
|   | %H   | hostname                                      | string      |
| H | %HM  | HTTP method (ex: POST)                        | string      |
| H | %HP  | HTTP request URI without query string (path)  | string      |
| H | %HQ  | HTTP request URI query string (ex: ?bar=baz)  | string      |
| H | %HU  | HTTP request URI (ex: /foo?bar=baz)           | string      |
| H | %HV  | HTTP version (ex: HTTP/1.0)                   | string      |
|   | %ID  | unique-id                                     | string      |
|   | %ST  | status_code                                   | numeric     |
|   | %T   | gmt_date_time                                 | date        |
|   | %Ta  | Active time of the request (from TR to end)   | numeric     |
|   | %Tc  | Tc                                            | numeric     |
|   | %Td  | Td = Tt - (Tq + Tw + Tc + Tr)                 | numeric     |
|   | %Tl  | local_date_time                               | date        |
|   | %Th  | connection handshake time (SSL, PROXY proto)  | numeric     |
| H | %Ti  | idle time before the HTTP request             | numeric     |
| H | %Tq  | Th + Ti + TR                                  | numeric     |
| H | %TR  | time to receive the full request from 1st byte| numeric     |
| H | %Tr  | Tr (response time)                            | numeric     |
|   | %Ts  | timestamp                                     | numeric     |
|   | %Tt  | Tt                                            | numeric     |
|   | %Tw  | Tw                                            | numeric     |
|   | %U   | bytes_uploaded       (from client to server)  | numeric     |
|   | %ac  | actconn                                       | numeric     |
|   | %b   | backend_name                                  | string      |
|   | %bc  | beconn      (backend concurrent connections)  | numeric     |
|   | %bi  | backend_source_ip       (connecting address)  | IP          |
|   | %bp  | backend_source_port     (connecting address)  | numeric     |
|   | %bq  | backend_queue                                 | numeric     |
|   | %ci  | client_ip                 (accepted address)  | IP          |
|   | %cp  | client_port               (accepted address)  | numeric     |
|   | %f   | frontend_name                                 | string      |
|   | %fc  | feconn     (frontend concurrent connections)  | numeric     |
|   | %fi  | frontend_ip              (accepting address)  | IP          |
|   | %fp  | frontend_port            (accepting address)  | numeric     |
|   | %ft  | frontend_name_transport ('~' suffix for SSL)  | string      |
|   | %lc  | frontend_log_counter                          | numeric     |
|   | %hr  | captured_request_headers default style        | string      |
|   | %hrl | captured_request_headers CLF style            | string list |
|   | %hs  | captured_response_headers default style       | string      |
|   | %hsl | captured_response_headers CLF style           | string list |
|   | %ms  | accept date milliseconds (left-padded with 0) | numeric     |
|   | %pid | PID                                           | numeric     |
| H | %r   | http_request                                  | string      |
|   | %rc  | retries                                       | numeric     |
|   | %rt  | request_counter (HTTP req or TCP session)     | numeric     |
|   | %s   | server_name                                   | string      |
|   | %sc  | srv_conn     (server concurrent connections)  | numeric     |
|   | %si  | server_IP                   (target address)  | IP          |
|   | %sp  | server_port                 (target address)  | numeric     |
|   | %sq  | srv_queue                                     | numeric     |
| S | %sslc| ssl_ciphers (ex: AES-SHA)                     | string      |
| S | %sslv| ssl_version (ex: TLSv1)                       | string      |
|   | %t   | date_time      (with millisecond resolution)  | date        |
| H | %tr  | date_time of HTTP request                     | date        |
| H | %trg | gmt_date_time of start of HTTP request        | date        |
| H | %trl | local_date_time of start of HTTP request      | date        |
|   | %ts  | termination_state                             | string      |
| H | %tsc | termination_state with cookie status          | string      |
+---+------+-----------------------------------------------+-------------+
R = Restrictions : H = mode http only ; S = SSL only

Our custom log-format built in order to fulfill our needs is as this:

log-format %ci:%cp\ %H\ [%t]\ [%f\ %fi:%fp]\ [%b/%s\ %si:%sp]\ %Tw/%Tc/%Tt\ %B\ %ts\ %ac/%fc/%bc/%sc/%sq/%bq

Once you place the log-format as a default for all haproxy frontend / backends or for a custom defined ones, the output you will get when tailing the log is:

# tail -f /var/log/haproxy.log
Apr 5 21:47:19 10.42.73.83:23262 haproxy-fqdn-hostname.com [05/Apr/2024:21:46:23.879] [ft_FRONTEND_NAME 10.46.108.6:61310] [bk_BACKEND_NAME/bk_appserv3 10.75.226.88:61310] 1/0/55250 55 sD 4/2/1/0/0/0
Apr 5 21:48:14 10.42.73.83:57506 haproxy-fqdn-hostname.com [05/Apr/2024:21:47:18.925] [ft_FRONTEND_NAME 10.46.108.6:61310] [bk_BACKEND_NAME//bk_appserv1 10.35.242.134:61310] 1/0/55236 55 sD 4/2/1/0/0/0
Apr 5 21:49:09 10.42.73.83:46520 haproxy-fqdn-hostname.com [05/Apr/2024:21:48:13.956] [ft_FRONTEND_NAME 10.46.108.6:61310] [bk_BACKEND_NAME//bk_appserv2 10.75.226.89:61310] 1/0/55209 55 sD 4/2/1/0/0/0

If you don't care about extra space and logs being filled with more naming, another variant of above log-format, that makes it even more readable even for most novice sys admin or programmer would look like this:

log-format [%t]\ %H\ [IN_IP]\ %ci:%cp\ [FT_NAME]\ %f:%fp\ [FT_IP]\ %fi:%fp\ [BK_NAME]\ [%b/%s:%sp]\ [BK_IP]\ %si:%sp\ [TIME_WAIT]\ {%Tw/%Tc/%Tt}\ [CONN_STATE]\ {%B\ %ts}\ [STATUS]\ [%ac/%fc/%bc/%sc/%sq/%bq]

Once you apply the config test the haproxy.cfg to make sure no syntax errors during copy / paste from this page

haproxy-serv:~# haproxy -c -f /etc/haproxy/haproxy.cfg
Configuration file is valid

Next restart graceously haproxy

haproxy-serv:~# /usr/sbin/haproxy -D -f /etc/haproxy/haproxy.cfg -p /var/run/haproxy.pid -sf $(cat /var/run/haproxy.pid)

Once you reload haproxy graceously without loosing the established connections in stead of restarting it completely via systemd sysctl restart haproxy:

2024-04-05T21:46:03+02:00 localhost haproxy[1897731]: 193.200.198.195:50714 haproxy-fqdn-hostname.com [05/Apr/2024:21:46:03.012] [FrotnendProd 10.55.0.20:27800] [BackendProd/<NOSRV> -:-] -1/-1/0 0 — 4/1/0/0/0/0
2024-04-05T21:46:03+02:00 localhost haproxy[1897731]: 193.100.193.189:54290 haproxy-fqdn-hostname.com [05/Apr/2024:21:46:03.056] [FrotnendProd 10.55.0.20:27900] [BackendProd/<NOSRV> -:-] -1/-1/0 0 — 4/4/3/0/0/0
2024-04-05T21:46:03+02:00 localhost haproxy[1897731]: 193.100.193.190:26778 haproxy-fqdn-hostname.com [05/Apr/2024:21:46:03.134] [FrotnendProd 10.55.0.20:27900] [BackendProd/tsefas02s 10.35.242.134:27900] 1/-1/0 0 CC 4/4/3/0/0/0

Note that in that log localhost haproxy[pid] is written by rsyslog, you can filter it out by modifying rsyslogd configurations

The only problem with this log-format is not everyone wants to have to much repeating information pointer on which field is what, but I personally liked this one as well because using it even though occuping much more space, makes the log much easier to process with perl or python scripting for data visualize and very for programs that does data or even "big data" analysis.

Tags: about, address, admin, AES, ALL, and, another, are, article, better, connections, custom, data, fi, handy, haproxy, HTTP, Improve, logging, string
Posted in Haproxy, Linux, System Administration, Various | No Comments »

Must have software on freshly installed windows – Essential Software after fresh Windows install

Friday, March 18th, 2016

If you're into IT industry even if you don't like installing frequently Windows or you're completely Linux / BSD user, you will certainly have a lot of friends which will want help from you to re-install or fix their Windows 7 / 8 / 10 OS. At least this is the case with me every year, I'm kinda of obliged to install fresh windowses on new bought friends or relatives notebooks / desktop PCs.

Of course according to for whom the new Windows OS installed the preferrences of necessery software varies, however more or less there is sort of standard list of Windows Software which is used daily by most of Avarage Computer user, such as:

CD Mounting ISO software ( VirtualClone Drive )
CD Recording ( CDBurnerXP )
Archiving / Unarchiving (Zip Rar) Software ( 7-Zip, Winrar, PeaZip )
Office Software ( Apache Open Office – also known widely as LibreOffice )
Internet Browsers ( Mozilla Firefox, Opera, Google Chrome )
Antivirus ( Avira Antivirus or AVG or Kaspersky, Nod32, McAffee (however rarely install non-freeware AV, usually only for companies or organizations) )
Anti-Malware ( Malware Bytes, SpyBot – Search & Destroy )
Chat Audio / Video Conferencing ( Skype / Pidgin (ICQ Client) / Telegram (A free software multiple platform Chat messanger alternative to Skype) )
Video Player ( VLC (Video Lan Client), KMplayer, GOM Player, Quick time, K-lite Codecs )
Music Player ( Winamp (nowadays discontinued), foobar2000, Audacity )
Graphic Design / Graphic Editor / Web Design tools ( The Gimp, Inkscape, PhotoShop and CorelDraw )
Picture Viewer ( IrfanView, FastStone, Picasa )
Java ( Java 7 / 8 JRE and at some PCs where necessery JDK )
PDF Viewers / DJViewPDF ( Adobe Acrobat Reader, Foxit Reader, Sumatra PDF, WinDJView )
Flash Player ( Adobe Shockware Flash player, Adobe Air (install it only on some PCs) )
SSH Client ( MobaXterm (Home Edition), PuTTY (Free Telnet SSH Client) )
FTP Client ( WinSCP, FileZilla )
Programming Text Editor ( Notepad++, Scintilla (SciTE), GVIM, Eclipse (only on developers and programmers PCs) )
Virtualization Software ( VirtualBox, VMWare (install VMWare only on Admin PCs))
Remote Desktop and VNC connector program ( RealVNC, TeamViewer )
Bittorrent Client ( uTorrent or qBittorrent or Transmission )
E-Mail POP3 / Imap Client ( Mozilla Thunderbird )
Screenshotting Tool ( GreenShot )
Password Safe program ( Password Safe or KeePass )
Remote Backup and Storage (Cloud) synchronization ( Dropbox, GoogleDrive, OneDrive (Comes preinstalled on Win 8.1 and Windows 10 )
Tools
– UnixUtils – Installs following common Linux command package tools for Windows CLI

bc-1.05, bison-1.28, bzip2-1.0.2, diffutils-2.7, fileutils-3.16, findutils-4.1, flex-2.5.4, gawk-3.1.0
grep-2.4.2, gsar110, gzip-1.2.4, indent-2.2.9, jwhois-2.4.1, less-340, m4-1.4, make-3.78.1
patch-2.5, recode-3.6, rman-3.0.7, sed-3.02, shellutils-1.9.4, tar-1.12, textutils-2.1
unrar-3.00, wget-1.8.2, which-2.4
– (WinDirStat or SpaceSniffer – Tools that can show you which is the biggest files and directory inside a directory tree)
– WinGrep (Grep: Grep for Windows)
– Everest Home Edition ( Hardware System Information – Shows you what is the PC hardware )

Not to forget a good candidate from the list to install on new fresh windows Installation candidates are:

Winrar
PeaZIP
WinZip
GreenShot (to be able to easily screenshot stuff and save pictures locally and to the cloud)
AnyDesk (non free but very functional alternative to TeamViewer) to be able to remotely access remote PC
TightVNC
ITunes / Spotify (for people who have also iPhone smart phone)
DropBox or pCloud (to have some extra cloud free space)
FBReader (for those reading a lot of books in different formats)
Rufus – Rufus is an efficient and lightweight tool to create bootable USB drives. It helps you to create BIOS or UEFI bootable devices. It helps you to create Windows TO Go drives. It provides support for various disk, format, and partition.
Recuva is a data recovery software for Windows 10 (non free)
EaseUS (for specific backup / restore data purposes but unfortunately (non free)
For designers
Adobe Photoshop
Adobe Illustrator
f.lux – to control brightness of screen and potentially Save your eyes
ImDisk virtual Disk Driver
KeePass / PasswordSafe – to Securely store your passwords
Putty / MobaXterm / SecureCRT / mPutty (for system administrators and programmers that has to deal with Linux / UNIX)

I tend to install on New Windows installs and thus I have more or less systematized the process.

I try to usually stick to free software where possible for each of the above categories as a Free Software enthusiast and luckily nowadays there is a lot of non-priprietary or at least free as in beer software available out there.

For Windows sysadmins or College and other public institutions networks including multiple of Windows Computers which are not inside a domain and also for people in computer repair shops where daily dozens of windows pre-installs or a set of software Automatic updates are necessery make sure to take a look at Ninite

ninite-automate-windows-program-deploy-and-update-on-new-windows-os-openoffice-screenshot

As official website introduces Ninite:

Ninite – Install and Update All Your Programs at Once

Of course as Ninite is used by organizations as NASA, Harvard Medical School etc. it is likely the tool might reports your installed list of Windows software and various other Win PC statistical data to Ninite developers and most likely NSA, but this probably doesn't much matter as this is probably by the moment you choose to have installed a Windows OS on your PC.

ninite-choises-to-build-an-install-package-with-useful-essential-windows-software-screenshot

For Windows System Administrators managing small and middle sized network PCs that are not inside a Domain Controller, Ninite could definitely save hours and at cases even days of boring install and maintainance work. HP Enterprise or HP Inc. Employees or ex-employees would definitely love Ninite, because what Ninite does is pretty much like the well known HP Internal Tool PC COE.

Ninite could also prepare an installer containing multiple applications based on the choice on Ninite's website, so that's also a great thing especially if you need to deploy a different type of Users PCs (Scientific / Gamers / Working etc.)

Perhaps there are also other useful things to install on a new fresh Windows installations, if you're using something I'm missing let me know in comments.

Tags: Archiving Unarchiving Zip Rar Software, case, Chat, com, common, course, daily, data, directory, dozens, Eclipse, fix, free software, inside, installed, list, look, lot, matter, multiple, necessery, New Windows, notebooks, official, OS, Pc, possible, process, public institutions, remote desktop, software, sort, standard, system administrators, things, various, website, Windows, windows computers, windows software, www
Posted in Everyday Life, Remote System Administration, System Administration, Various, Windows | 1 Comment »

Debugging routing and network issues on Linux common approaches. A step by step guide to find out why routing or network service fails

Thursday, November 30th, 2023

For system administrators having a Network issue is among the Hell-ish stuff that can happen every now and then. That is especially true in Heterogenous / Hybrid and complicated Network topologies (with missing well crafted documentation), that were build without an initial overview "on the fly".
Such a networking connectivity or routing issues are faced by every novice, mid or even expert system administrators as the Company's Network IT environments are becoming more and more complicated day by day.

When the "Disaster" of being unable to connect two servers or at times home laptops / PCs to see each other even though on the Physical layer / Transport Layer (Hardware such as external Switches / Routers / Repeaters / Cabling etc.) is Present machines are connected and everything on the 1 Physical Layer from OSI layears is present happens, then it is time to Debug it with some software tools and methods.

To each operating system the tools and methods to test networking connection and routings is a bit different but generally speaking most concepts are pretty much the same across different types of operating systems (Linux ditros / OpenBSD / FreeBSD / Mac OS / Android / iOS / HP-UX / IBM AIX / DOS / Windows etc.).

Debugging network issues across separate operating systems has its variations but in this specific (ideas) are much close to this article. As the goal at that guide will be to point out how to debug network issues on Linux, in future if I have the time or need to debug other OS-es from Linux, I'll try to put an article on how to debug Network issues on Windows when have some time to do it.

Consider to look for the issue following the basic TCP / IP OSI Level model, every system administrator should have idea about it already, it is part of most basic networking courses such as Cisco's CCNA

1. Check what is the Link status of the Interface with ethtool

root@freak:~# ethtool eno1
Settings for eno1:
   Supported ports: [ TP ]
   Supported link modes: 10baseT/Half 10baseT/Full
   100baseT/Half 100baseT/Full
   1000baseT/Full
   Supported pause frame use: Symmetric
   Supports auto-negotiation: Yes
   Supported FEC modes: Not reported
   Advertised link modes: 10baseT/Half 10baseT/Full
   100baseT/Half 100baseT/Full
   1000baseT/Full
   Advertised pause frame use: Symmetric
   Advertised auto-negotiation: Yes
   Advertised FEC modes: Not reported
   Speed: 100Mb/s
   Duplex: Full
   Auto-negotiation: on
   Port: Twisted Pair
   PHYAD: 1
   Transceiver: internal
   MDI-X: on (auto)
   Supports Wake-on: pumbg
   Wake-on: g
Current message level: 0x00000007 (7)
drv probe link
   Link detected: yes

root@freak:~# ethtool eno2
Settings for eno2:
   Supported ports: [ TP ]
   Supported link modes: 10baseT/Half 10baseT/Full
   100baseT/Half 100baseT/Full
   1000baseT/Full
   Supported pause frame use: Symmetric
   Supports auto-negotiation: Yes
   Supported FEC modes: Not reported
   Advertised link modes: 10baseT/Half 10baseT/Full
   100baseT/Half 100baseT/Full
   1000baseT/Full
   Advertised pause frame use: Symmetric
   Advertised auto-negotiation: Yes
   Advertised FEC modes: Not reported
   Speed: 1000Mb/s
   Duplex: Full
   Auto-negotiation: on
   Port: Twisted Pair
   PHYAD: 1
   Transceiver: internal
   MDI-X: on (auto)
   Supports Wake-on: pumbg
   Wake-on: g
Current message level: 0x00000007 (7)
drv probe link
   Link detected: yes

For example lets check only if Cable of Network card is plugged in and detected to have a network connection to remote node or switch and show the connection speed on which the 'autoneg on' (autonegiation option) of the LAN card has detected the network exat maximum speed:

root@pcfreak:~# ethtool eth0|grep -i 'link detected'; ethtool eth0 |grep 'Speed: '
Link detected: yes
Speed: 100Mb/s

1. Check ip command network configuration output

root@freak:~# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master xenbr0 state UP group default qlen 1000
link/ether 70:e2:84:13:44:15 brd ff:ff:ff:ff:ff:ff
altname enp7s0
3: eno2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master xenbr1 state UP group default qlen 1000
link/ether 70:e2:84:13:44:17 brd ff:ff:ff:ff:ff:ff
altname enp8s0
4: xenbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 70:e2:84:13:44:13 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.7/24 brd 192.168.1.255 scope global dynamic xenbr0
valid_lft 7361188sec preferred_lft 7361188sec
5: xenbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 70:e2:84:13:44:15 brd ff:ff:ff:ff:ff:ff
inet 192.168.0.5/24 brd 192.168.0.255 scope global dynamic xenbr1
valid_lft 536138sec preferred_lft 536138sec
10: vif2.0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master xenbr0 state UP group default qlen 2000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
11: vif2.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master xenbr1 state UP group default qlen 2000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
12: vif3.0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master xenbr0 state UP group default qlen 2000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
13: vif3.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master xenbr1 state UP group default qlen 2000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
14: vif4.0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master xenbr0 state UP group default qlen 2000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
15: vif4.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master xenbr1 state UP group default qlen 2000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
16: vif5.0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master xenbr0 state UP group default qlen 2000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
17: vif5.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master xenbr1 state UP group default qlen 2000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
18: vif6.0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master xenbr0 state UP group default qlen 2000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
19: vif6.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master xenbr0 state UP group default qlen 2000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
30: vif17.0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master xenbr0 state UP group default qlen 2000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
31: vif17.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master xenbr1 state UP group default qlen 2000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
34: vif21.0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master xenbr0 state UP group default qlen 2000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
35: vif21.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master xenbr1 state UP group default qlen 2000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
48: vif25.0-emu: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master xenbr0 state UNKNOWN group default qlen 1000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
49: vif25.1-emu: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master xenbr1 state UNKNOWN group default qlen 1000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
50: vif25.0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master xenbr0 state DOWN group default qlen 2000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
51: vif25.1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master xenbr1 state DOWN group default qlen 2000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
118: vif47.0-emu: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master xenbr0 state UNKNOWN group default qlen 1000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
119: vif47.1-emu: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master xenbr1 state UNKNOWN group default qlen 1000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
120: vif47.0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master xenbr0 state DOWN group default qlen 2000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
121: vif47.1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master xenbr1 state DOWN group default qlen 2000
link/ether fe:ff:ff:ff:ff:ff brd ff:ff:ff:ff:ff:ff
root@freak:~#

ip a s (is a also a shortcut command alias) you can enjoy if you have to deal with ip command frequently.

2. Check the status of the interfaces

Old fashioned way is to just do:

/sbin/ifconfig

root@freak:~# ifconfig
eno1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 70:e2:84:13:44:15 txqueuelen 1000 (Ethernet)
RX packets 52366502 bytes 10622469320 (9.8 GiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 242622195 bytes 274688121244 (255.8 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device memory 0xfb200000-fb27ffff
eno2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 70:e2:84:13:44:17 txqueuelen 1000 (Ethernet)
RX packets 220995454 bytes 269698276095 (251.1 GiB)
RX errors 0 dropped 7 overruns 0 frame 0
TX packets 192319925 bytes 166233773782 (154.8 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device memory 0xfb100000-fb17ffff
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
loop txqueuelen 1000 (Local Loopback)
RX packets 2553 bytes 147410 (143.9 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 2553 bytes 147410 (143.9 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
vif17.0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 14517375 bytes 133226551792 (124.0 GiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 139688950 bytes 145111993017 (135.1 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
vif17.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 86113294 bytes 156944058681 (146.1 GiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 181513904 bytes 267892940821 (249.4 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
vif2.0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 1521875 bytes 88282472 (84.1 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 152691174 bytes 278372314505 (259.2 GiB)
TX errors 0 dropped 3 overruns 0 carrier 0 collisions 0
vif2.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 454915 bytes 81069760 (77.3 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 266953989 bytes 425692364876 (396.4 GiB)
TX errors 0 dropped 26 overruns 0 carrier 0 collisions 0
vif21.0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 20043711 bytes 1283926794 (1.1 GiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 141580485 bytes 277396881113 (258.3 GiB)
TX errors 0 dropped 3 overruns 0 carrier 0 collisions 0
vif21.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 73004 bytes 3802174 (3.6 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 267151006 bytes 425621892663 (396.3 GiB)
TX errors 0 dropped 14 overruns 0 carrier 0 collisions 0
vif25.0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
vif25.1: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
vif25.0-emu: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 1000 (Ethernet)
RX packets 2736348 bytes 295661367 (281.9 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 260385509 bytes 265751226663 (247.5 GiB)
TX errors 0 dropped 200 overruns 0 carrier 0 collisions 0
vif25.1-emu: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 1000 (Ethernet)
RX packets 145387 bytes 36011655 (34.3 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 370314760 bytes 394725961081 (367.6 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
vif3.0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 55382861 bytes 130042280927 (121.1 GiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 99040097 bytes 147929196318 (137.7 GiB)
TX errors 0 dropped 1 overruns 0 carrier 0 collisions 0
vif3.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 5132631 bytes 295493762 (281.8 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 262314199 bytes 425416945203 (396.2 GiB)
TX errors 0 dropped 16 overruns 0 carrier 0 collisions 0
vif4.0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 4902015 bytes 615387539 (586.8 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 149342891 bytes 277802504143 (258.7 GiB)
TX errors 0 dropped 1 overruns 0 carrier 0 collisions 0
vif4.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 276927 bytes 30720101 (29.2 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 267132395 bytes 425745668273 (396.5 GiB)
TX errors 0 dropped 14 overruns 0 carrier 0 collisions 0
vif47.0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
vif47.1: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
vif47.0-emu: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 1000 (Ethernet)
RX packets 208745 bytes 20096596 (19.1 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 110905731 bytes 110723486135 (103.1 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
vif47.1-emu: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 1000 (Ethernet)
RX packets 140517 bytes 14596061 (13.9 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 150831959 bytes 162931572456 (151.7 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
vif5.0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 2030528 bytes 363988589 (347.1 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 152264264 bytes 278131541781 (259.0 GiB)
TX errors 0 dropped 1 overruns 0 carrier 0 collisions 0
vif5.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 4169244 bytes 1045889687 (997.4 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 263561100 bytes 424894400987 (395.7 GiB)
TX errors 0 dropped 7 overruns 0 carrier 0 collisions 0
vif6.0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 300242 bytes 16210963 (15.4 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 153909576 bytes 278461295620 (259.3 GiB)
TX errors 0 dropped 2 overruns 0 carrier 0 collisions 0
vif6.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 43 bytes 1932 (1.8 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 154205631 bytes 278481298141 (259.3 GiB)
TX errors 0 dropped 2 overruns 0 carrier 0 collisions 0
xenbr0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.1.8 netmask 255.255.255.0 broadcast 192.168.1.255
ether 70:e2:84:13:44:11 txqueuelen 1000 (Ethernet)
RX packets 13689902 bytes 923464162 (880.6 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 12072932 bytes 1307055530 (1.2 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
xenbr1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.0.3 netmask 255.255.255.0 broadcast 192.168.0.255
ether 70:e2:84:13:44:12 txqueuelen 1000 (Ethernet)
RX packets 626995 bytes 180026901 (171.6 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 12815 bytes 942092 (920.0 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

root@freak:~# ifconfig
eno1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 70:e2:84:13:44:11 txqueuelen 1000 (Ethernet)
RX packets 52373358 bytes 10623034427 (9.8 GiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 242660000 bytes 274734018669 (255.8 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device memory 0xfb200000-fb27ffff
eno2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether 70:e2:84:13:44:12 txqueuelen 1000 (Ethernet)
RX packets 221197892 bytes 269978137472 (251.4 GiB)
RX errors 0 dropped 7 overruns 0 frame 0
TX packets 192573206 bytes 166491370299 (155.0 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device memory 0xfb100000-fb17ffff
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
loop txqueuelen 1000 (Local Loopback)
RX packets 2553 bytes 147410 (143.9 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 2553 bytes 147410 (143.9 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
vif17.0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 14519247 bytes 133248290251 (124.0 GiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 139708738 bytes 145135168676 (135.1 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
vif17.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 86206104 bytes 157189755115 (146.3 GiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 181685983 bytes 268170806613 (249.7 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
vif2.0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 1522072 bytes 88293701 (84.2 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 152712638 bytes 278417240910 (259.2 GiB)
TX errors 0 dropped 3 overruns 0 carrier 0 collisions 0
vif2.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 454933 bytes 81071616 (77.3 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 267218860 bytes 426217224334 (396.9 GiB)
TX errors 0 dropped 26 overruns 0 carrier 0 collisions 0
vif21.0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 20045530 bytes 1284038375 (1.1 GiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 141601066 bytes 277441739746 (258.3 GiB)
TX errors 0 dropped 3 overruns 0 carrier 0 collisions 0
vif21.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 73010 bytes 3802474 (3.6 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 267415889 bytes 426146753845 (396.8 GiB)
TX errors 0 dropped 14 overruns 0 carrier 0 collisions 0
vif25.0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
vif25.1: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
vif25.0-emu: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 1000 (Ethernet)
RX packets 2736576 bytes 295678097 (281.9 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 260429831 bytes 265797660906 (247.5 GiB)
TX errors 0 dropped 200 overruns 0 carrier 0 collisions 0
vif25.1-emu: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 1000 (Ethernet)
RX packets 145425 bytes 36018716 (34.3 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 370770440 bytes 395263409640 (368.1 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
vif3.0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 55392503 bytes 130064444520 (121.1 GiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 99052116 bytes 147951838129 (137.7 GiB)
TX errors 0 dropped 1 overruns 0 carrier 0 collisions 0
vif3.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 5133054 bytes 295517366 (281.8 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 262578665 bytes 425941777243 (396.6 GiB)
TX errors 0 dropped 16 overruns 0 carrier 0 collisions 0
vif4.0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 4902949 bytes 615496460 (586.9 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 149363618 bytes 277847322538 (258.7 GiB)
TX errors 0 dropped 1 overruns 0 carrier 0 collisions 0
vif4.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 276943 bytes 30721141 (29.2 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 267397268 bytes 426270528575 (396.9 GiB)
TX errors 0 dropped 14 overruns 0 carrier 0 collisions 0
vif47.0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
vif47.1: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
vif47.0-emu: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 1000 (Ethernet)
RX packets 208790 bytes 20100733 (19.1 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 110950236 bytes 110769932971 (103.1 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
vif47.1-emu: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 1000 (Ethernet)
RX packets 140551 bytes 14599509 (13.9 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 151287643 bytes 163469024604 (152.2 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
vif5.0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 2030676 bytes 363997181 (347.1 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 152285777 bytes 278176471509 (259.0 GiB)
TX errors 0 dropped 1 overruns 0 carrier 0 collisions 0
vif5.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 4169387 bytes 1045898303 (997.4 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 263825846 bytes 425419251935 (396.2 GiB)
TX errors 0 dropped 7 overruns 0 carrier 0 collisions 0
vif6.0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 300266 bytes 16212271 (15.4 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 153931212 bytes 278506234302 (259.3 GiB)
TX errors 0 dropped 2 overruns 0 carrier 0 collisions 0
vif6.1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
ether fe:ff:ff:ff:ff:ff txqueuelen 2000 (Ethernet)
RX packets 43 bytes 1932 (1.8 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 154227291 bytes 278526238467 (259.3 GiB)
TX errors 0 dropped 2 overruns 0 carrier 0 collisions 0
xenbr0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.1.8 netmask 255.255.255.0 broadcast 192.168.1.255
ether 70:e2:84:13:44:11 txqueuelen 1000 (Ethernet)
RX packets 13690768 bytes 923520126 (880.7 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 12073667 bytes 1307127765 (1.2 GiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
xenbr1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.0.3 netmask 255.255.255.0 broadcast 192.168.0.255
ether 70:e2:84:13:44:12 txqueuelen 1000 (Ethernet)
RX packets 627010 bytes 180028847 (171.6 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 12815 bytes 942092 (920.0 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

To see ethernet interfaces that seem up and then do a ifconfig -a to check whether some interfaces are down (e.g. not shown in the simple ifconfig list).
/sbin/ifconfig -a

! Please note that some virtual IP configurations might not appear and noly be visible in an (ip addr show) command.

3. Check iproute2 for special rt_tables (Routing Tables) rules

By default Linux distributions does not have any additional rules in /etc/iproute2/rt_tables however some Linux router machines, needs to have a multiple Gateways. Perhaps the most elegant way to do multiple routings with Linux is to use iproute2's routing tables rt_tables.

Here is example of an OpenXEN system that has 2 Internet providers attached and routes different traffic via

root@freak:~# cat /etc/iproute2/rt_tables
#
# reserved values
#
255   local
254   main
253   default
100   INET1
200 INET2
0   unspec
#
# local
#
#1   inr.ruhep

root@freak:~# ip rule list
0:   from all lookup local
32762:   from all to 192.168.1.8 lookup INET2
32763:   from 192.168.1.8 lookup INET2
32764:   from all to 192.168.0.3 lookup INET1
32765:   from 192.168.0.3 lookup INET1
32766:   from all lookup main
32767:   from all lookup default
root@freak:~#

4. Using ip route get to find out traffic route (path)

root@freak:~# ip route get 192.168.0.1
192.168.0.1 via 192.168.0.1 dev xenbr1 src 192.168.0.3 uid 0
cache

root@freak:~# /sbin/route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.1.1 0.0.0.0 UG 0 0 0 xenbr0
192.168.0.0 192.168.0.1 255.255.255.0 UG 0 0 0 xenbr1
192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 xenbr1
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 xenbr0
root@freak:~#
root@freak:~# ip route show
default via 192.168.1.1 dev xenbr0
192.168.0.0/24 via 192.168.0.1 dev xenbr1
192.168.0.0/24 dev xenbr1 proto kernel scope link src 192.168.0.3
192.168.1.0/24 dev xenbr0 proto kernel scope link src 192.168.1.8

If you find that gateway is missing you might want to add it with:

root@freak:~# ip route add default via 192.168.5.1

If you need to add a speicic network IP range via separate gateways, you can use commands like:

To add routing for 192.168.0.1/24 / 192.168.1.1/24 via 192.168.0.1 and 192.168.1.1

# /sbin/route add -net 192.168.1.0 netmask 255.255.255.0 gw 192.168.1.1 dev eth1
# /sbin/route add -net 192.168.0.0 netmask 255.255.255.0 gw 192.168.0.1 dev eth1

If you need to delete a configured wrong route with ip command

# ip route del 192.168.1.0/24 via 0.0.0.0 dev eth1
# ip route del 192.168.0.0/24 via 0.0.0.0 dev eth1

5. Use ping (ICMP protocol) the Destionation IP

root@freak:~# ping -c 3 192.168.0.1
PING 192.168.0.1 (192.168.0.1) 56(84) bytes of data.
64 bytes from 192.168.0.1: icmp_seq=1 ttl=64 time=0.219 ms
64 bytes from 192.168.0.1: icmp_seq=2 ttl=64 time=0.295 ms
64 bytes from 192.168.0.1: icmp_seq=3 ttl=64 time=0.270 ms
— 192.168.0.1 ping statistics —
3 packets transmitted, 3 received, 0% packet loss, time 2048ms
rtt min/avg/max/mdev = 0.219/0.261/0.295/0.031 ms
root@freak:~# ping -c 3 192.168.0.39
PING 192.168.0.39 (192.168.0.39) 56(84) bytes of data.
From 192.168.1.80: icmp_seq=2 Redirect Host(New nexthop: 192.168.0.39)
From 192.168.1.80: icmp_seq=3 Redirect Host(New nexthop: 192.168.0.39)
From 192.168.1.80 icmp_seq=1 Destination Host Unreachable

— 192.168.0.39 ping statistics —
3 packets transmitted, 0 received, +1 errors, 100% packet loss, time 2039ms
pipe 3

Note that sometimes you might get 100% traffic loss but still have connection to the destionation in case if the ICMP protocol is filtered for security.

However if you get something like Network is unreachable that is usually an indicator of some routing problem or wrongly configured network netmask.

root@freak:~# ping 192.168.0.5
ping: connect: Network is unreachable

Test network with different packet size. To send 8972 bytes of payload in a Ethernet frame without fragmentation, the following command can be used:

root@pcfreak:~# ping -s 8972 -M do -c 4 freak
PING xen (192.168.1.8) 8972(9000) bytes of data.
ping: local error: message too long, mtu=1500
ping: local error: message too long, mtu=1500
ping: local error: message too long, mtu=1500
^C
— xen ping statistics —
3 packets transmitted, 0 received, +3 errors, 100% packet loss, time 2037ms
root@pcfreak:~#

-M pmtudisc_opt
Select Path MTU Discovery strategy. pmtudisc_option may be either do (prohibit fragmentation, even local one), want (do PMTU discovery, fragment locally when packet size is
large), or dont (do not set DF flag).

root@pcfreak:~# ping -s 8972 -M want -c 4 freak
PING xen (192.168.1.8) 8972(9000) bytes of data.
8980 bytes from xen (192.168.1.5): icmp_seq=1 ttl=64 time=2.18 ms
8980 bytes from xen (192.168.1.5): icmp_seq=2 ttl=64 time=1.90 ms
8980 bytes from xen (192.168.1.5): icmp_seq=3 ttl=64 time=2.10 ms
^C
— xen ping statistics —
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 1.901/2.059/2.178/0.116 ms
root@pcfreak:~#

-M do: prohibit fragmentation
-s 8972 8972 bytes of data
ICMP header: 8 bytes
IP header: 20 bytes (usually, it can be higher)
8980 bytes of bytes is the IP payload

These commands can be used to capture for MTU (maximum transmition units) related issues between hosts that are preventing for hosts to properly send traffic between themselves.
A common issue for Linux hosts to be unable to see each other on the same network is caused by Jumbo Frames (MTU 9000) packets enabled on one of the sides and MTU of 1500 on the other side.
Thus it is always a good idea to thoroughully look up all configured MTUs for all LAN Devices on each server.

6. Check traceroute path to host

If there is no PING but ip route get shows routing is properly configured and the routes existing in the Linux machine routing tables, next step is to check the output of traceroute / tracepath / mtr

raceroute to 192.168.0.1 (192.168.0.1), 30 hops max, 60 byte packets
1 pcfreak (192.168.0.1) 0.263 ms 0.166 ms 0.119 ms
root@freak:~# tracepath 192.168.1.1
1?: [LOCALHOST] pmtu 1500
1: vivacom-gigabit-router 0.925ms reached
1: vivacom-gigabit-router 0.835ms reached
Resume: pmtu 1500 hops 1 back 1

It might be useful to get a frequent output of the command (especially on Linux hosts) where mtr command is not installed with:

root@freak:~# watch -n 0.1 traceroute 192.168.0.1

root@freak:~# traceroute -4 google.com
traceroute to google.com (172.217.17.110), 30 hops max, 60 byte packets
1 vivacom-gigabit-router (192.168.1.1) 0.657 ms 1.280 ms 1.647 ms
2 213.91.190.130 (213.91.190.130) 7.983 ms 8.168 ms 8.097 ms
3 * * *
4 * * *
5 212-39-66-222.ip.btc-net.bg (212.39.66.222) 16.613 ms 16.336 ms 17.151 ms
6 * * *
7 142.251.92.65 (142.251.92.65) 18.808 ms 13.246 ms 209.85.254.242 (209.85.254.242) 15.541 ms
8 142.251.92.3 (142.251.92.3) 14.223 ms 142.251.227.251 (142.251.227.251) 14.507 ms 142.251.92.3 (142.251.92.3) 15.328 ms
9 ams15s29-in-f14.1e100.net (172.217.17.110) 14.097 ms 14.909 ms 142.251.242.230 (142.251.242.230) 13.481 ms
root@freak:~#

If you have MTR then you can get plenty of useful additional information such as the Network HOP name or the Country location of the HOP.

To get HOP name:

root@freak:~# mtr -z google.com

To get info on where (which Country) exactly network HOP is located physically:

root@freak:~# mtr -y 2 google.com

7. Check iptables INPUT / FORWARD / OUTPUT rules are messing with something

# iptables -L -n
…
# iptables -t nat -L -n
…

Ideally you would not have any firewall

# iptables -L -n
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination

# iptables -t nat -L -n
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination

In case if something like firewalld is enabled as a default serviceto provide some modern Linux firewall as Ubuntu and Redhat / CentOS / Fedoras has it often turned on as a service stop and disable the service

# systemctl stop firewalld
# systemctl disable firewalld

8. Debug for any possible MAC address duplicates

root@pcfrxen:~# arp -an
? (192.168.1.33) at 00:16:3e:59:96:9e [ether] on eth0
? (192.168.1.1) at 18:45:93:c6:d8:00 [ether] on eth1
? (192.168.0.1) at 8c:89:a5:f2:e8:d9 [ether] on eth1
? (192.168.1.1) at 18:45:93:c6:d8:00 [ether] on eth0
? (192.168.1.11) at 7c:0a:3f:89:b6:fa [ether] on eth1
? (192.168.1.17) at <incomplete> on eth0
? (192.168.1.37) at 00:16:3e:ea:05:ce [ether] on eth0
? (192.168.1.80) at 8c:89:a5:f2:e7:d8 [ether] on eth0
? (192.168.1.11) at 7c:0a:3f:89:a5:fa [ether] on eth0
? (192.168.1.30) at 00:16:3e:bb:46:45 [ether] on eth1
? (192.168.0.210) at 00:16:3e:68:d9:55 [ether] on eth1
? (192.168.1.30) at 00:16:3e:bb:46:45 [ether] on eth0
? (192.168.1.18) at 00:16:3e:0d:40:05 [ether] on eth1
? (192.168.0.211) at 00:16:3e:4d:41:05 [ether] on eth1
? (192.168.1.35) at 00:16:3e:d1:8f:77 [ether] on eth0
? (192.168.1.18) at 00:16:3e:0d:43:05 [ether] on eth0
? (192.168.1.28) at 00:16:3e:04:12:1c [ether] on eth1
? (192.168.0.3) at 70:e2:84:13:43:12 [ether] on eth1
? (192.168.0.208) at 00:16:3e:51:de:9c [ether] on eth1
? (192.168.0.241) at 00:16:3e:0d:48:06 [ether] on eth1
? (192.168.1.28) at 00:16:3e:04:12:1c [ether] on eth0
? (192.168.1.33) at 00:16:3e:59:97:8e [ether] on eth1
? (192.168.0.241) at 00:16:3e:0d:45:06 [ether] on eth0
? (192.168.0.209) at 00:16:3e:5c:df:96 [ether] on eth1
root@pcfrxen:~# ip neigh show
192.168.1.33 dev eth0 lladdr 00:16:3e:59:96:9e REACHABLE
192.168.1.1 dev eth1 lladdr 18:45:93:c6:d8:00 STALE
192.168.0.1 dev eth1 lladdr 8c:89:a5:f2:e8:d9 REACHABLE
192.168.1.1 dev eth0 lladdr 18:45:93:c6:d9:01 REACHABLE
192.168.1.11 dev eth1 lladdr 7c:0a:3f:89:a6:fb STALE
192.168.1.17 dev eth0 FAILED
192.168.1.37 dev eth0 lladdr 00:16:3e:ea:06:ce STALE
192.168.1.80 dev eth0 lladdr 8c:89:a5:f2:e8:d9 REACHABLE
192.168.1.11 dev eth0 lladdr 7c:0a:3f:89:a7:fa STALE
192.168.1.30 dev eth1 lladdr 00:16:3e:bb:45:46 STALE
192.168.0.210 dev eth1 lladdr 00:16:3e:68:d8:56 REACHABLE
192.168.1.30 dev eth0 lladdr 00:16:3e:bb:45:46 STALE
192.168.1.18 dev eth1 lladdr 00:16:3e:0d:48:04 STALE
192.168.0.211 dev eth1 lladdr 00:16:3e:4d:40:04 STALE
192.168.1.35 dev eth0 lladdr 00:16:3e:d2:8f:76 STALE
192.168.1.18 dev eth0 lladdr 00:16:3e:0d:48:06 STALE
192.168.1.28 dev eth1 lladdr 00:16:3e:04:11:2c STALE
192.168.0.3 dev eth1 lladdr 70:e2:84:13:44:13 STALE
192.168.0.208 dev eth1 lladdr 00:16:3e:51:de:9c REACHABLE
192.168.0.241 dev eth1 lladdr 00:16:3e:0d:48:07 STALE
192.168.1.28 dev eth0 lladdr 00:16:3e:04:12:1c REACHABLE
192.168.1.33 dev eth1 lladdr 00:16:3e:59:96:9e STALE
192.168.0.241 dev eth0 lladdr 00:16:3e:0d:49:06 STALE
192.168.0.209 dev eth1 lladdr 00:16:3e:5c:dd:97 STALE
root@pcfrxen:~#

9. Check out with netstat / ss for any irregularities such as high amount of error of faulty ICMP / TCP / UDP network packs

For example check out the netstat network stack output

# netstat -s

root@pcfrxen:~# netstat -s
Ip:
Forwarding: 2
440044929 total packets received
1032 with invalid addresses
0 forwarded
0 incoming packets discarded
439988902 incoming packets delivered
396161852 requests sent out
3 outgoing packets dropped
100 dropped because of missing route
Icmp:
1025 ICMP messages received
540 input ICMP message failed
ICMP input histogram:
destination unreachable: 1014
timeout in transit: 11
519 ICMP messages sent
0 ICMP messages failed
ICMP output histogram:
destination unreachable: 519
IcmpMsg:
InType3: 1014
InType11: 11
OutType3: 519
Tcp:
1077237 active connection openings
1070510 passive connection openings
1398236 failed connection attempts
111345 connection resets received
83 connections established
438293250 segments received
508143650 segments sent out
42567 segments retransmitted
546 bad segments received
329039 resets sent
Udp:
1661295 packets received
278 packets to unknown port received
0 packet receive errors
1545720 packets sent
0 receive buffer errors
0 send buffer errors
IgnoredMulti: 33046
UdpLite:
TcpExt:
1 invalid SYN cookies received
1398196 resets received for embryonic SYN_RECV sockets
1737473 packets pruned from receive queue because of socket buffer overrun
1118775 TCP sockets finished time wait in fast timer
638 time wait sockets recycled by time stamp
656 packetes rejected in established connections because of timestamp
2218959 delayed acks sent
2330 delayed acks further delayed because of locked socket
Quick ack mode was activated 7172 times
271799723 packet headers predicted
14917420 acknowledgments not containing data payload received
171078735 predicted acknowledgments
52 times recovered from packet loss due to fast retransmit
TCPSackRecovery: 337
Detected reordering 1551 times using SACK
Detected reordering 1501 times using reno fast retransmit
Detected reordering 61 times using time stamp
9 congestion windows fully recovered without slow start
38 congestion windows partially recovered using Hoe heuristic
TCPDSACKUndo: 241
104 congestion windows recovered without slow start after partial ack
TCPLostRetransmit: 11550
1 timeouts after reno fast retransmit
TCPSackFailures: 13
3772 fast retransmits
2 retransmits in slow start
TCPTimeouts: 24104
TCPLossProbes: 101748
TCPLossProbeRecovery: 134
TCPSackRecoveryFail: 3
128989224 packets collapsed in receive queue due to low socket buffer
TCPBacklogCoalesce: 715034
TCPDSACKOldSent: 7168
TCPDSACKOfoSent: 341
TCPDSACKRecv: 16612
150689 connections reset due to unexpected data
27063 connections reset due to early user close
17 connections aborted due to timeout
TCPDSACKIgnoredOld: 158
TCPDSACKIgnoredNoUndo: 13514
TCPSpuriousRTOs: 9
TCPSackMerged: 1191
TCPSackShiftFallback: 1011
TCPDeferAcceptDrop: 699473
TCPRcvCoalesce: 3311764
TCPOFOQueue: 14289375
TCPOFOMerge: 356
TCPChallengeACK: 621
TCPSYNChallenge: 621
TCPSpuriousRtxHostQueues: 4
TCPAutoCorking: 1605205
TCPFromZeroWindowAdv: 132380
TCPToZeroWindowAdv: 132441
TCPWantZeroWindowAdv: 1445495
TCPSynRetrans: 23652
TCPOrigDataSent: 388992604
TCPHystartTrainDetect: 69089
TCPHystartTrainCwnd: 3264904
TCPHystartDelayDetect: 4
TCPHystartDelayCwnd: 128
TCPACKSkippedPAWS: 3
TCPACKSkippedSeq: 2001
TCPACKSkippedChallenge: 2
TCPWinProbe: 123043
TCPKeepAlive: 4389
TCPDelivered: 389507445
TCPAckCompressed: 7343781
TcpTimeoutRehash: 23311
TcpDuplicateDataRehash: 8
TCPDSACKRecvSegs: 17335
IpExt:
InMcastPkts: 145100
OutMcastPkts: 9429
InBcastPkts: 18226
InOctets: 722933727848
OutOctets: 759502627470
InMcastOctets: 58227095
OutMcastOctets: 3284379
InBcastOctets: 1756918
InNoECTPkts: 440286946
InECT0Pkts: 936

List all listening established connections to host

# netstat -ltne

List all UDP / TCP connections

# netstat -ltua

or if you prefer to do it with the newer and more comprehensive tool ss:

List all listening TCP connections

# ss -lt

List all listening UDP connections

# ss -ua

Display statistics about recent connections

root@pcfrxen:~# ss -s
Total: 329
TCP: 896 (estab 70, closed 769, orphaned 0, timewait 767)
Transport Total IP IPv6
RAW   0 0 0
UDP   40 36 4
TCP   127 118 9
INET   167 154 13
FRAG   0 0 0

If you need to debug some specific sport or dport filter out the connection you need by port number

# ss -at '( dport = :22 or sport = :22 )'

Debug for any possible issues with ICMP unreachable but ports reachable with NMAP / telnet / Netcat

# nc 192.168.0.1 -vz

root@pcfrxen:/ # nc 192.168.0.1 80 -vz
pcfreak [192.168.0.1] 80 (http) open

root@pcfrxen:/ # nc 192.168.0.1 5555 -vz
pcfreak [192.168.0.1] 5555 (?) : Connection refused

root@pcfrxen:/# telnet 192.168.0.1 3128
Trying 192.168.0.1…
Connected to 192.168.0.1.
Escape character is '^]'.
^]
telnet> quit
Connection closed.

root@pcfrxen:/# nmap -sS -P0 192.168.0.1 -p 443 -O
Starting Nmap 7.80 ( https://nmap.org ) at 2023-11-27 19:51 EET
Nmap scan report for pcfreak (192.168.0.1)
Host is up (0.00036s latency).
PORT STATE SERVICE
443/tcp open https
MAC Address: 8C:89:A5:F2:E8:D8 (Micro-Star INT'L)
Warning: OSScan results may be unreliable because we could not find at least 1 open and 1 closed port
Aggressive OS guesses: Linux 3.11 (96%), Linux 3.1 (95%), Linux 3.2 (95%), AXIS 210A or 211 Network Camera (Linux 2.6.17) (94%), Linux 2.6.32 (94%), Linux 3.10 (94%), Linux 2.6.18 (93%), Linux 3.2 – 4.9 (93%), ASUS RT-N56U WAP (Linux 3.4) (93%), Linux 3.16 (93%)
No exact OS matches for host (test conditions non-ideal).
Network Distance: 1 hop
OS detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 6.24 seconds
root@pcfrxen:/#

10. Add static MAC address to Ethernet Interface (if you find a MAC address being wrongly assigned to interface)

Sometimes problems with network unrechability between hosts is caused by wrongly defined MAC addresses on a Switch that did not correspond correctly to the ones assigned on the Linux host.
The easiest resolution here if you don't have access to Switch in work environment is to reassign the default MAC addresses of interfaces to proper MAC addresses, expected by remote router.

root@pcfrxen:/#  /sbin/ifconfig eth2 hw ether 8c:89:a5:f2:e8:d6
root@pcfrxen:/#  /sbin/ifconfig eth1 hw ether 8c:89:a5:f2:e8:d5

root@pcfrxen:/#  ifconfig eth0|grep -i ether
ether 8c:89:a5:f2:e8:d6 txqueuelen 1000 (Ethernet)

11. Check for Network Address Translation (NAT) misconfigurations

If you do use some NAT-ing between Linux host and the remote Network Device you cannot reach, make sure IP Forwarding is enabled (i.e. /etc/sysctl.conf was not mistakenly overwritten by a script or admin for whatever reason).

root@server:~# sysctl net.ipv4.ip_forward
net.ipv4.ip_forward = 1
root@server:~# sysctl net.ipv4.conf.all.forwarding
net.ipv4.conf.all.forwarding = 1

root@server:~# sysctl net.ipv6.conf.all.forwarding
net.ipv6.conf.all.forwarding = 0

12. Check for Resolving DNS irregularities with /etc/resolv.conf

If network connectivity is okay on TCP / IP , UDP Level but problems with DNS of course, check what you have configured inside /etc/resolv.conf

And if use newer Linux distributions and have resolving managed by systemd check status of resolvectl

root@server:~# cat /etc/resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
# DO NOT EDIT THIS FILE BY HAND — YOUR CHANGES WILL BE OVERWRITTEN
# 127.0.0.53 is the systemd-resolved stub resolver.
# run "resolvectl status" to see details about the actual nameservers.
nameserver 127.0.0.1
search pc-freak.net
domain pc-freak.net
nameserver 8.8.8.8
nameserver 8.8.4.4
nameserver 109.104.195.2
nameserver 109.104.195.1
nameserver 208.67.222.222
nameserver 208.67.220.220
options timeout:2 rotate
root@pcfreak:~#

root@server:~# resolvectl status
Global
Protocols: -LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
resolv.conf mode: stub
Link 2 (ens3)
Current Scopes: DNS
Protocols: +DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
Current DNS Server: 192.168.5.1
DNS Servers: 192.168.5.1

As seen see, the systemd-resolved service is used to provide domain names resolution and we can modify its configuration file /etc/systemd/resolved.conf to add the DNS server – the following line is set (two DNS servers’ addresses are added):

For example …

DNS=8.8.8.8

13. Fix problems with wrongly configured Network Speed between hosts

It is not uncommon to have a Switch between two Linux hosts that is set to communicate on a certain maximum amount of Speed but a Linux host is set to communicate or lesser or more of Speed, this might create network issues so in such cases make sure either you use the Auto Negitionation network feature
or set both sides to be communicating on the same amount of network speed.

To turn on auto negotiation for ether interface

# ethtool -s eth1 speed 1000 duplex full autoneg on

For example to set a Linux network interface to communicate on 1 Gigabit speed and switch off autonegotiation off.

# ethtool -s eth1 speed 1000 duplex full autoneg off

14. Check arp and icmp traffic with tcpdump

On both sides where the IPs can't see each other we can run a tcpdump to check the ARP and ICMP traffic flowing between the hosts.

# tcpdump -i eth1 arp or icmp
cpdump: verbose output suppressed, use -v[v]… for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), snapshot length 262144 bytes
15:29:07.001841 IP freak-eth1 > pcfr_hware_local_ip: ICMP echo request, id 13348, seq 65, length 64
15:29:07.001887 IP pcfr_hware_local_ip > freak-eth1: ICMP echo reply, id 13348, seq 65, length 64
15:29:07.598413 ARP, Request who-has pcfr_hware_local_ip tell zabbix-server, length 46
15:29:07.598425 ARP, Reply pcfr_hware_local_ip is-at 8c:89:a5:f2:e8:d8 (oui Unknown), length 28
15:29:07.633055 ARP, Request who-has freak_vivacom_auto_assigned_dhcp_ip tell 192.168.1.1, length 46
15:29:08.025824 IP freak-eth1 > pcfr_hware_local_ip: ICMP echo request, id 13348, seq 66, length 64
15:29:08.025864 IP pcfr_hware_local_ip > freak-eth1: ICMP echo reply, id 13348, seq 66, length 64

# tcpdump -i eth1 -vvv
…

If you want to sniff for TCP protocol and specific port and look up for DATA transfered for SMTP you can use something like:

# tcpdump -nNxXi eth0 -s 0 proto TCP and port 25

If you need a bit more thorough explanation on what it would do check out my previous article How to catch / track mail server traffic abusers with tcpdump.

15. Debugging network bridge issues

Having bridge network interface is another brink where things could go totally wrong.
If you have network bridges configured, check out what is the status of the bridge.

root@freak:/etc/network# brctl show
bridge name   bridge id       STP enabled   interfaces
xenbr0       8000.70e284134411   yes       eno1
                           vif1.0
                           vif10.0
                           vif16.0
                           vif16.0-emu
                           vif2.0
                           vif3.0
                           vif4.0
                           vif5.0
                           vif6.0
                           vif9.0
                           vif9.0-emu
xenbr1       8000.70e284134412   yes       eno2
                           vif1.1
                           vif10.1
                           vif16.1
                           vif16.1-emu
                           vif2.1
                           vif3.1
                           vif4.1
                           vif5.1
                           vif6.1
                           vif9.1
                           vif9.1-emu

Check out any configurations such as /etc/sysconfig/network-scripts/ifcfg-* are not misconfigured if on Redhat / CentOS / Fedora.
Or if on Debian / Ubuntu and other deb based Linuxes look up for /etc/network/interfaces config problems that might be causing the bridge to misbehave.

For example one bridge network issue, I've experienced recently is related to bridge_ports variable configured as bridge_ports all.
This was causing the second bridge xenbr1 to be unable to see another local network that was directly connected with a cable to it.

The fix was bridge_ports none. Finding out this trivial issue caused by a restored network config from old backup took me days to debug.
As everything seemed on a network level to be perfect just like in Physical layer, same way and on Software level, routings were okay.
Checked everything multiple times and did not see anything irregular. ping was missing and hosts cannot see each other even though having the right netmask and
network configuration in place.

Below is my /etc/network/interfaces configuration with the correct bridge_ports none changed.

root@freak:/etc/network# cat /etc/network/interfaces
auto lo
iface lo inet loopback

auto eno1
allow-hotplug eno1
iface eno1 inet manual
dns-nameservers 127.0.0.1 8.8.8.8 8.8.4.4 207.67.222.222 208.67.220.220
auto eno2
allow-hotplug eno2
iface eno2 inet manual
dns-nameservers 127.0.0.1 8.8.8.8 8.8.4.4 207.67.222.222 208.67.220.220
auto xenbr0
allow-hotplug xenbr0
# Bridge setup
# fetching dhcp ip from 192.168.1.20 (vivacom fiber optics router) routing traffic via 1Gigabit network
iface xenbr0 inet dhcp
hwaddress ether 70:e2:84:13:44:11
# address 192.168.1.5/22
address 192.168.1.5
netmask 255.255.252.0
# address 192.168.1.8 if dhcp takes from vivacom dhcpd
bridge_ports eno1
gateway 192.168.1.20
bridge_stp on
bridge_waitport 0
bridge_fd 0
bridge_ports none
dns-nameservers 8.8.8.8 8.8.4.4
auto xenbr1
# fetching dhcp ip from pc-freak.net (192.168.0.1) bergon.net routing traffic through it
allow-hotplug xenbr1
iface xenbr1 inet dhcp
hwaddress ether 70:e2:84:13:44:11
## address 192.168.0.3/22
address 192.168.0.8
netmask 255.255.252.0
# address 192.168.0.8 if dhcp takes from vivacom dhcpd (currently mac deleted from vivacom router)
# address 192.168.0.9 if dhcp takes from pc-freak.net hware host
# hwaddress ether 70:e2:84:13:44:13
gateway 192.168.0.1
bridge_ports eno2
bridge_stp on
bridge_waitport 0
bridge_fd 0
bridge_ports none
dns-nameservers 8.8.8.8 8.8.4.4
root@freak:/etc/network#

root@freak:/etc/network# brctl showstp xenbr0
xenbr0
bridge id       8000.70e284134411
designated root   8000.70e284134411
root port       0           path cost       0
max age       20.00           bridge max age       20.00
hello time       2.00           bridge hello time   2.00
forward delay       15.00           bridge forward delay   15.00
ageing time       0.00
hello timer       1.31           tcn timer       0.00
topology change timer   0.00           gc timer       0.00
flags

eno1 (1)
port id       8001           state       forwarding
designated root   8000.70e284134411   path cost       19
designated bridge   8000.70e284134411   message age timer   0.00
designated port   8001           forward delay timer   0.00
designated cost   0           hold timer       0.31
flags
vif1.0 (2)
port id       8002           state       forwarding
designated root   8000.70e284134411   path cost       100
designated bridge   8000.70e284134411   message age timer   0.00
designated port   8002           forward delay timer   0.00
designated cost   0           hold timer       0.31
flags
vif10.0 (12)
port id       800c           state       forwarding
designated root   8000.70e284134411   path cost       100
designated bridge   8000.70e284134411   message age timer   0.00
designated port   800c           forward delay timer   0.00
designated cost   0           hold timer       0.31
flags
vif16.0 (13)
port id       800d           state       disabled
designated root   8000.70e284134411   path cost       100
designated bridge   8000.70e284134411   message age timer   0.00
designated port   800d           forward delay timer   0.00
designated cost   0           hold timer       0.00
flags
vif16.0-emu (14)
port id       800e           state       forwarding
designated root   8000.70e284134411   path cost       100
designated bridge   8000.70e284134411   message age timer   0.00
designated port   800e           forward delay timer   0.00
designated cost   0           hold timer       0.31
flags
vif2.0 (4)
port id       8004           state       forwarding
designated root   8000.70e284134411   path cost       100
designated bridge   8000.70e284134411   message age timer   0.00
designated port   8004           forward delay timer   0.00
designated cost   0           hold timer       0.31
flags
vif3.0 (5)
port id       8005           state       forwarding
designated root   8000.70e284134411   path cost       100
designated bridge   8000.70e284134411   message age timer   0.00
designated port   8005           forward delay timer   0.00
designated cost   0           hold timer       0.31
flags
vif4.0 (3)
port id       8003           state       forwarding
designated root   8000.70e284134411   path cost       100
designated bridge   8000.70e284134411   message age timer   0.00
designated port   8003           forward delay timer   0.00
designated cost   0           hold timer       0.31
flags
vif5.0 (6)
port id       8006           state       forwarding
designated root   8000.70e284134411   path cost       100
designated bridge   8000.70e284134411   message age timer   0.00
designated port   8006           forward delay timer   0.00
designated cost   0           hold timer       0.31
flags
vif6.0 (7)
port id       8007           state       forwarding
designated root   8000.70e284134411   path cost       100
designated bridge   8000.70e284134411   message age timer   0.00
designated port   8007           forward delay timer   0.00
designated cost   0           hold timer       0.31
flags
vif9.0 (10)
port id       800a           state       disabled
designated root   8000.70e284134411   path cost       100
designated bridge   8000.70e284134411   message age timer   0.00
designated port   800a           forward delay timer   0.00
designated cost   0           hold timer       0.00
flags
vif9.0-emu (11)
port id       800b           state       forwarding
designated root   8000.70e284134411   path cost       100
designated bridge   8000.70e284134411   message age timer   0.00
designated port   800b           forward delay timer   0.00
designated cost   0           hold timer       0.31
flags
root@freak:/etc/network#

Sum it up

We have learned how to debug various routing issues, how to add and remote default gateways, check network reachability with ICMP protocol with ping, traceroute as well check for DNS issues and given some hints how to resolve DNS misconfigurations.
We also learned how to check the configured Network interfaces certain settings and resolve issues caused by Network sides max Speed misconfigurations as well how to track and resolve communication issues caused by wrongly configured MAC addresses.
Further more learned on how to do a basic port and protocol debugging of state of Network packets with netstat and nc and check problems related to iptables Firewall and IP Forwarding misconfigurations.
Finally we learned some basic usage of tcpdump on how to track arp and MAC traffic and look up for a specific TCP / UDP protocol and its contained data.
There is certainly things this article is missing as the topic of debugging network connectivity issues on Linux is a whole ocean, especially as the complexity of Linux has grown dramatically these days.
I gues it is worthy to mention that unable to see remote network could be caused by wrong VLAN configurations on Linux or even buggy switches and router devices, due to hardware or software,
but I hope this article at least covers the very basics of network debugging and Linux.

Enjojy 🙂

Tags: article, broadcast, data, debugging, default, Display, fails, guide, histogram, ifconfig eth0, initial, internal, issues, link, linux?, loopback, mtr, MULTICAST, multiple times, network, network speed, Old, operating systems, ping, pipe, Present, running, RX, Settings, system administrators, tcp, time, Tx
Posted in Debian, Educational, Linux, Networking, System Administration | No Comments »

☩ Walking in Light with Christ – Faith, Computing, Diary

Posts Tagged ‘data’

How to convert .p12 ssl certificate to .pem with openssl command

Zabbix Power Shell PS1 script to write zero or one if string is matched inside log file

Daily Bible quote

GET ARTICLE UPDATES

Useful blog? Help it:

Recent Posts

Ads

Categories

Recent Comments

blogtopsites

Posts Tagged ‘data’

Prerequisites

Install Kibana

I. On Debian/Ubuntu

Import the Elastic GPG key:

Add the repository:

Update and install:

II. On RHEL/CentOS Linux

Create repo file:

Install Kibana:

2. Configure Kibana

3. Start and Enable Kibana

4. Access Kibana Web Interface

5. Import and Visualize Logs

Option A: Use Filebeat to Send Logs

Option B: Ingest Logs via Logstash or Elasticsearch API

6. Create Index Pattern

7. Create Visualizations and Dashboards

8. Secure Kibana

What is Kibana used for and what it can do for you?

Access Kibana

Visualize Data

Search & Query Logs

Set Alerts (Optional)

Access Kibana

Visualize Data

Create Dashboards

Search & Query Logs

Set Alerts (Optional)

Access Kibana

Connect to Elasticsearch

Visualize Data

Create Dashboards

Search & Query Logs

Set Alerts (Optional)

Summary closing words (what we did)

Why Backups Are Important (Even on Linux)

What Makes a Good GUI Backup Tool?

1. Déjà Dup – The Simplest Backup Tool

Key Features

How to Use Déjà Dup

2. Timeshift – System Snapshots Made Easy

Key Features

When to Use Timeshift

3. Use Timeshift alongside a file backup tool like Déjà Dup as a backup solution for OS and data

a. Set up Timeshift (system snapshots)

What to include

Schedule (typical)

b. Set up Deja Dup (personal backups)

Folders to back up

Schedule

c. Prevent overlap (this matters)

d. Real-world recovery scenarios

Scenario 1: Bad update / system won’t boot

Scenario 2: Deleted or corrupted files

Scenario 3: New machine / fresh install

e. Optional pro tips (to avoid data loss)

4. Back In Time – More Control features tool to create GUI-Based backups on Linux

Key Features

5. Backing Up your Data on Linux with Grsync (rsync GUI frontend backup tool interface)

Best Backup Strategy for Desktop Linux Users

Final Thoughts

What Is Freedoom (and Why Should You Care)?

1. Install Freedoom on Linux

Debian / Ubuntu / Linux Mint

2. Choose a DOOM Engine (a.k.a. Source Port)

2.1. Chocolate Doom (Classic Experience)

2.2. PrBoom+ (Classic but Smoother)

2.3. GZDoom (Modern, Fancy, Wild)

3. Installing Other DOOM Engines

3.2. Where Are the WAD Files located ?

3.3. Keyboard, Mouse, and Sanity Settings

Final Thoughts, DOOM A Forever Classic

1. Identify the resource issue Bottleneck

2. Use tmpfs for Cache and Temporary Files

3. Tune Kernel Parameters

4. Use tmpfs for MySQL / MariaDB tmpdir (Optional)

5. Monitor the Effects and tune up if necessery

Short rephrasal

Intro: Understanding Process Accounting in Linux/Unix Systems

**3. Installing Other DOOM Engines**

**What are /var/account/pacct and /var/account/acct dirs and common locations across *nix OS-es?**