Posts Tagged ‘access’

Digital Vigilance: Practical Cyber Defense for the New Era of All connected dependency on technology

Friday, October 24th, 2025

 

Introduction

There was a time when cybersecurity was mostly about erecting a firewall, installing antivirus software and hoping no one clicked a suspicious link. That era is steadily fading. Today, as more work moves to the cloud, as AI tools proliferate, and as threat actors adopt business-like models, the battlefield has shifted dramatically. According to analysts at Gartner, 2025 brings some of the most significant inflections in cybersecurity in recent memory. 

In this article we’ll cover the major trends, why they matter, and — importantly — what you as an individual or sysadmin can start doing today to stay ahead.

1. Generative AI: Weapon and Shield

AI / ML (Machine Learning)) is now deeply ingrained in both the offence and defence sides of cybersecurity.

  • On the defence side: AI models help detect anomalies, process huge volumes of logs, and automate responses. 
  • On the offence side: Attackers use AI to craft more convincing phishing campaigns, automate vulnerability discovery, generate fake identities or even design malware. 
  • Data types are changing: It’s no longer just databases and spreadsheets. Unstructured data (images, video, text) used by AI models is now a primary risk.

What to do:

  • Make sure any sensitive AI-training data or inference logs are stored securely.
  • Build anomaly-detection systems that don’t assume “normal” traffic anymore.
  • Flag when your organisation uses AI tools: do you know what data the tool uses, where it stores it, who can access it?

2. Zero Trust Isn’t Optional Anymore

 

cyber-security-threats-to-watch-in-2025

The old model — trust everything inside the perimeter, block everything outside — is obsolete. Distributed workforces, cloud services, edge devices: they all blur the perimeter. Hence the rise of Zero Trust Architecture (ZTA) — “never trust, always verify.” INE+1

Key features:

  • Every device, every user, every session must be authenticated and authorised.
  • Least-privilege access: users should have the minimum permissions needed.
  • Micro-segmentation: limit lateral movement in networks.
  • Real-time monitoring and visibility of sessions and devices.

What to do:

  • Audit your devices and users: who has broad permissions? Who accesses critical systems?
  • Implement multifactor authentication (MFA) everywhere you can.
  • Review network segmentation: can a compromised device access everything? If yes, that’s a red flag.
     

3. Ransomware & RaaS – The Business Model of Cybercrime

Cybercriminals are organizing like businesses: they have supply chains, service models, profit centres. The trend of Ransomware‑as‑a‑Service (RaaS) continues to expand. Dataconomy+1

What’s changed:

  • Ransomware doesn’t just encrypt data. Attackers often steal data first, then threaten to release it. 
  • Attackers are picking higher-value targets and critical infrastructure.
  • The attack surface has exploded: IoT devices, cloud mis-configurations, unmanaged identity & access.

What to do:

  • Back up your critical systems regularly — test restores, not just backups.
  • Keep systems patched (though even fully patched systems can be attacked, so patching is necessary but not sufficient).
  • Monitor for abnormal behaviour: large data exfiltration, new admin accounts, sudden access from odd places.
  • Implement strong incident response procedures: when it happens, how do you contain it?

4. Supply Chains, IoT & Machine Identities

Modern IT is no longer just endpoints and servers. We have IoT devices, embedded systems, cloud services, machine-to-machine identities. According to Gartner, machine identities are expanding attack surfaces if unmanaged.

Key issues:

  • Devices (especially IoT) often ship with weak/default credentials.
  • Machine identities: software services, APIs, automation tools need their own identity/access management.
  • Supply chains: your vendor might be the weakest link — compromise of software or hardware upstream affects you.

What to do:

  • Create an inventory of all devices and services — yes all.
  • Enforce device onboarding processes: credentials changed, firmware up-to-date, network segmented.
  • Review your vendors: what security standards do they follow? Do they give you visibility into their supply chain risk?
     

5. Cloud & Data Privacy — New Rules, New Risks

As data moves into the cloud and into AI systems, the regulatory and technical risks converge. For example, new laws like the EU AI Act will start affecting how organisations handle AI usage and data. Source: Gcore
Cloud environments also bring mis-configurations, improper access controls, shadow-IT and uncontrolled data sprawl. techresearchs.com+1

What to do:
 

  • If using cloud services, check settings for major risk zones (e.g., S3 buckets, unsecured APIs).
  • Implement strong Identity & Access Management (IAM) controls for cloud resources.
  • Make data-privacy part of your security plan: what data you collect, where it is stored, for how long.
  • Perform periodic audits and compliance checks especially if you handle users from different jurisdictions.
     

6. Skills, Culture & Burn-out — The Human Factor

Often overlooked: no matter how good your tech is, people and culture matter. Gartner Security behaviour programs help reduce human-error incidents — and they’re becoming more essential.
Also, the cybersecurity talent shortage and burnout among security teams is real.

What to do:

 

  • Invest in security awareness training: phishing simulation, strong password practices, device hygiene.
  • Foster a culture where security is everyone’s responsibility, not just the “IT team’s problem.”
  • For small teams: consider managed security services or cloud-based monitoring to lean on external support.

7. What This Means for Smaller Organisations & Individual Users

Often the big reports focus on enterprises. But smaller organisations (and individual users) are just as vulnerable — sometimes more so, because they have fewer resources and less mature security.
Here are some concrete actions:

  • Use strong, unique passwords and a password manager.
  • Enable MFA everywhere (email, online services, VPNs).
  • Keep your systems updated — OS, applications, firmware.
  • Be suspicious of unexpected communications (phishing).
  • Have an incident response plan: who do you call if things go wrong?
  • Backup your data offline and test restores.
  • If you run services (web-server, mail server): monitor logs, check for new accounts, stray network connections.
     

Conclusion

Cybersecurity in 2025 is not a “set once and forget” system. It’s dynamic, multi-layered and deeply integrated into business functions and personal habits. The trends above — generative AI, zero trust, supply chain risks, cloud data sprawl — are changing the rules of the game.
Thus for all of us and especially sysadmins / system engineers or Site Reliabiltiy Managers (SRE), Developers, Testers or whatever you call it this meen we need to keep learning, be careful with the tech stuff we use, and build security as a continuous practice rather than a one-off box-to-tick.

 

What is oddjobd and How to Use It Instead of sudo to run limited privileged execution of scripts requiring admin

Tuesday, September 30th, 2025

oddjobd-sudoers-linux-elevate-script-running-linux

In Linux environments, managing privileged operations for unprivileged users is a critical task. Traditionally, tools like sudo have been used to allow users to execute specific commands with elevated privileges. However, in more secure or fine-tuned environments — such as enterprise networks or identity-managed systems — oddjobd offers a more controlled, D-Bus-driven alternative.

This article explains what oddjobd is, how it works, and when you might prefer it over sudo, complete with real-world examples.

What is oddjobd?

oddjobd is a system service (daemon) that runs in the background and allows limited, controlled execution of privileged tasks on behalf of unprivileged users.

Key Features:

  • Allows secure execution of predefined scripts or programs as root (or another user).
  • Communicates over D-Bus for fine-grained access control.
  • Uses Polkit (PolicyKit) to manage who can run which tasks.
  • Commonly used in FreeIPA, SSSD, and LDAP-based environments.
  • Configuration files live in: /etc/oddjobd.conf.d/

How It Works

  • System administrators define specific jobs (scripts or commands) in config files.
  • These jobs are exposed via D-Bus.
  • Unprivileged users (or applications) can request jobs to be executed.
  • Access is granted or denied by Polkit rules, not passwords.
  • No full shell or terminal access is granted — just the job.
 

oddjobd vs sudo

Feature

sudo

oddjobd

Control granularity

Medium (commands)

High (methods, scripts only)

Interactive shell

Yes

No

Config complexity

Simple (/etc/sudoers)

Moderate (conf.d + Polkit)

Uses system user password

Yes

Optional (can be passwordless via Polkit)

Security

Medium

High (no shell, strict policy control)

D-Bus compatible

No

Yes

Ideal for

Power users

Controlled environments (e.g., FreeIPA)

Typical Use Cases for oddjobd


1. Automatically Creating Home Directories


Problem: LDAP/FreeIPA users don’t have home directories created on login.

Solution: Enable oddjobd to create them via oddjob-mkhomedir.

# authconfig –enablemkhomedir –update

On login, PAM calls oddjobd, which creates the home directory as root.
 

2.  Restarting a Service without sudo

Let's say you want a user to restart Apache, but not give them full sudo rights.

a. Create a script

# /usr/local/bin/restart_apache.sh

#!/bin/bash

systemctl restart apache2

echo "Apache restarted by oddjob at $(date)"

chmod +x /usr/local/bin/restart_apache.sh

b. Create Oddjob config
 

# /etc/oddjobd.conf.d/restart_apache.conf

[restart_apache]

program = /usr/local/bin/restart_apache.sh

user = root

c. Polkit rule

 

// /etc/polkit-1/rules.d/60-restart-apache.rules

polkit.addRule(function(action, subject) {

    if (action.id == "org.freedesktop.oddjob.restart_apache" &&

        subject.isInGroup("apacheadmins")) {

        return polkit.Result.YES;

    }

});

 

d. Add user to group

# groupadd apacheadmins

# usermod -aG apacheadmins alice


e. Restart and test

# systemctl restart oddjobd


# As user "alice":

oddjob_request restart_apache


Only the defined method runs — no sudo shell access, no arbitrary commands.
 

3. GUI-friendly Device Control


Use Case: A user wants to reset a USB device via a button in a GUI app.

  • Define the method in oddjobd.
  • Use Polkit for GUI D-Bus permission.
  • The app can call the method securely, without sudo.

Advantages of oddjobd

More Secure Than sudo:

  • No interactive shell or terminal.
  • No command-line injection risks.
  • Can’t “escape” to a shell like with sudo bash.

Granular Control:

  • Limit tasks to a specific script or even script arguments.

D-Bus and GUI Friendly:

  • Apps can call privileged methods without shell hacks.

Policy-Based Authorization (Polkit):

  • Fine-grained user/group access control.
  • No password prompts if not desired.

Enterprise-Ready:

  • Works well with LDAP, FreeIPA, and centralized login environments.

Oddjobd Limitations / Downsides

Limitation

Description

Learning Curve

More complex to set up than sudo

Configuration Overhead

Requires writing config files and Polkit rules

Debugging

Issues may be harder to trace than sudo logs

Not for Ad-hoc Commands

Only predefined jobs can be run

Not Installed by Default

Often needs to be manually installed (oddjob, oddjob-mkhomedir)

When to Use oddjobd Instead of sudo

Use oddjobd when you:

  • Need to allow users or apps to run very specific privileged operations.
  • Want to avoid giving full shell access via sudo.
  • Are working in a managed enterprise environment.
  • Need GUI or D-Bus-based privilege escalation.
  • Require scripted access to root tasks without exposing credentials.

Conclusion

oddjobd is a powerful tool for securely handling privileged operations in Linux, especially where tight access control and automation are required. While sudo is simple and flexible, oddjobd shines in structured, security-conscious environments — particularly those using FreeIPA, LDAP, or automated tools.

If you need a more scriptable, policy-driven, and safer alternative to sudo for specific tasks, oddjobd is well worth exploring.

How to Install and Use auditd for System Security Auditing on Linux

Thursday, September 25th, 2025

System auditing is essential for monitoring user activity, detecting unauthorized access, and ensuring compliance with security standards. On Linux, the Audit Daemon (auditd) provides powerful auditing capabilities for logging system events and actions.

This short article will walk you through installing, configuring, and using auditd to monitor your Linux system.

What is auditd?

auditd is the user-space component of the Linux Auditing System. It logs system calls, file access, user activity, and more — offering administrators a clear trail of what’s happening on the system.


1. Installing auditd

The auditd package is available by default in most major Linux distributions.

 On Debian/Ubuntu

# apt update
# apt install auditd audispd-plugins

 On CentOS/RHEL/Fedora

# yum install audit

After installation, start and enable the audit daemon

# systemctl start auditd

# systemctl enable auditd

Check its status

# systemctl status auditd

2. Setting Audit Rules

Once auditd is running, you need to define rules that tell it what to monitor.

Example: Monitor changes to /etc/passwd

# auditctl -w /etc/passwd -p rwxa -k passwd_monitor

Explanation:

  • -w /etc/passwd: Watch this file. When the file is accessed, the watcher will generate events.
  • -p rwxa: Monitor read, write, execute, and attribute changes
  • -k passwd_monitor: Assign a custom key name to identify logs. Later on, we could search for this (arbitrary) passwd string to identify events tagged with this key.

List active rules:

# auditctl -l

3. Common auditd Rules for Security Monitoring

Here are some common and useful auditd rules you can use to monitor system activity and enhance Linux system security. These rules are typically added to the /etc/audit/rules.d/audit.rules or /etc/audit/audit.rules file, depending on your system.

a. Monitor Access to /etc/passwd and /etc/shadow
 

-w /etc/passwd -p wa -k passwd_changes
-w /etc/shadow -p wa -k shadow_changes

  • Monitors read/write/attribute changes to password files.

b. Monitor sudoers file and directory
 

-w /etc/sudoers -p wa -k sudoers
-w /etc/sudoers.d/ -p wa -k sudoers

  • Tracks any change to sudo configuration files.

c. Monitor Use of chmod, chown, and passwd
 

-a always,exit -F arch=b64 -S chmod -S fchmod -S fchmodat -k perm_mod
-a always,exit -F arch=b64 -S chown -S fchown -S fchownat -k perm_mod
-a always,exit -F arch=b64 -S passwd -k passwd_changes

  • Watches permission and ownership changes.

d. Monitor User and Group Modifications

-w /etc/group -p wa -k group_mod
-w /etc/gshadow -p wa -k gshadow_mod
-w /etc/security/opasswd -p wa -k opasswd_mod

  • Catches user/group-related config changes.

e. Track Logins, Logouts, and Session Initiation

-w /var/log/lastlog -p wa -k logins
-w /var/run/faillock/ -p wa -k failed_login
-w /var/log/faillog -p wa -k faillog

  • Tracks login attempts and failures.

f. Monitor auditd Configuration Changes

-w /etc/audit/ -p wa -k auditconfig
-w /etc/audit/audit.rules -p wa -k auditrules

  • Watches changes to auditd configuration and rules.

g. Detect Changes to System Binaries

-w /bin/ -p wa -k bin_changes
-w /sbin/ -p wa -k sbin_changes
-w /usr/bin/ -p wa -k usr_bin_changes
-w /usr/sbin/ -p wa -k usr_sbin_changes

  • Ensures core binaries aren't tampered with.

h. Track Kernel Module Loading and Unloading

-a always,exit -F arch=b64 -S init_module -S delete_module -k kernel_mod

  • Detects dynamic kernel-level changes.

l. Monitor File Deletions

-a always,exit -F arch=b64 -S unlink -S unlinkat -S rename -S renameat -k delete

  • Tracks when files are removed or renamed.

m. Track Privilege Escalation via setuid/setgid

-a always,exit -F arch=b64 -S setuid -S setgid -k priv_esc

  • Helps detect changes in user or group privileges.

n. Track Usage of Dangerous Binaries (e.g., su, sudo, netcat)

-w /usr/bin/su -p x -k su_usage
-w /usr/bin/sudo -p x -k sudo_usage
-w /bin/nc -p x -k netcat_usage

  • Useful for catching potentially malicious command usage.

o. Monitor Cron Jobs

-w /etc/cron.allow -p wa -k cron_allow
-w /etc/cron.deny -p wa -k cron_deny
-w /etc/cron.d/ -p wa -k cron_d
-w /etc/crontab -p wa -k crontab
-w /var/spool/cron/ -p wa -k user_crontabs

  • Alerts on cron job creation/modification.

p. Track Changes to /etc/hosts and DNS Settings

-w /etc/hosts -p wa -k etc_hosts
-w /etc/resolv.conf -p wa -k resolv_conf

  • Monitors potential redirection or DNS manipulation.

q. Monitor Mounting and Unmounting of Filesystems

-a always,exit -F arch=b64 -S mount -S umount2 -k mounts

  • Useful for detecting USB or external drive activity.

r. Track Execution of New Programs

-a always,exit -F arch=b64 -S execve -k exec

  • Captures command execution (can generate a lot of logs).
     

A complete list of rules you can get from the hardening.rules auditd file place it under /etc/audit/rules.d/hardening.rules
and reload auditd to load the configurations.

Tips

  • Use ausearch -k <key> to search audit logs for matching rule.
  • Use auditctl -l to list active rules.
  • Use augenrules –load after editing rules in /etc/audit/rules.d/.


4. Reading Audit Logs

Audit logs events are stored in:

/var/log/audit/audit.log

By default, the location, this can be changed through /etc/auditd/auditd.conf

View recent entries:
 

# tail -f /var/log/audit/audit.log

Search by key:
 

# ausearch -k passwd_monitor

Generate a summary report:

# aureport -f

# aureport


Example: Show all user logins / IPs :

# aureport -au

 

5. Making Audit Rules Persistent

Rules added with auditctl are not persistent and will be lost on reboot. To make them permanent:

Edit the audit rules configuration:

# vim /etc/audit/rules.d/audit.rules

Add your rules, for example:

-w /etc/passwd -p rwxa -k passwd_monitor

Apply the rules:

# augenrules –load

7. Some use case examples of auditd in auditing Linux servers by sysadmins / security experts
 

Below are real-world, practical examples where auditd is actively used by sysadmins, security teams, or compliance officers to detect suspicious activity, meet compliance requirements, or conduct forensic investigations.

a. Detect Unauthorized Access to /etc/shadow

Use Case: Someone tries to read or modify password hashes.

Audit Rule:

-w /etc/shadow -p wa -k shadow_watch

Real-World Trigger:

sudo cat /etc/shadow

Check Logs:
 

# ausearch -k shadow_watch -i

Real Output:
 

type=SYSCALL msg=audit(09/18/2025 14:02:45.123:1078):

  syscall=openat

  exe="/usr/bin/cat"

  success=yes

  path="/etc/shadow"

  key="shadow_watch"

b. Detect Use of chmod to Make Files Executable

Use Case: Attacker tries to make a script executable (e.g., malware).

Audit Rule:

-a always,exit -F arch=b64 -S chmod -k chmod_detect

Real-World Trigger:
 

 # chmod +x /tmp/evil_script.sh

Check Logs:

# ausearch -k chmod_detect -i

c. Monitor Execution of nc (Netcat)

Use Case: Netcat is often used for reverse shells or unauthorized network comms.

Audit Rule:
 

-w /bin/nc -p x -k netcat_usage
 

Real-World Trigger:

nc -lvp 4444

Log Entry:

type=EXECVE msg=audit(09/18/2025 14:35:45.456:1123):

  argc=3 a0="nc" a1="-lvp" a2="4444"

  key="netcat_usage"

 

d. Alert on Kernel Module Insertion
 

Use Case: Attacker loads rootkit or malicious kernel module.

Audit Rule:

-a always,exit -F arch=b64 -S init_module -S delete_module -k kernel_mod

Real-World Trigger:

# insmod myrootkit.ko

Audit Log:
 

type=SYSCALL msg=audit(09/18/2025 15:00:13.100:1155):

  syscall=init_module

  exe="/sbin/insmod"

  key="kernel_mod"

e. Watch for Unexpected sudo Usage

Use Case: Unusual use of sudo might indicate privilege escalation.

Audit Rule:

-w /usr/bin/sudo -p x -k sudo_watch

Real-World Trigger:

sudo whoami

View Log:
 

# ausearch -k sudo_watch -i


f. Monitor Cron Job Modification

Use Case: Attacker schedules persistence via cron.

Audit Rule:

-w /etc/crontab -p wa -k cron_mod

Real-World Trigger:
 

echo "@reboot /tmp/backdoor" >> /etc/crontab

Logs:
 

type=SYSCALL msg=audit(09/18/2025 15:05:45.789:1188):

  syscall=open

  path="/etc/crontab"

  key="cron_mod"

g. Detect File Deletion or Renaming
 

Use Case: Attacker removes logs or evidence.

Audit Rule:

-a always,exit -F arch=b64 -S unlink -S unlinkat -S rename -S renameat -k file_delete

Real-World Trigger:

# rm -f /var/log/syslog

Logs:
 

type=SYSCALL msg=audit(09/18/2025 15:10:33.987:1210):

  syscall=unlink

  path="/var/log/syslog"

  key="file_delete"


h. Detect Script or Malware Execution
 

Use Case: Capture any executed command.

Audit Rule:
 

-a always,exit -F arch=b64 -S execve -k exec

Real-World Trigger:

/tmp/myscript.sh

Log View:

# ausearch -k exec -i | grep /tmp/myscript.sh

l. Detect Manual Changes to /etc/hosts

Use Case: DNS hijacking or phishing setup.

Audit Rule:

-w /etc/hosts -p wa -k etc_hosts

Real-World Trigger:
 

# echo "1.2.3.4 google.com" >> /etc/hosts

Logs:

type=SYSCALL msg=audit(09/18/2025 15:20:11.444:1234):

  path="/etc/hosts"

  syscall=open

  key="etc_hosts"


8. Enable Immutable Mode (if necessery)

For enhanced security, you can make audit rules immutable, preventing any changes until reboot:

# auditctl -e 2


To make this setting persistent, add the following to the end of /etc/audit/rules.d/audit.rules:

-e 2


Common Use Cases

Here are a few more examples of what you can monitor:

Monitor all sudo usage:

# auditctl -w /var/log/auth.log -p wa -k sudo_monitor


Monitor a directory for file access:

# auditctl -w /home/username/important_dir -p rwxa -k dir_watch

Audit execution of a specific command (e.g., rm):

# auditctl -a always,exit -F arch=b64 -S unlink,unlinkat -k delete_cmd

(Adjust arch=b64 to arch=b32 if on 32-bit system.)

9. Managing the Audit Log Size

Audit logs can grow large over time. To manage log rotation and size, edit:
 

# vim /etc/audit/auditd.conf

Set log rotation options like:

max_log_file = 8

num_logs = 5

Then restart auditd:
 

# systemctl restart auditd

Conclusion

The Linux Audit Daemon (auditd) is a powerful tool to track system activity, enhance security, and meet compliance requirements. With just a few configuration steps, you can monitor critical files, user actions, and system behavior in real time.

 

References

  • man auditd
  • man auditctl
  • Linux Audit Wiki

 

Unlocking the Power of lnav: Logfile Navigator – ncurses text based tool guide to mutiple Logs on multiple servers easy analysis on Linux

Saturday, September 13th, 2025

lnav-syslog-screenshot-linux-virtual-machine

If you've ever found yourself buried under a mountain of log files, tailing multiple outputs, or grepping through endless lines trying to spot an error, it's time to meet your new best friend: lnav, the Logfile Navigator.

Lightweight, terminal-based, and surprisingly powerful, lnav is one of the most underrated tools for developers, sysadmins, and anyone who regularly digs into logs. It turns your chaotic logs into something that’s not only readable—but genuinely useful.

What is lnav and why use it ?

lnav (Logfile Navigator) is a command-line tool for viewing and analyzing log files. It goes beyond tail, less, or grep by:

  • Automatically detecting and merging log formats.
  • Highlighting timestamps, log levels, and errors.
  • Providing SQL-like queries over your logs.
  • Offering interactive navigation with a UI inside the terminal.

And yes, all of that without needing to set up a database or a server.

1. Installing lnav on Linux

Installation is straightforward. On most systems, you can install it via package managers:

On Ubuntu/Debian:

# apt install lnav

On Fedora:

# dnf install lnav

On Arch Linux:

# pacman -S lnav

Or build from source via GitHub if you want the latest version.

2. Use lnav Instead of Tail / Grep why?

Traditional tools are powerful, but they require manual work to chain together functionality. lnav gives you:

  • Automatic multi-log parsing: Drop multiple logs in, and it merges them chronologically.
  • Syntax highlighting: Errors and warnings stand out.
  • SQL querying: Run queries like SELECT * FROM syslog_log WHERE log_level = 'error';
  • Filtering and searching: Use intuitive filters and bookmarks to highlight specific entries.

3. Basic tool Usage is simple

Let’s say you want to inspect a system log:

# lnav /var/log/syslog

You'll immediately get:

  • Color-coded output (timestamps, levels, messages).
  • Scrollable view (arrow keys, PgUp, PgDn).
  • Real-time updates (like tail -f).
  • Search with /, filter with :filter-in, and even SQL queries.

Lets say you need to analyze Apache webserver logs recursively including the logs already rotated and gunzipped with *.gz extension on CentOS / Fedora / RHEL, you can do it with:

# lnav -r /var/log/httpd

You can parse the log file and get additional information about requests as well as you can print overall summary of log file.

Choose the line you want to parse. The selected line is always the one at the top of the window. Then press 'p' and you should see the following result:

https://pc-freak.net/images/lnav-get-extra-information-about-apache-query-with-P-press-key-screenshot-linux

Now, if you want to see a summary view of the logs by date and time, simply press 'i'.

lnav-linux-apache-log-review-summary-of-errors-warnings-normal-screenshot

To quit a screen you have chosen press 'q'.

4. LNAV helpful options and hotkeys

Once you've opened a log file/s for analyze you can use few hotkeys that will allow us to move through the output of lnav and the available views more easily:

e or E to jump to the next / previous error message.
w or W to jump to the next / previous warning message.
b or Backspace to move to the previous page.
Space to move to the next page.
g or G to move to the top / bottom of the current view.

To take a closer look at the way lnav operates, use -d option, the debug information is to be spit inside a .txt file:

# lnav /var/log/httpd -d lnav.txt

In this example, the debug information that is generated when lnav starts will be written to a file named lnav.txt inside the current working directory.

5. Real-World Use Cases

a. Troubleshooting application or system process Crashes

Open all relevant logs in one go:

# lnav /var/log/*.log

Errors are highlighted, and you can jump between them with n / N kbd keys.

b. Combining Multiple Logs

Working with an app that logs to different files and you need to combine:

# lnav /var/log/nginx/access.log /var/log/nginx/error.log


Or lets say you want to combine Apache Webserver with Haproxy log and get log summaries or filter out stuff:

lnav /var/log/apache2/access.log /var/log/haproxy.log


Now you will get a single, chronological timeline of events.

 

If you want to Search for a concrete occurance of Error / Warning or IP address inside a bunch of loaded combined logs you can do it with the same command like in simple vim by pressing / (slash) from kbd and type out what you want to filter out to get shown.

c. Analyze SQL Queries Logs

Yes, you can actually do this by passing it query in its command prompt :

:.schema
:SELECT log_time, log_level, log_message FROM syslog_log WHERE log_level = 'error';

You get a table of filtered logs, sortable by columns.
 

6. lnav more usage command tips

  • :help — Opens the help menu.
  • :filter-in <string> — Show only lines matching <string>.
  • :filter-out <string> — Hide lines matching <string>.
  • :export-to <filename> — Export current view to a file.
  • :tag <tagname> — Tag lines for later reference.
  • q — Quit (but why would you want to?).

 

7. Using lnav as a pager for systemd-journald

journalctl | lnav
# journalctl -f | lnav
# journalctl -u ssh.service | lnav

https://pc-freak.net/images/lnav_sshservice-log-view-screenshot-linux
 

8. Use lnav to review remote ssh logs

Newer versions after 0.10 supports ssh protocol as well and theoretically should work:

# lnav user@server-name-here:/var/log/file.log


To read all logs inside /var/log

# lnav root@server-name-here:/var/log/
# lnav root@server-name-here:/var/log/*.err

9. Using lnav to view docker container logs

# docker logs 811ab84aa95l | lnav
# docker logs -f application | lnav

The latest version of lnav supports even the following  simplified docker:// URL syntax:

# lnav docker://{container_id_or_name}/dir_path/to/log/file
# lnav docker://{container_id_or_name}/var/dir_path/log
# lnav docker://application/var/log/
# lnav docker://applcation/var/log/nginx/nginx.app.log

10. Monitoring compilation and command output useful for developers
 

Compilation from archived tar balls with ./configure && make etc. generate lot of outputs and logs while working. 
Here is where the tool can come handy. 
For example, here is how to watch the output of make command when compiling something:

# lnav -e './configure && make'

 11. Learning lnav tool through online ssh service availability via lnav.org

f you're lazy to install it and want to test it anyways:
 

# Start The Basic Tutorial:
ssh -o PubkeyAuthentication=no -o PreferredAuthentications=password tutorial1@demo.lnav.org


# Playground:
ssh -o PubkeyAuthentication=no -o PreferredAuthentications=password playground@demo.lnav.org


Closure

While tools like Kibana, Grafana, and ELK stacks are powerful, they can be overkill for many use cases—especially when you're SSHed into a box and just need to get answers fast. That’s where lnav shines as it is fast, lightweight, visual and can be used offline.

If you’re a developer, sysadmin, SRE (Site Reliability Engineer), or just someone who cares about logs, give lnav a spin. It might just become among your favorite sysadm tools on Linux and safe you pretty much of time if you have to do log reading and analyzing on daily basis (for example if you're admining 20+ or more Linux servers.

 

Debugging Jitsi Meet Server Problems: A Practical Guide

Saturday, April 26th, 2025

Jitsi Meet is a powerful open-source video conferencing platform. But like any real-time communication system, it can run into issues—from video/audio glitches to full-blown connection failures. Debugging Jitsi Meet can be tricky due to its multi-component architecture. This guide walks you through a systematic approach to identify and resolve common server-side issues.

1. Understand the Architecture

Before diving into logs, it's important to understand Jitsi Meet's core components:
 

  • Jitsi Meet (Web UI) – The front-end interface.
  • Jicofo (Focus component) – Manages conference sessions.
  • Prosody (XMPP Server) – Handles user authentication and signaling.
  • JVB (Jitsi Videobridge) – Routes video/audio streams.
  • Nginx or Apache – Web server proxy (often with HTTPS and WebSocket forwarding).


Knowing how these interact helps pinpoint the failing layer.

2. Check Logs in the Right Places

Each component has its own logs. Check them in the following order:

Prosody Logs

Location: /var/log/prosody/prosody.log and prosody.err
​Look for: Authentication issues, connection denials, or component registration problems.
 

Jicofo Logs

Location: /var/log/jitsi/jicofo.log
Look for: Room creation errors, XMPP connection failures, conference creation attempts.
 

JVB Logs

Location: /var/log/jitsi/jvb.log
​Look for: ICE failures, STUN/TURN issues, packet loss, and bridge reachability.
 

Web Server Logs (Nginx/Apache)

Location (Nginx): /var/log/nginx/error.log and access.log
Look for: HTTP errors (404, 502), WebSocket connection problems.
 

Browser Console Logs
 

Tools: Press F12 in browser → Console/Network tabs.
Look for: WebSocket failures, CORS issues, or media permission problems.
 

3. Common Problems & Fixes

"Failed to join conference"

  • Cause: Prosody may not be running or not configured correctly.​

Fix: Restart Prosody and check domain configuration in /etc/prosody/conf.avail/

 

 

No Audio or Video
 

Usual Cause: Media not reaching the bridge or blocked by firewall

Fix:

  • Verify JVB is listening on correct ports (UDP 10000).
  • ​Check firewall/NAT settings (especially on cloud VMs).
  • Use tcpdump or ss to check traffic flow.
     

WebSocket Connection Fails

 

Usual Cause: Web server (Proxy) misconfiguration.

Fix:

Ensure Nginx is forwarding WebSocket requests to /xmpp-websocket/.
Add proper proxy settings in nginx.conf
 

Authentication Not Working


Cause: Misconfigured JWT or internal authentication.

Fix:

  • Check Prosody's config for authentication method.
  • If using JWT, verify token structure and shared secret.
     

4. Use Debugging Tools

  • Jitsi Meet in debug mode:


​Add #config.debug=true to your meeting URL.
 

  • ICE Debugging:

     

     

     

    Check about:webrtc (Firefox) or WebRTC Internals (Chrome).
    Look at ICE candidate gathering and connectivity checks.
    Test TURN/STUN:

    • Use tools like trickle-ice to validate your server's ICE configuration.

5. Networking and Firewall Checks

Make sure these ports are open:
 

  • TCP 443 – HTTPS
  • UDP 10000 – Media (JVB)
  • TCP 4443 – (Optional, fallback media)
  • TCP 5222 – XMPP (if not using BOSH/WebSocket)
     

# ss -tuln ufw status


6. Component Health Checks

Do 
# systemctl status for each main jitsi component services:

# systemctl status prosody
# systemctl status jicofo
# systemctl status jitsi-videobridge2

Check uptime, errors, or failure restarts.

7. Enable More Verbose Logs

Increase logging levels for deeper debugging:
 

  • Prosody: Edit /etc/prosody/prosody.cfg.lua → set log = { ... debug = "*" }.
  • Jicofo/JVB: Edit /etc/jitsi/jicofo/logging.properties and /etc/jitsi/videobridge/logging.properties
    → change log level to FINE or ALL.

 

8. Update & Restart Services

Sometimes updates or configs don’t apply until services are restarted:
 

# apt update && apt upgrade systemctl restart prosody jicofo jitsi-videobridge2 nginx

 

Final Closure Thoughts

Debugging Jitsi Meet requires a structured approach, start from the user-facing symptoms, trace through each service, and verify network and authentication configurations.
Debug the status of prosody, jicofo and jitsi-videobridge2, check the firewall openings are okay to the jitsi server
With some log analysis and a bit of patience, experimentation and the help of forums or Artificial Intelligence tool like ChatGPT, the Jitsi server errors will get solved.

How to Install Jitsi Meet on Debian Linux to have your own free software video conferencing secure server

Thursday, April 24th, 2025

 

jitsi-meet-create-new-room-for-video-meetings-linux

 

Jitsi Meet is a free, open-source video conferencing platform that allows you to host secure and scalable video calls both using a Mobile Phone / Tablet / PC or any other electronic device for which jitsi client has available port. Jitsi meet is the best free alternative one can get to Rakuten Viber / Facebook (Meta) / Zoom / Apples' Facetime etc.
What makes Jitsi really worthy is it can make your Video streaming communication give you flexibility to keep your communication a little bit private and harder to be captured than if you use the general Video streaming platforms. 
Jitsi is also a very simple to use and can be used either with a Desktop Client on Windows / Linux and Mac OS or Smart Phone running Android (Samsung / Huawei etc.) or iOS (iPhones) you can configure to use the Jitsi server or directly via a SSL encryption secured web URL address. The only thing i really don't like about Jitsi is it uses Java and its way of work is cryptic just like it is pretty hard to debug or understand exactly how the software works, as when errors came the usual crazzy Java exceptions are filling the jitsi logs.

In below short guide, I'll try provides a simple step-by-step instructions for installing Jitsi Meet on a Debian-based systems, hoping that anyone can benefit from Jitsi by building his own server.

 

jitsi-meet-conference-free-open-source-video-streaming-viber-and-facebook-alternative


What you should have before you start buillding your new Jitsi meet server

Before you begin, ensure that your system meets the following requirements:

  • A fresh installation of Debian 10 (Buster) or newer.

  • A non-root user with sudo privileges.

  • A fully updated system.

  • A domain name pointing to your server's IP address.

  • OpenJDK 11 installed.​

To get a better understanding on how Jitsi meet works it is worthy to take a quick look on Jitsi Architectural diagram:

Jitsi-meet-video-conferencing-software-linux-windows-mac-Architectural-diagram
 

1. Update Your System

Start by updating your system's package list and upgrading existing packages:​

# apt update sudo apt upgrade -y

2. Install Required Dependencies

Install the necessary packages for adding repositories and managing keys:​

# apt install apt-transport-https curl gnupg2 -y

3 Add Jitsi Repository

Add the Jitsi repository key to your system:

# curl https://download.jitsi.org/jitsi-key.gpg.key | sudo gpg --dearmor -o /usr/share/keyrings/jitsi-keyring.gpg

Then, add the Jitsi repository:​

# echo "deb [signed-by=/usr/share/keyrings/jitsi-keyring.gpg] https://download.jitsi.org stable/" | sudo tee /etc/apt/sources.list.d/jitsi-stable.list > /dev/null

Update your package list to include the Jitsi repository into apt database:​

 # apt update

4. Install Jitsi Meet

Install the Jitsi Meet package:​  

# apt install jitsi-meet -y

During installation, you'll be prompted to:​

  • Enter the hostname: Provide your domain name (e.g., meet.example.com).

  • Choose SSL certificate: Select "Generate a new self-signed certificate" or "Obtain a Let's Encrypt certificate" if you have a valid domain.​JitsiScaleway

If you opt for Let's Encrypt, ensure that ports 80 and 443 are open on your firewall.​

5. Configure Firewall openings

If you have already a configured firewall to filter out traffic, open the necessary ports to allow traffic to your Jitsi Meet server from your router or entry firewall device as well as on the Linux itself: ​

Allow access to SSH server

# ufw allow 22/tcp


Allow access to HTTP unecrypted to Jitsi meet server

# ufw allow 80/tcp
# ufw allow 443/tcp


Allow access necessery for proper operation of Jitsi VideoBridge (port range 10000 to 20000)
 

# ufw allow 10000:20000/udp
# ufw enable

 

Verify the firewall status is Okay​ 

# ufw status

6. Access Jitsi Meet in a browser

Open a web browser and navigate to your server's domain or IP address:​

https://meet.your-custom-domain-or-IP.com

Hopefully all is okay and You should see the Jitsi Meet interface, where you can start or join a meeting.​

7. Secure Conference Creation (Optional extra)

By default, anyone can create a conference. To restrict this:​

  1. Install and configure Prosody for authentication.
    For those who don't know Prosody is a modern XMPP communication server

  2. Set up secure domains and configure authentication settings.​

For detailed instructions, refer to the Jitsi DevOps Guide. ​
 

Conclusion

Now You should have successfully installed Jitsi Meet on your Debian server.
Installing to Ubuntu and Redhat OSes such as Fedora or Redhat based distros should be not much difrerent from on this guide, except you have to use
the correct RPM repositories.

Now you can further now host secure video conferences using your own infrastructure and have an increased privacy and perhaps be more calm that the CIA or Mussat, MI6 / FSB might be not spying your Video conference talks (except if they don't already do it on an OS level which most likely the case but this doesn't matter. :).

For advanced configurations and features, consult the Jitsi Handbook and the Jitsi DevOps Guide.​Jitsi

That's all folks Enjoy !

How to Install and Set Up an NFS Server network Shares on on Linux to easify data transfer across multiple hosts

Monday, April 7th, 2025

How to Configure NFS Server in Redhat,CentOS,RHEL,Debian,Ubuntu and Oracle Linux

Network File System (NFS) is a protocol that allows one system to share directories and files with others over a network. It's commonly used in Linux environments for file sharing between systems. In this guide, we'll walk you through the steps to install and set up an NFS server on a Linux system.

Prerequisites

Before you start, make sure you have:

  • A Linux system distros (e.g., Ubuntu, CentOS, Debian, etc.)
  • Root or sudo privileges on the system.
  • A network connection between the server (NFS server) and clients (machines that will access the shared directories).
     

1. Install NFS Server Package

 

On Ubuntu / Debian based Linux systems:

a. First, update the package list 

# apt update

b. Install the NFS server package
 

# apt install nfs-kernel-server

On CentOS/REL-based systems:

 2. Install the NFS server package
 

      # yum install nfs-utils 

Once the package is installed, ensure that the necessary services are enabled.

 3. Create Shared Directory for file sharing

Decide which directory you want to share over NFS. If the directory doesn't exist, you can create one. For example:

# mkdir -p /nfs_srv_dir/nfs_share

Make sure the directory has the appropriate permissions so that the nfs clients can access it.

# chown nobody:nogroup /nfs_srv_dir/nfs_share 
# chmod 755 /nfs_srv_dir/nfs_share

4. Configure NFS Exports ( /etc/exports file)

The NFS exports file (/etc/exports) is perhaps most important file you will have to create and deal with regularly to define the expored shares, this file contains the configuration settings for directories you want to share with other systems.

       a. Open the /etc/exports file for editing:

vi /etc/exports

Add an entry for the directory you want to share. For example, if you're sharing /nfs_srv_dir/nfs_share and allowing access to all systems on the network (192.168.1.0/24), add the following line:
 

/nfs_srv_dir/nfs_share 192.168.1.0/24(rw,sync,no_subtree_check)


Here’s what each option means:

  • rw: Read and write access.
  • sync: Ensures that changes are written to disk before responding to the client.

 

Here is few lines of  example of my working /etc/exports on my home running NFS server

/var/www 192.168.0.209/32(rw,no_root_squash,async,subtree_check)
/home/jordan 192.168.0.209/32(rw,no_root_squash,async,subtree_check)
/mnt/sda1/icons-frescoes/ 192.168.0.209/32(rw,no_root_squash,async,subtree_check)
/home/mobfiles 192.168.0.209/32(rw,no_root_squash,async,subtree_check)
/mnt/sda1/icons-frescoes/ 192.168.0.200/32(rw,no_root_squash,async,subtree_check)
/home/hipo/public_html 192.168.0.209/32(rw,no_root_squash,async,subtree_check)
/home/alex/public_html 192.168.0.209/32(rw,no_root_squash,async,subtree_check)
/home/necroleak/public_html 192.168.0.209/32(rw,no_root_squash,async,subtree_check)
/bashscripts 192.168.0.209/32(rw,no_root_squash,async,subtree_check)
/backups/Family-Videos 192.168.0.200/32(ro,no_root_squash,async,subtree_check)

 

5. Export the NFS Shares with exportfs command

Once the export file is configured, you need to inform the NFS server to start sharing the directory:
 

# exportfs -a


The -a flag will make it export all the sharings.

6. Start and Enable NFS Services

You need to start and enable the NFS server so it will run on system boot.

On Ubuntu / Debian Linux run the following commands:
 

# systemctl start nfs-kernel-server 
# systemctl enable nfs-kernel-server


On CentOS / RHEL Linux:
 

# systemctl start nfs-server
# systemctl enable nfs-server


7. Allow NFS Traffic Through the Firewall

If your server has a firewall configured / enabled, you will need to allow NFS-related ports through the firewall.
These ports include 2049 TCP protocol Ports (NFS) and 111 (RPCbind) UDP and TCP protocol , and some additional ports.

On Ubuntu/Debian (assuming you are using ufw [UNCOMPLICATED FIREWALL]):

# ufw allow from 192.168.1.0/24 to any port nfs sudo ufw reload

On CentOS / RHEL Linux:

# firewall-cmd –permanent –add-service=nfs sudo firewall-cmd –permanent –add-service=mountd sudo firewall-cmd –permanent –add-service=rpc-bind sudo firewall-cmd –reload

8. Verify NFS Server is Running

To ensure the NFS server is running properly, use the following command:
 

# systemctl status nfs-kernel-server

or

# systemctl status nfs-server

You should see output indicating that the service is active and running.

 

9. Test the NFS Share (Client-Side)

To test the NFS share, you will need to mount it on a client machine. Here's how to mount it:

On the client machine, install the NFS client utilities:

Ubuntu / Debian Linux

# apt install nfs-common

For CentOS / RHEL Linux

# yum install nfs-utils


Create a mount point (Nomatter the distro),:
 

# mkdir -p /mnt/nfs_share


Mount the NFS share:

# mount -t nfs <nfs_server_ip>:/nfs_srv_dir/nfs_share /mnt/nfs_share

Replace <nfs_server_ip> with the IP address of the NFS server or DNS host alias if you have one defined in /etc/hosts file.

Verify that the share is mounted:

​# df -h

You should see the NFS share listed under the mounted file systems.

10. Configure Auto-Mount at Boot (Optional)

To have the NFS share automatically mounted at boot, you can add an entry to the /etc/fstab file on the client machine.

Open /etc/fstab for editing:

# vi /etc/fstab

Add the following line: 

<server-ip>:/nfs_srv_dir/nfs_share /mnt/nfs_share nfs defaults 0 0

Save and close the file.

The NFS share will now be automatically mounted whenever the system reboots.

Debug NFS configuration issues (basics)

 

You can continue to modify the /etc/exports file to share more directories or set specific access restrictions depending on your needs.

If you encounter any issues, checking the server logs or using
 

# exportfs -v
/var/www          192.168.0.209/32(async,wdelay,hide,sec=sys,rw,secure,no_root_squash,no_all_squash)
/home/var_data      192.168.0.205/32(async,wdelay,hide,sec=sys,rw,secure,no_root_squash,no_all_squash)
/mnt/sda1/
        192.168.0.209/32(async,wdelay,hide,sec=sys,rw,secure,no_root_squash,no_all_squash)
/mnt/sda2/info
        192.168.0.200/32(async,wdelay,hide,sec=sys,rw,secure,no_root_squash,no_all_squash)
/home/mobfiles    192.168.0.209/32(async,wdelay,hide,sec=sys,rw,secure,no_root_squash,no_all_squash)
/home/var_data/public_html
        192.168.0.209/32(async,wdelay,hide,sec=sys,rw,secure,no_root_squash,no_all_squash)
/var/public
        192.168.0.209/32(async,wdelay,hide,sec=sys,rw,secure,no_root_squash,no_all_squash)
/neon/data
        192.168.0.209/32(async,wdelay,hide,sec=sys,rw,secure,no_root_squash,no_all_squash)
/scripts      192.168.0.209/32(async,wdelay,hide,sec=sys,rw,secure,no_root_squash,no_all_squash)
/backups/data-limited
        192.168.0.200/32(async,wdelay,hide,sec=sys,ro,secure,no_root_squash,no_all_squash)
/disk/filetransfer
        192.168.0.200/23(async,wdelay,hide,sec=sys,ro,secure,no_root_squash,no_all_squash)
/public_shared/data
        192.168.0.200/23(async,wdelay,hide,sec=sys,ro,secure,no_root_squash,no_all_squash)


 Of course there is much more to be said on that you can for example, check /var/log/messages /var/log/syslog and other logs that can give you hints about issues, as well as manually try to mount / unmount a NFS stuck share to know more on what is going on, but for a starter that should be enough.

command can help severely in troubleshooting the NFS configuration.

Sum it up what learned ?

We learned how to  set up basic NFS server and mounted its shared directory on a client machine.
This is a great solution for centralized file sharing and collaboration on Linux systems (even though many companies are trying to not use it due to its lack of connection encryption for historical reasons NFS has been widely used over the years and has helped dramatically for the Internet as we know it to become the World Wide Web of today. Thus for a well secured network and perhaps not a critical files infrastructure, still NFS is a key player in file sharing among heterogenous networks for multitudes of Gigabytes or Terra Pentabytes of data you would like to share amoung your Personal Computers / Servers / Phones / Tablets and generally all kind of digital computer equipment devices.

How to install and use WSL 2 Windows native Linux emulation Debian and Ubuntu Linux on Windows 10 / Windows 11

Thursday, October 31st, 2024

start-with-wsl-windows-emulation-linux-install-and-use-easily-linux-and-windows-together-with-no-external-software-tux-penguin-logo

WSL (Windows Subsystem for Linux) is perhaps relatively rarely known to the old school sys admins rats who usually use stuff like QEMU / KVM for Windows or Virtualbox / VMWare for Host machine.
However most people most lileky heard but never used or heard about the native (container like) virtualization WSL which was introduced in Windows 10 and Windows 11  as an attempt from Microsoft to improve the interoperability between Windows and Linux.
WSL version 1 and ver 2 allows Microsoft Windows for using a Linux environment without the need for a separate virtual machine.

In Windows 10, it is existing in Windows 10 Professional version can be installed either by joining the Windows Insider program or manually via Microsoft Store or Winget.
Hence perhaps you don't know that WSL virtualization can be used by those who want to mix Linux and Windows or for example get an advantages against dual-boot (installing Linux and Windows on the same computer).
Even better most significant WSL pros is you can literally running both systems at the same time without the need to run or stop every software that’s running and reboot to another system.

Procedure to set up a WSL is simple and similar to setting up a real Linux OS, therefore this guide can also be used as a reference to Linux setup.The specifications of WSL setup procedure are mainly in Install WSL and then setup any packages you would like to use for example if you want to be able to access remotely the WSL emulated Debian / Ubuntu or other of the installable distros via OpenSSH server.

1. Requirements to install and use WSL Linux emulation

To have the wsl subsystem used on Windows 10 or Windows 11 requirements:

You must be running Windows 10 version 2004 and higher (Build 19041 and higher) or Windows 11 to use the commands below. If you are on earlier versions please see the manual install page.

2. List available installable Linux distributions
 

WSL subsystem has ported only a certain set of Linux distributions, so if you need a very specific and unique Linux distribution, you would perhaps need to use Hyper-V virtualization or Virtualbox / VMWare.
However for people like me who are mainly using Debian GNU / Linux on daily basis as well as some OracleLinux admins / SUSE it is a perfect solution.

PS C:\Windows\System32\WindowsPowerShell\v1.0> wsl –list –online
The following is a list of valid distributions that can be installed.
Install using 'wsl.exe –install <Distro>'.

NAME                            FRIENDLY NAME
Ubuntu                          Ubuntu
Debian                          Debian GNU/Linux
kali-linux                      Kali Linux Rolling
Ubuntu-18.04                    Ubuntu 18.04 LTS
Ubuntu-20.04                    Ubuntu 20.04 LTS
Ubuntu-22.04                    Ubuntu 22.04 LTS
Ubuntu-24.04                    Ubuntu 24.04 LTS
OracleLinux_7_9                 Oracle Linux 7.9
OracleLinux_8_7                 Oracle Linux 8.7
OracleLinux_9_1                 Oracle Linux 9.1
openSUSE-Leap-15.6              openSUSE Leap 15.6
SUSE-Linux-Enterprise-15-SP5    SUSE Linux Enterprise 15 SP5
SUSE-Linux-Enterprise-15-SP6    SUSE Linux Enterprise 15 SP6
openSUSE-Tumbleweed             openSUSE Tumbleweed


 

3. Install Linux distribution for a first time

PS C:\Windows\System32\WindowsPowerShell\v1.0> wsl –install

wsl2-windows-virtualization-install-virtual-machine-debian1

The default Linux distribution that will get installed inside WLS Virtlualization is Ubuntu.

4. Install Debian GNU / Linux distribution as a second distro

 

PS C:\Windows\System32\WindowsPowerShell\v1.0> wsl –install Debian

windows-wsl-linux-emulation/wsl2-windows-virtualization-install-virtual-machine-debian3

That second installed distro would make Debian now the default one to boot by WSL.

To run the fresh installed Debian GNU / Linux distribution, run only wsl command with no arguments.

# wsl

 

PS C:\Windows\System32\WindowsPowerShell\v1.0> wsl –set-version Debian 2
For information on key differences with WSL 2 please visit https://aka.ms/wsl2
Conversion in progress, this may take a few minutes.
The distribution is already the requested version.
Error code: Wsl/Service/WSL_E_VM_MODE_INVALID_STATE
PS C:\Windows\System32\WindowsPowerShell\v1.0> wsl –set-version 2
There is no distribution with the supplied name.
Error code: Wsl/Service/WSL_E_DISTRO_NOT_FOUND
PS C:\Windows\System32\WindowsPowerShell\v1.0>

Simply pressting CTRL + D from the actively running WSL emulated Linux (that is pretty much like a native Windows docker container if we have to compare to Linux) would stop the VM.
 

5. List runnable / installed VM Linux distributions
 

To list the available runnable Linux VMs on your Windows  status on Windows Subsystem for Linux:

PS C:\Windows\System32\WindowsPowerShell\v1.0> wsl –list –verbose
  NAME      STATE           VERSION
* Debian    Stopped         2
  Ubuntu    Stopped         2

PS C:\Windows\System32\WindowsPowerShell\v1.0>


6. Run and check recent installed Linux distribution version

wsl2-windows-virtualization-install-virtual-machine-debian4

To run the newly install Debian Virtualized Linux (which as you can see is the default set distribution to run by WSL virtualization) simply type 

PS C:\Windows\System32\WindowsPowerShell\v1.0> wsl

hipo@PC2LP3:/mnt/c/Windows/System32/WindowsPowerShell/v1.0$hipo@WL-2SLPWL3:/mnt/c/Windows/System32/WindowsPowerShell/v1.0$ cd ~
hipo@PC2LP3:~$

 

hipo@PC2LP3:~$ cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"
NAME="Debian GNU/Linux"
VERSION_ID="12"
VERSION="12 (bookworm)"
VERSION_CODENAME=bookworm
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
hipo@WL-2SLPWL3:~$

 

7. Update the Debian distribuion packages to latest available

hipo@PC2LP3:~$ sudo su – root
hipo@PC2LP3:~# apt update –fix-missing


8. Install openssh server to be able to connect to the WSL hosted Virtual Machine

hipo@PC2LP3:/home/hipo# apt install openssh-server –yes


windows-wsl-linux-emulation

 

root@PC2LP3:/home/hipo# systemctl start openssh-server telnet
System has not been booted with systemd as init system (PID 1). Can't operate.
Failed to connect to bus: Host is down
root@WL-2SLPWL3:/home/hipo# /etc/init.d/ssh start
Starting OpenBSD Secure Shell server: sshd.
root@WL-2SLPWL3:/home/hipo# ps -ef|grep -i ssh
root        30     9  0 18:19 ?        00:00:00 sshd: /usr/sbin/sshd [listener] 0 of 10-100 startups
root        32    15  0 18:20 pts/1    00:00:00 grep -i ssh

 

windows-wsl-linux-emulation

By default a fresh new installed VM would have a process list like below:

root@PC2LP3:/home/hipo# ps axuwef


wsl2-windows-virtualization-install-virtual-machine-debian7

To be able to have ifconfig and a number of other network tools it is useful to install net-tools package

root@PC2LP3:/home/hipo# apt install net-tools –yes

root@PC2LP3:/home/hipo# /sbin/ifconfig

Once the WSL VM and OpenSSHD is run you can try to telnet or ssh to the VM locally or remotely.

root@PC2LP3:/home/hipo# telnet localhost 22
Trying 127.0.0.1…
Connected to localhost.
Escape character is '^]'.
SSH-2.0-OpenSSH_9.2p1 Debian-2+deb12u3

9. Run commands directly from Windows command line or Powershell
 

You can also use the powershell to run commands via the virtualized Linux environment using simple syntax

# wsl [cmd-to-run]

PS C:\Windows\System32\WindowsPowerShell\v1.0> wsl ls /
bin   dev  home  lib    lost+found  mnt  proc  run   srv  tmp  var
boot  etc  init  lib64  media       opt  root  sbin  sys  usr
PS C:\Windows\System32\WindowsPowerShell\v1.0> wsl ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0  0 18:07 hvc0     00:00:00 /init
root         5     1  0 18:07 hvc0     00:00:00 plan9 –control-socket 5 –log-level 4 –server-fd 6 –pipe-fd 8 –log-t
root         8     1  0 18:07 ?        00:00:00 /init
root         9     8  0 18:07 ?        00:00:00 /init
hipo        10     9  0 18:07 pts/0    00:00:00 ps -ef

PS C:\Windows\System32\WindowsPowerShell\v1.0>

10. Enable systemd on Linux distribution in WSL 2

Once you boot into the WSL installed distro shell edit /etc/wsl.conf:

$ vim /etc/wsl.conf

[boot]
systemd=true

11. Setting extra useful variables to boot the WSL emulated Linux VM 
 

root@debian-wsl:/home/hipo# cat /etc/wsl.conf
[boot]
systemd=true

# Automatically mount Windows drive when the distribution is launched
[automount]

# Set to true will automount fixed drives (C:/ or D:/) with DrvFs under the root directory set above. Set to false means drives won't be mounted automatically, but need to be mounted manually or with fstab.
enabled = true

# Sets the directory where fixed drives will be automatically mounted. This example changes the mount location, so your C-drive would be /c, rather than the default /mnt/c.
root = /

# DrvFs-specific options can be specified.
options = "metadata,uid=1003,gid=1003,umask=077,fmask=11,case=off"

# Sets the `/etc/fstab` file to be processed when a WSL distribution is launched.
mountFsTab = true

# Network host settings that enable the DNS server used by WSL 2. This example changes the hostname, sets generateHosts to false, preventing WSL from the default behavior of auto-generating /etc/hosts, and sets generateResolvConf to false, preventing WSL from auto-generating /etc/resolv.conf, so that you can create your own (ie. nameserver 1.1.1.1).
[network]
hostname = debian-wsl
generateHosts = true
generateResolvConf = true

# Set whether WSL supports interop processes like launching Windows apps and adding path variables. Setting these to false will block the launch of Windows processes and block adding $PATH environment variables.
[interop]
enabled = false
appendWindowsPath = false

# Set the user when launching a distribution with WSL.
[user]
default = hipo

# Set a command to run when a new WSL instance launches. This example starts the Docker container service.
#[boot]
#command = service docker start

root@debian-wsl:/home/hipo#

To learn about on Advanced settings configuration in WSL check out official Microsoft documentation here

12. Shutting down a running emulated Linux VM

If you have run a WSL VM and you want to shut it down do:

# wsl shutdown


If you at a point want to delete / uninstall the installed distribution you can do

# wsl –terminate Distro_Name
# wsl –uninstall Distro_Name


Or you if you want to do a cleanup of the stored files inside the installed distribution (if you have stored files), do:

# wsl –unregister Distro_Name


For more in depth details check out the manual
 

PS C:\Windows\System32\WindowsPowerShell\v1.0> wsl –help
Copyright (c) Microsoft Corporation. All rights reserved.
For privacy information about this product please visit https://aka.ms/privacy.

Usage: wsl.exe [Argument] [Options…] [CommandLine]

Arguments for running Linux binaries:

    If no command line is provided, wsl.exe launches the default shell.

    –exec, -e <CommandLine>
        Execute the specified command without using the default Linux shell.

    –shell-type <standard|login|none>
        Execute the specified command with the provided shell type.

    —
        Pass the remaining command line as-is.

Options:
    –cd <Directory>
        Sets the specified directory as the current working directory.
        If ~ is used the Linux user's home path will be used. If the path begins
        with a / character, it will be interpreted as an absolute Linux path.
        Otherwise, the value must be an absolute Windows path.

    –distribution, -d <Distro>
        Run the specified distribution.

    –user, -u <UserName>
        Run as the specified user.

    –system
        Launches a shell for the system distribution.

Arguments for managing Windows Subsystem for Linux:

    –help
        Display usage information.

    –debug-shell
        Open a WSL2 debug shell for diagnostics purposes.

    –install [Distro] [Options…]
        Install a Windows Subsystem for Linux distribution.
        For a list of valid distributions, use 'wsl.exe –list –online'.

        Options:
            –no-launch, -n
                Do not launch the distribution after install.

            –web-download
                Download the distribution from the internet instead of the Microsoft Store.

            –no-distribution
                Only install the required optional components, does not install a distribution.

            –enable-wsl1
                Enable WSL1 support.

    –manage <Distro> <Options…>
        Changes distro specific options.

        Options:
            –move <Location>
                Move the distribution to a new location.

            –set-sparse, -s <true|false>
                Set the vhdx of distro to be sparse, allowing disk space to be automatically reclaimed.

    –mount <Disk>
        Attaches and mounts a physical or virtual disk in all WSL 2 distributions.

        Options:
            –vhd
                Specifies that <Disk> refers to a virtual hard disk.

            –bare
                Attach the disk to WSL2, but don't mount it.

            –name <Name>
                Mount the disk using a custom name for the mountpoint.

            –type <Type>
                Filesystem to use when mounting a disk, if not specified defaults to ext4.

            –options <Options>
                Additional mount options.

            –partition <Index>
                Index of the partition to mount, if not specified defaults to the whole disk.

    –set-default-version <Version>
        Changes the default install version for new distributions.

    –shutdown
        Immediately terminates all running distributions and the WSL 2
        lightweight utility virtual machine.

    –status
        Show the status of Windows Subsystem for Linux.

    –unmount [Disk]
        Unmounts and detaches a disk from all WSL2 distributions.
        Unmounts and detaches all disks if called without argument.

    –uninstall
        Uninstalls the Windows Subsystem for Linux package from this machine.

    –update
        Update the Windows Subsystem for Linux package.

        Options:
            –pre-release
                Download a pre-release version if available.

    –version, -v
        Display version information.

Arguments for managing distributions in Windows Subsystem for Linux:

    –export <Distro> <FileName> [Options]
        Exports the distribution to a tar file.
        The filename can be – for stdout.

        Options:
            –vhd
                Specifies that the distribution should be exported as a .vhdx file.

    –import <Distro> <InstallLocation> <FileName> [Options]
        Imports the specified tar file as a new distribution.
        The filename can be – for stdin.

        Options:
            –version <Version>
                Specifies the version to use for the new distribution.

            –vhd
                Specifies that the provided file is a .vhdx file, not a tar file.
                This operation makes a copy of the .vhdx file at the specified install location.

    –import-in-place <Distro> <FileName>
        Imports the specified .vhdx file as a new distribution.
        This virtual hard disk must be formatted with the ext4 filesystem type.

    –list, -l [Options]
        Lists distributions.

        Options:
            –all
                List all distributions, including distributions that are
                currently being installed or uninstalled.

            –running
                List only distributions that are currently running.

            –quiet, -q
                Only show distribution names.

            –verbose, -v
                Show detailed information about all distributions.

            –online, -o
                Displays a list of available distributions for install with 'wsl.exe –install'.

    –set-default, -s <Distro>
        Sets the distribution as the default.

    –set-version <Distro> <Version>
        Changes the version of the specified distribution.

    –terminate, -t <Distro>
        Terminates the specified distribution.

    –unregister <Distro>
        Unregisters the distribution and deletes the root filesystem.
PS C:\Windows\System32\WindowsPowerShell\v1.0>

Once wsl is installed you can run it directly from Windows start menu, by searching for the name of the distribution you would like to run for example to run my Debian WSL running emulator::

Sum it up

What was shown up is how to run in parallel virtualized Linux distribution on Windows 10 and Windows 11 and how to install update to latest and run opensshd server to be able to ssh into the WSL Linux virtual machine remotely.
.Also i've shown you, How to test ssh is reachable and how to stop / start or destroy and cleanup any stored files for  VM if necessery, as well as how to apply some extra advanced configurations to boot VM for.

Using WSL is not the best virtualization ever but anyways it is an alternative for people employed in Domain attached Windows PCs part of Big Corporations, where VirtualBox use is blocked / prohibited and you still need to experiment or develop Shell scripts or software on Python / Perl / Ruby on Linux before you  do stuff on the PreProd or Production Linux host.

That's all folks, Enjoy ! 🙂
 

Check the Type and Model of available installed Memory on Linux / Unix / BSD Server howto

Monday, October 30th, 2023

how-linux-kernel-manages-memory-picture

As a system administrator one of the common task, one has to do is Add / Remove or Replace (of Broken or failing Bank of RAM memory) a piece of additional Bank of memory Bank to a Linux / BSD / Unix server.  Lets say you need to fullfil the new RAM purchase and provide some information to the SDM (Service Delivery Manager) of the compnay you're hirder in or you need to place the purchase yourself. Then you  need to know the exact speed and type of RAM currently installed on the server installed.

In this article i'll shortly explain how do I find out ram (SDRAM) information from a via ordinary remote ssh shell session cmd prompt. In short will be shown how can one check RAM speed configured and detected by Linux / Unix kernel ? 
As well as  how to Check the type of memory (if it is DDR / DDR2 / DDR or DDR4) or ECC with no access to Hardware Console.  Please note this article will be definitely boring for the experienced sysadmins but might help to a starter sysadmins to get on board with a well know basic stuff.

There are several approaches, of course easiest one is to use remote hardware access interrace statistics web interface of ILO (on IBM machine) or the IDRAC on (Dell Server) or Fujitsu's servers iRMC. However as not always access to remote Remote hardware management interface is available to admin. Linux comes with few commands that can do the trick, that are available to most Linux distributions straight for the default package repositories.

Since mentioning about ECC a bit up, most old school admins and computer users knows pretty well about DDRs as they have been present over time but ECC is being used over actively on servers perhaps over the last 10 / 15 years and for those not dealt with it below is a short description on what is ECC RAM Memory.

ECC RAM, short for Error Correcting Code Random Access Memory, is a kind of RAM can detect most common kinds of memory errors and correct a subset of them. ECC RAM is common in enterprise deployments and most server-class hardware. Above a certain scale and memory density, single-bit errors which were up to this point are sufficiently statistically unlikely begin to occur with enough frequency that they can no longer be ignored. At certain scales and densities of memory arbitrary memory errors that are literally "one in a million chances" (or more) may in fact occur several times throughout a system's operational life.

Putting some basics, Lets proceed and Check RAM speed and type (line DDR or DDR2 or DDR3 or DDR4) without having to physically go to the the Data Center numbered rack that is containing the server.


Most famous and well known (also mentioned) on few occasions in my previous articles are: dmidecode and lshw

Quickest way to get a quick overview of installed servers memory is with:
 

root@server:~# dmidecode -t memory | grep -E "Speed:|Type:" | sort | uniq -c
      4     Configured Memory Speed: 2133 MT/s
     12     Configured Memory Speed: Unknown
      4     Error Correction Type: Multi-bit ECC
      2     Speed: 2133 MT/s
      2     Speed: 2400 MT/s
     12     Speed: Unknown
     16     Type: DDR4

 

To get more specifics on the exact type of memory installed on the server, the respective slots that are already taken and the free ones:

root@server:~# dmidecode –type 17 | less

Usually the typical output the command would produce regarding lets say 4 installed Banks of RAM memory on the server will be like:

Handle 0x002B, DMI type 17, 40 bytes
Memory Device
        Array Handle: 0x0029
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
       
Size: 16 GB
        Form Factor: RIMM
        Set: None
        Locator: CPU1 DIMM A1
        Bank Locator: A1_Node0_Channel0_Dimm1
       
Type: DDR4
        Type Detail: Synchronous
       
Speed: 2400 MT/s
        Manufacturer: Micron
       
Serial Number: 15B36358
        Asset Tag: CPU1 DIMM A1_AssetTag
       
Part Number: 18ASF2G72PDZ-2G3B1 
        Rank: 2
       
Configured Memory Speed: 2133 MT/s
        Minimum Voltage: Unknown
        Maximum Voltage: Unknown
        Configured Voltage: Unknown

Handle 0x002E, DMI type 17, 40 bytes
Memory Device
        Array Handle: 0x0029
        Error Information Handle: Not Provided
        Total Width: Unknown
        Data Width: Unknown
        Size: No Module Installed
        Form Factor: RIMM
        Set: None
        Locator: CPU1 DIMM A2
        Bank Locator: A1_Node0_Channel0_Dimm2
        Type: DDR4
        Type Detail: Synchronous
        Speed: Unknown
        Manufacturer: NO DIMM
        Serial Number: NO DIMM
        Asset Tag: NO DIMM
        Part Number: NO DIMM
        Rank: Unknown
        Configured Memory Speed: Unknown
        Minimum Voltage: Unknown
        Maximum Voltage: Unknown
        Configured Voltage: Unknown

 

Handle 0x002D, DMI type 17, 40 bytes
Memory Device
        Array Handle: 0x0029
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 16 GB
        Form Factor: RIMM
        Set: None
        Locator: CPU1 DIMM B1
        Bank Locator: A1_Node0_Channel1_Dimm1
        Type: DDR4
        Type Detail: Synchronous
        Speed: 2400 MT/s
        Manufacturer: Micron
        Serial Number: 15B363AF
        Asset Tag: CPU1 DIMM B1_AssetTag
        Part Number: 18ASF2G72PDZ-2G3B1 
        Rank: 2
        Configured Memory Speed: 2133 MT/s
        Minimum Voltage: Unknown
        Maximum Voltage: Unknown
        Configured Voltage: Unknown

Handle 0x0035, DMI type 17, 40 bytes
Memory Device
        Array Handle: 0x0031
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 16 GB
        Form Factor: RIMM
        Set: None
        Locator: CPU1 DIMM D1
        Bank Locator: A1_Node0_Channel3_Dimm1
        Type: DDR4
        Type Detail: Synchronous
        Speed: 2133 MT/s
        Manufacturer: Micron
        Serial Number: 1064B491
        Asset Tag: CPU1 DIMM D1_AssetTag
        Part Number: 36ASF2G72PZ-2G1A2  
        Rank: 2
        Configured Memory Speed: 2133 MT/s
        Minimum Voltage: Unknown
        Maximum Voltage: Unknown
        Configured Voltage: Unknown

Handle 0x0033, DMI type 17, 40 bytes
Memory Device
        Array Handle: 0x0031
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
        Size: 16 GB
        Form Factor: RIMM
        Set: None
        Locator: CPU1 DIMM C1
        Bank Locator: A1_Node0_Channel2_Dimm1
        Type: DDR4
        Type Detail: Synchronous
        Speed: 2133 MT/s
        Manufacturer: Micron
        Serial Number: 10643A5B
        Asset Tag: CPU1 DIMM C1_AssetTag
        Part Number: 36ASF2G72PZ-2G1A2  
        Rank: 2
        Configured Memory Speed: 2133 MT/s
        Minimum Voltage: Unknown
        Maximum Voltage: Unknown
        Configured Voltage: Unknown

 

The marked in green are the banks of memory that are plugged in the server. The

field Speed: and Configured Memory Speed: are fields indicating respectively the Maximum speed on which a plugged-in RAM bank can operate and the the actual Speed the Linux kernel has it configured and uses is at.

It is useful for the admin to usually check the complete number of available RAM slots on a server, this can be done with command like:

root@server:~#  dmidecode –type 17 | grep -i Handle | grep 'DMI'|wc -l
16


As you can see at this specific case 16 Memory slots are avaiable (4 are already occupied and working configured on the machine at 2133 Mhz and 12 are empty and can have installed a memory banks in).


Perhaps the most interesting information for the RAM replacement to be ordered is to know the data communication SPEED on which the Memory is working on the server and interacting with Kernel and Processor to find out.

root@server:~#  dmidecode –type 17 | grep -i "speed"|grep -vi unknown
    Speed: 2400 MT/s
    Configured Memory Speed: 2133 MT/s
    Speed: 2400 MT/s
    Configured Memory Speed: 2133 MT/s
    Speed: 2133 MT/s
    Configured Memory Speed: 2133 MT/s
    Speed: 2133 MT/s
    Configured Memory Speed: 2133 MT/s

 

If you're lazy to remember the exact dmidecode memory type 17 you can use also memory keyword:

root@server:~# dmidecode –type memory | more

For servers that have the lshw command installed, a quick overview of RAM installed and Full slots available for memory placement can be done with:
 

root@server:~#  lshw -short -C memory
H/W path                 Device        Class          Description
=================================================================
/0/0                                   memory         64KiB BIOS
/0/29                                  memory         64GiB System Memory
/0/29/0                                memory         16GiB RIMM DDR4 Synchronous 2400 MHz (0.4 ns)
/0/29/1                                memory         RIMM DDR4 Synchronous [empty]
/0/29/2                                memory         16GiB RIMM DDR4 Synchronous 2400 MHz (0.4 ns)
/0/29/3                                memory         RIMM DDR4 Synchronous [empty]
/0/29/4                                memory         16GiB RIMM DDR4 Synchronous 2133 MHz (0.5 ns)
/0/29/5                                memory         RIMM DDR4 Synchronous [empty]
/0/29/6                                memory         16GiB RIMM DDR4 Synchronous 2133 MHz (0.5 ns)
/0/29/7                                memory         RIMM DDR4 Synchronous [empty]
/0/29/8                                memory         RIMM DDR4 Synchronous [empty]
/0/29/9                                memory         RIMM DDR4 Synchronous [empty]
/0/29/a                                memory         RIMM DDR4 Synchronous [empty]
/0/29/b                                memory         RIMM DDR4 Synchronous [empty]
/0/29/c                                memory         RIMM DDR4 Synchronous [empty]
/0/29/d                                memory         RIMM DDR4 Synchronous [empty]
/0/29/e                                memory         RIMM DDR4 Synchronous [empty]
/0/29/f                                memory         RIMM DDR4 Synchronous [empty]
/0/43                                  memory         768KiB L1 cache
/0/44                                  memory         3MiB L2 cache
/0/45                                  memory         30MiB L3 cache

Now once we know the exact model and RAM Serial and Part number you can google it online and to purchase more of the same RAM Model and Type you need so the installed memory work on the same Megaherzes as the installed ones.