Posts Tagged ‘ssh’

Speed Up Linux: 10 Underrated Hacks for Faster Workstations and Servers

Tuesday, December 9th, 2025

make-linux-faster-underrated-hacks-cute-tux-penguin-toolbox-logo

 

Most Linux “performance tuning guides” recycle the same tips: disable services, add RAM, install a lighter desktop … and then you end up with a machine that feels basically not too much quicker.

Below is a collection of practical, field-tested, sysadmin-grade hacks that actually change how responsive your system feels—on desktops, laptops, and even small VPS servers.

Most of these are rarely mentioned (or uknown) by novice sys admins / sys ops / system engineers / dev ops but time has proved them extremely effective.

1. Enable zram (compressed RAM) instead of traditional swap

Most distros ship with a slow swap partition on disk. Enabling zram keeps swap in memory and compresses it on the fly. On low/mid RAM systems, the difference is night and day.

On Debian/Ubuntu:

# apt install zram-tools

# systemctl enable –now zramswap.service

Expect snappier multitasking, fewer freezes, and far less disk thrashing.

2. Speed up SSH connections with ControlMaster multiplexing

If you SSH multiple times into the same server, you can cut connection time to almost zero using SSH socket multiplexing:

Add to ~/.ssh/config:

Host *

    ControlMaster auto

    ControlPath ~/.ssh/cm-%r@%h:%p

    ControlPersist 10m

Your next ssh/scp commands will feel instant.

 

3. Replace grep, find, and ls with faster modern tools

 

The classic GNU tools are fine—until you try the modern replacements:

  • ripgrep (rg) → insanely faster grep
  • fd → user-friendly, colored alternative to find
  • exa or lsd → modern ls with icons, colors, Git info

# apt install ripgrep fd-find exa

You’ll be surprised how often you use them.

4. Improve boot time via systemd-analyze

Run cmd:

# systemd-analyze blame

This shows what’s delaying your boot. Disable useless services like:
 

# systemctl disable bluetooth.service

# systemctl disable ModemManager

# systemctl disable cups

Great for lightweight servers and old laptops.

5. Make your terminal 2–3× faster with GPU rendering (Kitty/Alacritty)

Gnome Terminal and xterm are CPU-bound and choke when printing thousands of lines.

Switch to a GPU-rendered terminal:

  • Kitty
  • Alacritty

They scroll like butter – even on old ThinkPads.

6. Use eatmydata when installing packages

APT and DPKG spend a lot of time syncing packages safely to disk. If you're working in a Docker container, VM, or test machine where safety is less critical, use:

# apt install eatmydata

# eatmydata apt install PACKAGE

It cuts install time by 30–70% !

7. Enable TCP BBR congestion control (massive network speed boost)

Google’s BBR can double upload throughput on many servers.

Check if supported:

sysctl net.ipv4.tcp_available_congestion_control

Enable:

# echo "net.core.default_qdisc=fq" | sudo tee -a /etc/sysctl.conf

# echo "net.ipv4.tcp_congestion_control=bbr" | sudo tee -a /etc/sysctl.conf

# sysctl -p

On VPS hosts with slow TCP stacks, this is a cheat code.

8. Reduce SSD wear and speed up disk with noatime

Every time a file is read, Linux updates its “access time.” Useless on most systems.

Edit /etc/fstab and change:

defaults

to:

defaults,noatime

Fewer writes + slightly better I/O performance.

9. Use htop + iotop + dstat to diagnose “mystery lag”

When a Linux system hangs, it’s rarely the CPU. It’s almost always:

  • I/O wait
  • swap usage
  • blocked processes
  • misbehaving snaps or flatpaks

Install the golden server stats trio:

# apt install htop iotop dstat

Check:

  • htop → load, processes, swap
  • iotop → who’s killing your disk
  • dstat -cdlmn → real-time performance overview

Using them can solve you 90% of slowdowns in minutes.

10. Speed up your shell prompt dramatically

Fancy PS1 prompts (Git branches, Python envs, emojis) look nice but slow down your shell launch.

Fix it by using async Git status tools:

  • starship prompt
  • powerlevel10k for zsh

Both load instantly even in large repos.

11. Another underrated improvement (if not already done), 
/home directory storage location

Put your $HOME on a separate partition or disk.

Why?

  • faster reinstalls
  • easier backups
  • safer experiments
  • fewer “oops I nuked my system” moments by me or someone else who has account doing stuff on the system

Experienced sysadmins and a good corporate server still does this for a reason.

Final Thoughts

Performance isn’t just about raw hardware—it's about removing the subtle bottlenecks Linux accumulates over time. With the changes above, even a 10-year-old laptop or low-tier VPS feels like a new machine.

How I Stopped My AWS workspace Linux Desktop From Going to Sleep… Without Root Access

Thursday, November 20th, 2025

keeping-session-alive-stop-aws-workspace-to-auto-suspend-with-systemd-inhibit-or-a-simple-loop-scriptcover

If you've spent enough time around Linux servers and desktops, you already know one universal truth:

Linux never does exactly what you expect… especially when you don’t have root.

A few weeks ago, I found myself in a situation that’s probably familiar to anyone who works on shared servers, university machines, or restricted corporate environments:

My session kept going to sleep, killing long-running scripts, dropping SSH tunnels, freezing terminals—for absolutely no good reason.

To make things worse, I didn’t have sudo on this box.
No changing systemd settings, no tweaking /etc/systemd/logind.conf, and definitely no masking sleep targets.

So I went down the rabbit hole of how to keep a Linux machine awake without any superuser privileges.
Here’s the write-up of that journey—and yes, the final solution is surprisingly elegant.

The Problem: When the System Sleeps, Your Work Dies

My main issue: every 15 minutes of inactivity, the system would suspend the session.
Not the entire PC — just my user session. It didn't matter if I had background jobs running or SSH tunnels open; if I wasn’t interacting, the session was toast.

The machines were managed centrally, so root was a luxury I simply didn’t have.

What followed was a typical sysadmin debugging sequence:

  1. Angry at the stupidity
  2. Google.
  3. Try things that shouldn’t work.
  4. Try things that definitely shouldn't work.
  5. Accidentally discover the correct solution while reading some random docs if point 3 doesn’t already solve it

The Trick: systemd-inhibit (Works Without Root!)

While digging through the systemd documentation, I discovered something beautiful:

Non-root users can create inhibitor locks.

This means your normal user account can ask systemd:
Please don’t put this session to sleep while this program is running.”

And systemd says:
“Okay. I respect that.”

All it takes is:

systemd-inhibit –what=handle-lid-switch:sleep –mode=block sleep infinity

This command runs a never-ending sleep process—
and while it runs, the system is forbidden to suspend.

You can even run it in the background:

$ nohup systemd-inhibit sleep infinity &

Want to verify it’s working?

$ systemd-inhibit –list

You’ll see your inhibitor lock listed like a VIP pass at a nightclub.

If You Have caffeinate, Even Better

Some Linux distros ship with a utility called caffeinate (similar to macOS).
It’s almost poetic:

$ caffeinate -di sleep infinity

This one also blocks sleep while the command runs.
Just leave it running as a background job and your session stays alive.

The Primitive but Always-Working Hack: Keepalive Script

If neither systemd-inhibit nor caffeinate exist, you can fall back to a caveman approach and still have the basic functionality of the Move Mouse Windows tool on Linux :):
 

#!/bin/bash

while true; do

    echo "Still here: $(date)"

    sleep 60

done

This prevents session idleness by emitting activity every minute.
Not elegant, but reliable.

Sometimes the caveman wins.

Why This Matters

Keeping a session awake might sound trivial, but for sysadmins, developers, pentesters, researchers, or anyone running long processes on managed machines, it’s a lifesaver.

You avoid:

  • broken SSH tunnels
  • silent failure of long-running scripts
  • GUI sessions locking themselves
  • losing tmux/screen sessions
  • interrupted compiles or renders
  • VPN disconnects

And you don’t need to bug IT or break policy.

Final Thoughts

What surprised me most is how simple the final solution was:

  • No root
  • No configuration changes
  • No hacks
  • No kernel tweaks

Just one systemd command used properly.

Sometimes Linux feels like an inscrutable labyrinth… and sometimes it gives you a quiet, elegant tool hiding in plain sight.

If you ever find yourself fighting unwanted auto-suspend on a machine you don’t control –
give systemd-inhibit a try.

How to Create Secure Stateful Firewall Rules with nftables on Linux

Monday, October 6th, 2025

nftables-logo-linux-mastering-stateful-firewall-rules-with-nftables-firewall

Firewalls are the frontline defense of any server or network. While many sysadmins are familiar with iptables, nftables is the modern Linux firewall framework offering more power, flexibility, and performance.

One of the key features of nftables is stateful packet inspection, which lets you track the state of network connections and write precise rules that dynamically accept or reject packets based on connection status.

In this guide, we’ll deep dive into stateful firewall rules using nftables — what they are, why they matter, and how to master them for a robust, secure network.

What is Stateful Firewalling?

A stateful firewall keeps track of all active connections passing through it. It monitors the state of a connection (new, established, related, invalid) and makes decisions based on that state rather than just static IP or port rules.

This allows:

  • Legitimate traffic for existing connections to pass freely
  • Blocking unexpected or invalid packets
  • Better security and less manual rule writing

Understanding Connection States in nftables

nftables uses the conntrack subsystem to track connections. Common states are:

State

Description

new

Packet is trying to establish a new connection

established

Packet belongs to an existing connection

related

Packet related to an existing connection (e.g. FTP data)

invalid

Packet that does not belong to any connection or is malformed

Basic Stateful Rule Syntax in nftables

The key keyword is ct state. For example:

nft add rule inet filter input ct state established,related accept

This means: allow any incoming packets that are part of an established or related connection.

Step-by-Step: Writing a Stateful Firewall with nftables

  1. Create the base table and chains


nft add table inet filter

nft add chain inet filter input { type filter hook input priority 0 \; }

nft add chain inet filter forward { type filter hook forward priority 0 \; }

nft add chain inet filter output { type filter hook output priority 0 \; }

  1. Allow loopback traffic

nft add rule inet filter input iif lo accept

  1. Allow established and related connections

nft add rule inet filter input ct state established,related accept

  1. Drop invalid packets

nft add rule inet filter input ct state invalid drop

  1. Allow new SSH connections

nft add rule inet filter input tcp dport ssh ct state new accept

  1. Drop everything else

nft add rule inet filter input drop

Why Use Stateful Filtering?

  • Avoid writing long lists of rules for each connection direction
  • Automatically handle protocols with dynamic ports (e.g. FTP, SIP)
  • Efficient resource usage and faster lookups
  • Better security by rejecting invalid or unexpected packets
nftables-tcpip-model-diagram-logo

Advanced Tips for Stateful nftables Rules

  • Use ct helper for protocols requiring connection tracking helpers (e.g., FTP)
  • Combine ct state with interface or user match for granular control
  • Use counters with rules to monitor connection states
  • Rate-limit new connections using limit rate with ct state new

Real-World Example: Preventing SSH Brute Force with Stateful Rules

nft add rule inet filter input tcp dport ssh ct state new limit rate 5/minute accept

nft add rule inet filter input tcp dport ssh drop

This allows only 5 new SSH connections per minute.

Troubleshooting Stateful Rules

  • Use conntrack -L to list tracked connections
  • Logs can help; enable logging on dropped packets temporarily
  • Check if your firewall blocks ICMP (important for some connections)
  • Remember some protocols may require connection helpers

Making Your nftables Rules Permanent

By default, any rules you add using nft commands are temporary — they live in memory and are lost after a reboot.

To make your nftables rules persistent, you need to save them to a configuration file and ensure they're loaded at boot.

Option 1. Using the nftables Service (Preferred on Most Distros)

Most modern Linux distributions (Debian ≥10, Ubuntu ≥20.04, CentOS/RHEL ≥8) come with a systemd service called nftables.service that automatically loads rules from /etc/nftables.conf at boot.

 Do the following to make nftables load on boot:

Dump your current rules into a file:

# nft list ruleset > /etc/nftables.conf

Enable the nftables service to load them at boot:

# systemctl enable nftables

(Optional) Start the service immediately if it’s not running:

# systemctl start nftables

Check status:

# systemctl status nftables

Now your rules will survive reboots.

Alternative way to load nftables on network UP, Use Hooks in
/etc/network/if-pre-up.d/ or Custom Scripts (Advanced)

If your distro doesn't use nftables.service or you're on a minimal setup (e.g., Alpine, Slackware, older Debian), you can load the rules manually at boot:

Save your rules:

# nft list ruleset > /etc/nftables.rules

Create a script to load them (e.g., /etc/network/if-pre-up.d/nftables):

#!/bin/sh

nft -f /etc/nftables.rules

Make it executable:

chmod +x /etc/network/if-pre-up.d/nftables

This method works on systems without systemd.

Sample /etc/nftables.conf config

We first define variables which we can use later on in our ruleset:

 

define NIC_NAME = "eth0"

define NIC_MAC_GW = "DE:AD:BE:EF:01:01"

define NIC_IP = "192.168.1.12"

define LOCAL_INETW = { 192.168.0.0/16 }

define LOCAL_INETWv6 = { fe80::/10 }

define DNS_SERVERS = { 1.1.1.1, 8.8.8.8 }

define NTP_SERVERS = { time1.google.com, time2.google.com, time3.google.com, time4.google.com }

define DHCP_SERVER = "192.168.1.1"

Next code block shows ip filter and ip6 filter sample:

We first create an explicit deny rule (policy drop;) for the chain input and chain output.
This means all network traffic is dropped unless its explicitly allowed later on.

Next we have to define these exceptions based on network traffic we want to allow.
Loopback network traffic is only allowed from the loopback interface and within RFC loopback network space.

nftables automatically maps network protocol names to port numbers (e.g. HTTPS 443).
In this example, we only allow incoming sessions which we initiated (ct state established accept) from ephemeral ports (dport 32768-65535). Be aware an app or web server should allow newly initiated sessions (ct state new).

Certain network sessions initiated by this host (ct state new,established accept) in the chain output are explicitly allowed in the output chain. We also allow outgoing ping requests (icmp type echo-request), but do not want others to ping this host, hence ct state established in the icmp type input chain. 

table ip filter {

    chain input {

       type filter hook input priority 0; policy drop;

       iifname "lo" accept

       iifname "lo" ip saddr != 127.0.0.0/8 drop

       iifname $NIC_NAME ip saddr 0.0.0.0/0 ip daddr $NIC_IP tcp sport { ssh, http, https, http-alt } tcp dport 32768-65535 ct state established accept

       iifname $NIC_NAME ip saddr $NTP_SERVERS ip daddr $NIC_IP udp sport ntp udp dport 32768-65535 ct state established accept

       iifname $NIC_NAME ip saddr $DHCP_SERVER ip daddr $NIC_IP udp sport bootpc udp dport 32768-65535 ct state established log accept

       iifname $NIC_NAME ip saddr $DNS_SERVERS ip daddr $NIC_IP udp sport domain udp dport 32768-65535 ct state established accept

       iifname $NIC_NAME ip saddr $LOCAL_INETW ip daddr $NIC_IP icmp type echo-reply ct state established accept

    }

 

    chain output {

       type filter hook output priority 0; policy drop;

       oifname "lo" accept

       oifname "lo" ip daddr != 127.0.0.0/8 drop

       oifname $NIC_NAME ip daddr 0.0.0.0/0 ip saddr $NIC_IP tcp dport { ssh, http, https, http-alt } tcp sport 32768-65535 ct state new,established accept

       oifname $NIC_NAME ip daddr $NTP_SERVERS ip saddr $NIC_IP udp dport ntp udp sport 32768-65535 ct state new,established accept

       oifname $NIC_NAME ip daddr $DHCP_SERVER ip saddr $NIC_IP udp dport bootpc udp sport 32768-65535 ct state new,established log accept

       oifname $NIC_NAME ip daddr $DNS_SERVERS ip saddr $NIC_IP udp dport domain udp sport 32768-65535 ct state new,established accept

       oifname $NIC_NAME ip daddr $LOCAL_INETW ip saddr $NIC_IP icmp type echo-request ct state new,established accept

    }

 

    chain forward {

       type filter hook forward priority 0; policy drop;

    }

}

 

The next code block is used to block incoming and outgoing IPv6 traffic, except ping requests (icmpv6 type echo-request) and IPv6 network discovery (nd-router-advert, nd-neighbor-solicit, nd-neighbor-advert).

vNICs are often automatically provisioned with IPv6 addresses and left untouched. These interfaces can be abused by malicious entities to tunnel out confidential data or even a shell.

table ip6 filter {

    chain input {

       type filter hook input priority 0; policy drop;

       iifname "lo" accept

       iifname "lo" ip6 saddr != ::1/128 drop

       iifname $NIC_NAME ip6 saddr $LOCAL_INETWv6 icmpv6 type { destination-unreachable, packet-too-big, time-exceeded, parameter-problem, echo-reply, nd-router-advert, nd-neighbor-solicit, nd-neighbor-advert } ct state established accept

    }

 

    chain output {

       type filter hook output priority 0; policy drop;

       oifname "lo" accept

       oifname "lo" ip6 daddr != ::1/128 drop

       oifname $NIC_NAME ip6 daddr $LOCAL_INETWv6 icmpv6 type echo-request ct state new,established accept

    }

 

    chain forward {

       type filter hook forward priority 0; policy drop;

    }

}

 Last code block is used for ARP traffic which limits ARP broadcast network frames:

table arp filter {

   chain input {

       type filter hook input priority 0; policy accept;

       iif $NIC_NAME limit rate 1/second burst 2 packets accept

   }

 

   chain output {

       type filter hook output priority 0; policy accept;

   }

}

To load up nftables rules

# systemctl restart nftables && systemctl status nftables && nft list ruleset

Test Before Save and Apply

NB !!! Always test your rules before saving them permanently. A typo can lock you out of your server !!!

Try:

# nft flush ruleset

# nft -f /etc/nftables.conf


!!! Make sure to test your ports are truly open or closed. You can use nc, telnet or tcpdump for this. !!!

Or use a screen or tmux session and set a watchdog timer (e.g., at now +2 minutes reboot) so you can recover if something goes wrong.

Conclusion

In the ever-evolving landscape of network security, relying on static firewall rules is no longer enough. Stateful filtering with nftables gives sysadmins the intelligence and flexibility needed to deal with real-world traffic — allowing good connections, rejecting bad ones, and keeping things efficient.

With just a few lines, you can build a firewall that’s not only more secure but also easier to manage and audit over time.Whether you're protecting a personal server, a VPS, or a corporate gateway, understanding ct state is a critical step in moving from "good enough" security to proactive, intelligent defense.
If you're still relying on outdated iptables chains with hundreds of line-by-line port filters, maybe it's time to embrace the modern way.
nftables isn’t just the future — it’s the present. Further on log, monitor, and learn from your own traffic.

Start with the basics, then layer on your custom rules and monitoring and enjoy your system services and newtork being a bit more secure than before.


Cheers ! 🙂

How to Install and use FreeIPA forcentralized SSO authention on Linux computer domain

Wednesday, October 1st, 2025

freeipa-gnu-linux-free-sso-solution-logo

FreeIPA is a popular open-source identity management solution that centralizes user, host, and service authentication for Linux environments. It combines LDAP, Kerberos, DNS, and certificate management into a single platform, making it easier to manage large Linux deployments securely.

In this article, we’ll cover how to install FreeIPA on a Linux server, perform initial configuration, and start using it for basic user management.

Prerequisites

  • A clean Linux server (CentOS, RHEL, Fedora, or similar)
  • Root or sudo access
  • A fully qualified domain name (FQDN) for your server (e.g., ipa.example.com)
  • Proper DNS setup (recommended but can be configured during installation)
     

1. Update system to the latest

Start by updating your system to ensure all packages are current.
 

# dnf update -y


2. Install FreeIPA Server Packages

Install the FreeIPA server and its dependencies:

# dnf install -y ipa-server ipa-server-dns

  • ipa-server-dns is optional but recommended if you want FreeIPA to manage DNS for your domain.

3. Configure FreeIPA server

Run the FreeIPA installation script to configure the server. Replace ipa.example.com with your actual server hostname.

sudo ipa-server-install

You will be prompted for:

  • Realm name: Usually uppercase of your domain, e.g., EXAMPLE.COM
  • Directory Manager password: LDAP admin password
  • IPA admin password: FreeIPA admin user password
  • DNS configuration: Enable if you want FreeIPA to manage DNS

Sample configuration flow:

Realm name: EXAMPLE.COM

DNS domain name: example.com

Server host name: ipa.example.com

Directory Manager password: [choose a strong password]

IPA admin password: [choose a strong password]

Do you want to configure integrated DNS (BIND)? [yes/no]: yes

The installer will set up Kerberos, LDAP, the CA, DNS (if chosen), and the Web UI.

4. Start and Enable FreeIPA Services

The installer usually starts services automatically, but you can verify with:

# systemctl status ipa

Enable the service to start on boot:
 

# systemctl enable ipa


5. Access FreeIPA Web Interface

Open your browser and navigate to:

https://ipa.example.com/ipa/ui/

Log in using the admin username and the password you set during installation.

6. Add Users and Groups

You can manage users and groups either via the Web UI or the CLI.

Using the CLI:

Add a new user:

# ipa user-add johndoe –first=John –last=Doe –email=johndoe@example.com

Set a password for the new user:

# ipa passwd johndoe


Add a new group:

# ipa group-add developers –desc="Development Team"


Add user to the group:

# ipa group-add-member developers –users=johndoe


7. Join Client Machines to the FreeIPA Domain
 

On a client machine, install the client packages:

# dnf install -y ipa-client

Run the client setup:

# ipa-client-install –mkhomedir

Follow the prompts to join the client to the FreeIPA domain.

8. Test Authentication
 

Try logging into the client machine with the FreeIPA user you created:
 

# ssh username@client-machine-host.com

You should be able to authenticate using the FreeIPA credentials.
 

Conclusion


You now have a basic FreeIPA server up and running, managing users and authentication across your Linux network. FreeIPA simplifies identity management by providing a centralized, secure, and integrated solution. From here, you can explore advanced features like role-based access control, host-based access control, and certificate management.

 

Here's a practical example of how FreeIPA can be used in a real-world Linux environment.

Scenario: Centralized Authentication in a DevOps Environment
 

Tech Problem

Lets say you are managing a growing team of DevOps engineers and developers across multiple Linux servers (e.g., for CI/CD, staging, and production). Manually creating and maintaining user accounts, SSH keys, and sudo permissions on each server is:

  • Time-consuming
  • Error-prone
  • A security risk (inconsistent policies, orphaned accounts)

Solution: Use FreeIPA to Centralize Identity & Access Management

By deploying FreeIPA, you can:

  • Create user accounts once and manage them centrally
  • Enforce SSO across servers using Kerberos
  • Automatically apply sudo rules, group permissions, and access control policies
  • Easily revoke access for offboarded employees
  • Use host-based access control (HBAC) to control who can log in to what
     

Solution Walkthrough
 

1. Set up FreeIPA server

  • Installed on: ipa.internal.example.com
  • Domain: internal.example.com
  • Realm: INTERNAL.EXAMPLE.COM


2. Add User Accounts

Let's add two users: alice (developer) and bob (DevOps).
 

# ipa user-add alice –first=Alice –last=Smith –email=alice@internal.example.com

# ipa user-add bob –first=Bob –last=Jones –email=bob@internal.example.com

# ipa passwd alice

# ipa passwd bob


3. Create Groups and Roles necessery

Create functional groups for managing permissions.
 

# ipa group-add developers –desc="Developers Team"

# ipa group-add devops –desc="DevOps Team"

# ipa group-add-member developers –users=alice

# ipa group-add-member devops –users=bob

4. Configure Sudo Rules

Let’s allow DevOps team members to use sudo on all servers:
 

# ipa sudorule-add devops-sudo –cmdcat=all

# ipa sudorule-add-user devops-sudo –groups=devops

# ipa sudorule-add-host devops-sudo –hostgroups=all

5. Control Access with HBAC Rules

Let’s say:

  • Developers can access dev and staging servers
  • DevOps can access all servers

# Create host groups
 

# ipa hostgroup-add dev-servers –desc="Development Servers"

# ipa hostgroup-add staging-servers –desc="Staging Servers"

 

# Add hosts to groups
 

# ipa hostgroup-add-member dev-servers –hosts=dev1.internal.example.com

# ipa hostgroup-add-member staging-servers –hosts=staging1.internal.example.com

 

# HBAC rule for developers

# ipa hbacrule-add allow-developers

# ipa hbacrule-add-user allow-developers –groups=developers

# ipa hbacrule-add-host allow-developers –hostgroups=dev-servers

# ipa hbacrule-add-host allow-developers –hostgroups=staging-servers

# ipa hbacrule-add-service allow-developers –hbacsvcs=sshd

 

# HBAC rule for DevOps (all access)

# ipa hbacrule-add allow-devops

# ipa hbacrule-add-user allow-devops –groups=devops

# ipa hbacrule-add-host allow-devops –hostgroups=all

# ipa hbacrule-add-service allow-devops –hbacsvcs=sshd


6. Join Client Servers to FreeIPA

On each Linux server (e.g., dev1, staging1, prod1), run:

 

# ipa-client-install –mkhomedir –server=ipa.internal.example.com –domain=internal.example.com

 

Now, user alice can log in to dev1 and staging1, but not prod1. bob can log in to all servers and use sudo.

7. What Happens When Alice Leaves the Company?

Just disable the user in FreeIPA:

# ipa user-disable alice

This immediately revokes her access across all servers — no need to touch individual machines.

Benefits in This Example

Feature

Outcome

Centralized user management

No need to manually create accounts on every server

Group-based sudo

DevOps has privileged access, others don’t

Access control

Developers only access dev/staging, not prod

Kerberos SSO

Secure, passwordless SSH with ticketing

Auditing

Central logs of who accessed what and when

Quick offboarding

Instant account disablement from a single location

Summary

FreeIPA is not just a replacement for LDAP — it's a full-blown identity and access management solution tailored for Linux systems. In this practical example, it brings enterprise-grade access control, authentication, and user management to a DevOps workflow with minimal friction.

Deploying a Server and Managing a 10-Node Linux Infrastructure with Ansible

Friday, May 30th, 2025

File:Ansible Logo.png - Wikimedia Commons

As organizations grow, manually configuring and maintaining servers quickly becomes inefficient, error-prone, and unscalable. Ansible, a powerful open-source automation tool, allows sysadmins and DevOps engineers to automate server provisioning, configuration management, and application deployment across multiple nodes—making it ideal for managing a fleet of Linux servers.

In this article, you’ll learn how to deploy a server using Ansible and manage an infrastructure of 10 Linux servers with ease.


What Is Ansible?

Ansible is a suite of software tools that enables infrastructure as code. It is open-source and the suite includes software provisioning, configuration management, and application deployment functionality.Ansible is an agentless IT automation tool. It uses SSH to connect to remote machines and YAML-based playbooks to define automation tasks. With Ansible, you can:

  • Install and configure software packages
  • Enforce configuration consistency
  • Perform updates across nodes
  • Deploy applications
  • Orchestrate complex workflows
     

Pre-requisites

To follow this guide, you need:

  • A control node (the machine from which you run Ansible)
  • 10 Linux servers (can be VMs or physical machines)
  • SSH access from the control node to all servers
  • Python installed on the target machines (most Linux distros have this by default)
     

1. Install Ansible on the Control Node 

On Ubuntu / Debian Linux

# apt update sudo apt install ansible -y

Or for CentOS/RHEL

# yum install epel-release -y
# yum install ansible -y

Verify installation:

# ansible –version

2. Configure Your Inventory File

Ansible uses an inventory file to define which servers to manage. By default, this is located at /etc/ansible/hosts , but you can also create a custom one.

Create an inventory file hosts.ini

[webservers]
server1 ansible_host=192.168.1.101
server2 ansible_host=192.168.1.102
server3 ansible_host=192.168.1.103
[dbservers]
server4 ansible_host=192.168.1.104
[all:vars]
ansible_user=ansible_user
ansible_ssh_private_key_file=~/.ssh/id_rsa

You can also use hostnames if DNS is configured.

3. Test Connectivity

# ansible -i hosts.ini all -m ping

If everything is set up correctly, you should see pong responses from all servers.

4. Write Your First Playbook (Provisioning Example)

Create a file called server_setup.yml:


– name: Provision and configure Linux servers
  hosts: all
  become: yes
  tasks:

    – name: Update apt cache (Debian/Ubuntu)
      apt:
        update_cache: yes
      when: ansible_os_family == "Debian"

    – name: Install common packages
      package:
        name:
          – curl
          – git
          – htop
        state: present

    – name: Ensure Nginx is installed on webservers
      apt:
        name: nginx
        state: present
      when: "'webservers' in group_names"

    – name: Start and enable Nginx
      service:
        name: nginx
        state: started
        enabled: yes
      when: "'webservers' in group_names"
 

5. Run the Playbook

ansible-playbook -i hosts.ini server_setup.yml

Ansible will SSH into each server, execute the tasks, and return status messages.

6. Scaling to 10+ Servers

Managing a growing infrastructure is simple with Ansible:

  • Add new servers to hosts.ini
  • Reuse existing playbooks for setup
  • Use roles to modularize configuration (e.g., webserver, database, monitoring)
  • Schedule playbooks using cron or CI/CD pipelines (e.g., GitHub Actions, Jenkins)

7. Sample Ansible Project Structure

Organizing your Ansible project effectively is crucial for scalability and maintainability. Here's a sample recommended directory layout:

ansible-infra/
├── hosts.ini
├── server_setup.yml
├── roles/
│   ├── common/
│   │   ├── tasks/
│   │   │   └── main.yml
│   │   └── files/
│   └── webserver/
│       ├── tasks/
│       │   └── main.yml
│       └── templates/
└── group_vars/
    └── all.yml
 

Key Components:
 

  • ansible.cfg: Configuration file defining paths and settings.
  • inventories/: Directory containing environment-specific inventory files.
  • playbooks/: Directory for playbooks that define automation tasks.
  • roles/: Directory for reusable roles, each with its own tasks and variables.
  • requirements.yml: File listing external roles or collections.
     

For a detailed explanation of this structure, refer to the Ansible Documentation.

Visual Workflow (text) Diagram

To illustrate the workflow, here's a diagram depicting how Ansible interacts with your infrastructure:

+——————+        +——————+        +——————+
|   Control Node   |        |    Ansible      |        |   Managed Nodes  |
|  (Your Machine)  |        |  (Ansible Core) |        |  (10 Linux Servers)|
+——————+        +——————+        +——————+
        |                               |                                 |
        | SSH                      | Execute Playbooks  | Apply Configurations
        |                               |                                 |
        +————————->+————————->+

Workflow Steps:

  1. Control Node: You initiate commands from your local machine.

  2. Ansible Core: Ansible connects via SSH to each managed node.

  3. Managed Nodes: Ansible applies configurations as defined in your playbooks and roles.

This workflow ensures consistent and automated management of your infrastructure.

Best Practices

  • Use roles: Organize playbooks into reusable roles
     (ansible-galaxy init myrole)
  • Maintain version control: Keep playbooks and inventory in Git
  • Encrypt secrets: Use ansible-vault for secure credentials
  • Test in staging: Always validate changes before pushing to production

Conclusion

Ansible is a game-changer for managing Linux infrastructure. With a few simple playbooks, you can provision, configure, and maintain your servers consistently and securely.
For a 10-node infrastructure, it offers the perfect balance of simplicity and power—letting you scale efficiently without the overhead of agent-based solutions.

How to run SSH server Mac OS X and set it to auto boot on Mac Book system start

Monday, February 5th, 2024

mac os X

How to run SSH Server on Mac OS X to administrate remotely your MAC OS to access remote MacBook Air or Mac OS 

Linux / UNIX users know it is pretty easy to run OpenSSH server on old Linux SystemV releases

it is done with cmd:

# /etc/init.d/sshd start


On newer Linux distros where systemd is the standard it is done wtih:

# systemctl start ssh.service

To enable ssh service on boot on systemd distros

# systemctl enable ssh.service


To enable SSH access on Mac OS X this is done wtih a simple command

To check the status of SSH server being on or OFF, either connect with netcat to TCP port 22, which is usually installed by default on most MAC OS-es or run:

# systemsetup -getremotelogin

To start and enable SSH service on Mac OS X run:

# systemsetup -setremotelogin on 


If you later need to turn off the SSH service

# systemsetup -setremotelogin off

Actually systemsetup command can do pretty much on MAC OS X and it is worthy to take a look at it, if you're running a MAC PC or Mac Book laptop.

systemsetup can set the current date, change time server host, set computer name (hostname) and much more.

sh-3.2# systemsetup -help

systemsetup Help Information
————————————-
Usage: systemsetup -getdate
        Display current date.

Usage: systemsetup -setdate <mm:dd:yy>
        Set current date to <mm:dd:yy>.

Usage: systemsetup -gettime
        Display current time.

Usage: systemsetup -settime <hh:mm:ss>
        Set current time to <hh:mm:ss>.

Usage: systemsetup -gettimezone
        Display current time zone.

Usage: systemsetup -settimezone <timezone>
        Set current time zone to <timezone>. Use "-listtimezones" to list time zones.

Usage: systemsetup -listtimezones
        List time zones supported by this machine.

Usage: systemsetup -getusingnetworktime
        Display whether network time is on or off.

Usage: systemsetup -setusingnetworktime <on off>
        Set using network time to either <on> or <off>.

Usage: systemsetup -getnetworktimeserver
        Display network time server.

Usage: systemsetup -setnetworktimeserver <timeserver>
        Set network time server to <timeserver>.

Usage: systemsetup -getsleep
        Display amount of idle time until computer, display and hard disk sleep.

Usage: systemsetup -setsleep <minutes>
        Set amount of idle time until computer, display and hard disk sleep to <minutes>.
        Specify "Never" or "Off" for never.

Usage: systemsetup -getcomputersleep
        Display amount of idle time until computer sleeps.

Usage: systemsetup -setcomputersleep <minutes>
        Set amount of idle time until compputer sleeps to <minutes>.
        Specify "Never" or "Off" for never.

Usage: systemsetup -getdisplaysleep
        Display amount of idle time until display sleeps.

Usage: systemsetup -setdisplaysleep <minutes>
        Set amount of idle time until display sleeps to <minutes>.
        Specify "Never" or "Off" for never.

Usage: systemsetup -getharddisksleep
        Display amount of idle time until hard disk sleeps.

Usage: systemsetup -setharddisksleep <minutes>
        Set amount of idle time until hard disk sleeps to <minutes>.
        Specify "Never" or "Off" for never.

Usage: systemsetup -getwakeonmodem
        Display whether wake on modem is on or off.

Usage: systemsetup -setwakeonmodem <on off>
        Set wake on modem to either <on> or <off>.

Usage: systemsetup -getwakeonnetworkaccess
        Display whether wake on network access is on or off.

Usage: systemsetup -setwakeonnetworkaccess <on off>
        Set wake on network access to either <on> or <off>.

Usage: systemsetup -getrestartpowerfailure
        Display whether restart on power failure is on or off.

Usage: systemsetup -setrestartpowerfailure <on off>
        Set restart on power failure to either <on> or <off>.

Usage: systemsetup -getrestartfreeze
        Display whether restart on freeze is on or off.

Usage: systemsetup -setrestartfreeze <on off>
        Set restart on freeze to either <on> or <off>.

Usage: systemsetup -getallowpowerbuttontosleepcomputer
        Display whether the power button is able to sleep the computer.

Usage: systemsetup -setallowpowerbuttontosleepcomputer <on off>
        Enable or disable whether the power button can sleep the computer.

Usage: systemsetup -getremotelogin
        Display whether remote login is on or off.

Usage: systemsetup -setremotelogin <on off>
        Set remote login to either <on> or <off>. Use "systemsetup -f -setremotelogin off" to suppress prompting when turning remote login off.

Usage: systemsetup -getremoteappleevents
        Display whether remote apple events are on or off.

Usage: systemsetup -setremoteappleevents <on off>
        Set remote apple events to either <on> or <off>.

Usage: systemsetup -getcomputername
        Display computer name.

Usage: systemsetup -setcomputername <computername>
        Set computer name to <computername>.

Usage: systemsetup -getlocalsubnetname
        Display local subnet name.

Usage: systemsetup -setlocalsubnetname <name>
        Set local subnet name to <name>.

Usage: systemsetup -getstartupdisk
        Display current startup disk.

Usage: systemsetup -setstartupdisk <disk>
        Set current startup disk to <disk>.

Usage: systemsetup -liststartupdisks
        List startup disks on this machine.

Usage: systemsetup -getwaitforstartupafterpowerfailure
        Get the number of seconds after which the computer will start up after a power failure.

Usage: systemsetup -setwaitforstartupafterpowerfailure <seconds>
        Set the number of seconds after which the computer will start up after a power failure. The <seconds> value must be a multiple of 30 seconds.

Usage: systemsetup -getdisablekeyboardwhenenclosurelockisengaged
        Get whether or not the keyboard should be disabled when the X Serve enclosure lock is engaged.

Usage: systemsetup -setdisablekeyboardwhenenclosurelockisengaged <yes no>
        Set whether or not the keyboard should be disabled when the X Serve enclosure lock is engaged.

Usage: systemsetup -version
        Display version of systemsetup tool.

Usage: systemsetup -help
        Display help.

Usage: systemsetup -printCommands
        Display commands.

 

Enabling SSH in Mac OS X computers can be done also from Graphical interface for the lazy ones.

enable-ssh-mac-remote-login-from-mac-OS-X-gui

iSH, the best free SSH / Telnet client for iOS iPhone, iPad equivallent of MobaXterm and fully functional Alpine Linux emulator

Wednesday, February 8th, 2023

ish-linux-terminal-emulator-for-iphone-ipad-ios-logo-screenshoticon

Since few months I've switched my old BLU r1 HD Phone (a great old low budget phone for its price) to a friend's iPhone 10 ( X ) who gifted it for me. Coming from Android world, everyone who has experience with it is a pain in the ass as some of the Apps, which are into Google's play store does not have the same equivalent into Apple's install Package manager tool AppStore. Some of the crucial tools which I was interested as a freshly new migrated user from Android to iPhone was to have a decent SSH / Telnet client and Terminal, with which I can easily connect to my Linux servers both home and work. 

As Android Phone user, to connect and manage my SSH sessions I used most often some of the most popular Connectbot / SSHDroid / JuiceSSH.
On Android I've usually installed all of these tools but most frequently used Connectbot, which quickly become my favourite SSH client for Android over time.

The reasons why I really loved Connectbot and used it on Android OS in short:

  • It is Completely free
  • Ad-free
  • Open-source (too bad not Free software but still step better)
  • Copy and paste text between Applications
  • Customizable interface (i.e. font size, keyboard layout, SSH auth agent, etc.)


connectbot-android-ssh-remote-connect-client-screenshot

I've seen some people used and preferred Termius but never myself really liked this client, as it was including some Advertisements or for don't remember why reason.
Switching to iOS mobile operating system, of course was quite a shock especially the moment I found out the standard loved SSH Remote Client programs are used are not available or have only a paid version. Thus it took me quite a while of a research and googling until I found some decent stuff.

termius-ssh-telnet-client-ios-screenshot

Tried for a time with Termius as well but again, its Ads and lack of some functionality pissed me off, so I've moved on to Shelly.

shelly-iphone-ssh-telnet-client-ios-screenshot

Shelly is really not a bad tool but has limitation over the SSH sessions you can add and other limitations, which can only be unlocked with an "Upgrade", to its paid version, thus I decided after few weeks of attempts to make it my remote server management mobile tool for iPhone, I've dropped it off as well.

Then I found the Blink Shell App – Blink Shell is a professional, desktop grade terminal for iOS. As overall the tool is really great and is easy to use but again to have it used in its full power you need the paid version and until you pay for it every now and then you got interruption of your shell for some really annoying ads.
Thus even though I used it for a times this few tools with whom basicly you can do basic remote ssh / telnet session operations eventually,  started looking for a better SSH Client Free alternative for iPhone Users.

Then came a friend at home for a dinner my dear friend Milen (Static) and he show me iOS.
The moment I saw this tool I totally loved it, for its simplicity and its resemblance to a classical TTY Physical old Linux console I used back in the days and its ability to resemble easily any improved functionaltiy through simple screen (multiple session management) command tool or tmux.

Wait, what's iSH ? And why it is the Best SSH / Telnet client to manage your servers remotely on iOS Mobiles (iPhone and IPads) ? 

iSH is a project to get a Linux shell environment running locally on your iOS device, using a usermode x86 emulator.


In other wors iSH is Linux emulator with busybox and a package ports for many of the standard Linux tools you get by simple apt-get / yum or if I have to compare you get via the MobaXterm's advanced apt-cyg (Cygwin packages) tool capabilities.

Once iSH is installed it comes with pre-installed apk command line package management tool, with which you can install stuff like openssh-client / screen / tmux / mc (midnight commander) etc. apk, is an apt like command like tool which uses as a basis for installing its packages Alpine Linux repositories.
Alpine Linux is perhaps little known as it is not one of these main stream disributions, such as Fedora or Ubuntu, but for those more concerned about security  Alpine Linux is well known as it is a security-oriented, lightweight Linux distribution based on musl libc and busybox. What makes the Linux even more attractive and perhaps the reason why the iSH developers decided to use it as a basis for their iSH emulator is it being actively developed and its tightened security makes it a good compliment to the quite closed and security focused mobile platform iOS.

iSH is available straight from AppStore , so to use it install it and run it (it is really a great news that iOS does not require iphone to be jailbreak – ed, and it is an ordinary installable software straight from AppStore):
iSH, already comes with some of the standard programs you would expect in a Linux environment such as Vi, wget, zip / unzip, and tar.
However to fit it better for my use over ssh and improve its capabilities, as well as support and use multiple Virtual windows ssh, just like you do on a Linux xterm
run from ish shell: 

# apk add openssh-client
# apk add screen
# apk add vim
# apk add mc


ish-screenshot-terminal3-linux-emulator-iphone-alpine

ish-screenshot-terminal2-linux-emulator-iphone-alpine

ish-screenshot-terminal1-linux-emulator-iphone-alpine-linux

I also like to have a Midnight Commander and VIM Text editor installed out of the box to be able to move around in Ncurses interface through my iPhone.

ish-iphone-keyboard-key-shortcuts

Note that, just like most GNU / Linux distributions, iOS shell will run a normal bash shell.
From there on to use iSH as my default SSH client and enable my just installed GNU screen some Windowing beauty for readability whence I use the screen with multiple ssh logins to different servers as well make the screen Virtual consoles to have ability for scroll back and scroll up of console text to work, I do set up the following .screenrc inside my /home/iPhoneuser

The .screenrc to setup on the iSH to easify your work with screen is as follows:
 

# An alternative hardstatus to display a bar at the bottom listing the
# windownames and highlighting the current windowname in blue. (This is only
# enabled if there is no hardstatus setting for your terminal)
hardstatus on
hardstatus alwayslastline
hardstatus string "%{.bW}%-w%{.rW}%n %t%{-}%+w %=%{..G} %H %{..Y} %m/%d %C%a "
# Enable scrolling fix the annoying screen scrolling problem
termcapinfo xterm* ti@:te@
# Scroll up
bindkey -d "^[[5S" eval copy "stuff 5\025"
bindkey -m "^[[5S" stuff 5\025

# Scroll down
bindkey -d "^[[5T" eval copy "stuff 5\004"
bindkey -m "^[[5T" stuff 5\004

# Scroll up more
bindkey -d "^[[25S" eval copy "stuff \025"
bindkey -m "^[[25S" stuff \025

# Scroll down more
bindkey -d "^[[25T" eval copy "stuff \004"
bindkey -m "^[[25T" stuff \004

You can download the same .screenrc file from here straight with wget from the console:

# wget https://www.pc-freak.net/files/.screenrc


Run GNU screen manager

 

 # screen

You will end up with a screen session, to open a new session for Virtual Terminal use virtual keyboard from ISH and Press

CTRL + A + C

To open other Virtual Windows inside screen just press CTRL + A + C as many times as you need it, each session will appear ina small window on the down corner as you can see in screenshot

ish-terminal-with-screen-multiple-virtual-terminals-screenshot-iphone-ios

To move across the Screen unnamed 3 Virtual Windows 0 ash 1 ash and 2 ash use the Virtual keyboard

for next WIndow use key combination:
 

CTRL + A + N (where + is just to indicate you have to press them once after another and not actually press the + 🙂 )


For Previous Window use:

CTRL + A + P

Or use CTRL + A and type 

:number 3 (where number is the number of window)

The available iSH commands without adding any further packages which are part of the busybox install are as follows:

Available /bin/ directory commands:

arch  ash  base64  bbconfig  busybox  cat  chgrp  chmod  chown  conspy  cp  date  dd  df  dmesg  dnsdomainname  dumpkmap  echo  ed  egrep  false  fatattr  fdflush  fgrep  fsync  getopt  grep  gunzip  gzip  hostname  ionice  iostat  ipcalc  kbd_mode  kill  link  linux32  linux64  ln  login  ls  lzop  makemime  mkdir  mknod  mktemp  more  mount  mountpoint  mpstat  mv  netstat  nice  pidof  ping  ping6  pipe_progress  printenv  ps  pwd  reformime  rev  rm  rmdir  run-parts  sed  setpriv  setserial  sh  sleep  stty  su  sync  tar  touch  true  umount  uname  usleep  watch  zcat  


Available /usr/bin/ commands:    

awk  basename  beep  blkdiscard  bunzip2  bzcat  bzip2  cal  chvt  cksum  clear  cmp  comm  cpio  crontab  cryptpw  cut  dc  deallocvt  diff  dirname  dos2unix  du  dumpleases  eject  env  expand  expr  factor  fallocate  find  flock  fold  free  fuser  getconf  getent  groups  hd  head  hexdump  hostid  iconv  id  install  ipcrm  ipcs  killall  ldd  less  logger  lsof  lsusb  lzcat  lzma  lzopcat  md5sum  mesg  microcom  mkfifo  mkpasswd  nc  nl  nmeter  nohup  nproc  nsenter  nslookup  od  passwd  paste  patch  pgrep  pkill  pmap  printf  pscan  pstree  pwdx  readlink  realpath  renice  reset  resize  scanelf  seq  setkeycodes  setsid  sha1sum  sha256sum  sha3sum  sha512sum  showkey  shred  shuf  smemcap  sort  split  ssl_client  strings  sum  tac  tail  tee  test  time  timeout  top  tr  traceroute  traceroute6  truncate  tty  ttysize  udhcpc6  unexpand  uniq  unix2dos  unlink  unlzma  unlzop  unshare  unxz  unzip  uptime  uudecode  uuencode  vi  vlock  volname  wc  wget  which  whoami  whois  xargs  xxd  xzcat  yes  


If you're a maniac developer you can even use iSH, to do some programs development with vim with Python / Perl or PHP as these are available from the Alpine repositories and installable via a simple apk add packagename for security experts nmap and some security tools are also available but unfortunately not everything is still working as this project is in active development and iOS has some security limitations if OS is not ROOTED 🙂

Hence some of the packages you can install via apk manager will be failing actually.
There is a list of What works and what doesn't still on iSH on the project github wiki check it out here.

There is much more funny stuff you can do with it, and actually my quick research on how people use iSH on their phones lead me to some Videos talking about iOS and Ethical hacking etc, but I'll stop here as I dont have the time to dig deeper to it. 
If you know or have some good use of iSH or some other goody you are using as a hack please share in comments.

Enjoy ! 🙂

Megaraid SAS software installation on CentOS Linux

Saturday, October 20th, 2012

With a standard el5 on a new Dell server, it may be necessary to install the Dell Raid driver, otherwise the OMSA always reports an error and hardware monitoring is therefore obsolete:

Previously, the megaraid_sys package was now called mptlinux

For this we need the following packages in advance:

# yum install gcc kernel-devel
Now the driver stuff:

# yum install dkms mptlinux
That should have built the new module, better test it:

# modinfo mptsas

# dkms status
After a kernel update it may be necessary to build the driver for the new version:

# dkms build -m mptlinux -v 4.00.38.02

# dkms install -m mptlinux -v 4.00.38.02

Hack: Using ssh / curl or wget to test TCP port connection state to remote SSH, DNS, SMTP, MySQL or any other listening service in PCI environment servers

Wednesday, December 30th, 2020

using-curl-ssh-wget-to-test-tcp-port-opened-or-closed-for-web-mysql-smtp-or-any-other-linstener-in-pci-linux-logo

If you work on PCI high security environment servers in isolated local networks where each package installed on the Linux / Unix system is of importance it is pretty common that some basic stuff are not there in most cases it is considered a security hole to even have a simple telnet installed on the system. I do have experience with such environments myself and thus it is pretty daunting stuff so in best case you can use something like a simple ssh client if you're lucky and the CentOS / Redhat / Suse Linux whatever distro has openssh-client package installed.
If you're lucky to have the ssh onboard you can use telnet in same manner as netcat or the swiss army knife (nmap) network mapper tool to test whether remote service TCP / port is opened or not. As often this is useful, if you don't have access to the CISCO / Juniper or other (networ) / firewall equipment which is setting the boundaries and security port restrictions between networks and servers.

Below is example on how to use ssh client to test port connectivity to lets say the Internet, i.e.  Google / Yahoo search engines.
 

[root@pciserver: /home ]# ssh -oConnectTimeout=3 -v google.com -p 23
OpenSSH_7.9p1 Debian-10+deb10u2, OpenSSL 1.1.1g  21 Apr 2020
debug1: Connecting to google.com [172.217.169.206] port 23.
debug1: connect to address 172.217.169.206 port 23: Connection timed out
debug1: Connecting to google.com [2a00:1450:4017:80b::200e] port 23.
debug1: connect to address 2a00:1450:4017:80b::200e port 23: Cannot assign requested address
ssh: connect to host google.com port 23: Cannot assign requested address
root@pcfreak:/var/www/images# ssh -oConnectTimeout=3 -v google.com -p 80
OpenSSH_7.9p1 Debian-10+deb10u2, OpenSSL 1.1.1g  21 Apr 2020
debug1: Connecting to google.com [172.217.169.206] port 80.
debug1: connect to address 172.217.169.206 port 80: Connection timed out
debug1: Connecting to google.com [2a00:1450:4017:807::200e] port 80.
debug1: connect to address 2a00:1450:4017:807::200e port 80: Cannot assign requested address
ssh: connect to host google.com port 80: Cannot assign requested address
root@pcfreak:/var/www/images# ssh google.com -p 80
ssh_exchange_identification: Connection closed by remote host
root@pcfreak:/var/www/images# ssh google.com -p 80 -v -oConnectTimeout=3
OpenSSH_7.9p1 Debian-10+deb10u2, OpenSSL 1.1.1g  21 Apr 2020
debug1: Connecting to google.com [172.217.169.206] port 80.
debug1: connect to address 172.217.169.206 port 80: Connection timed out
debug1: Connecting to google.com [2a00:1450:4017:80b::200e] port 80.
debug1: connect to address 2a00:1450:4017:80b::200e port 80: Cannot assign requested address
ssh: connect to host google.com port 80: Cannot assign requested address
root@pcfreak:/var/www/images# ssh google.com -p 80 -v -oConnectTimeout=5
OpenSSH_7.9p1 Debian-10+deb10u2, OpenSSL 1.1.1g  21 Apr 2020
debug1: Connecting to google.com [142.250.184.142] port 80.
debug1: connect to address 142.250.184.142 port 80: Connection timed out
debug1: Connecting to google.com [2a00:1450:4017:80c::200e] port 80.
debug1: connect to address 2a00:1450:4017:80c::200e port 80: Cannot assign requested address
ssh: connect to host google.com port 80: Cannot assign requested address
root@pcfreak:/var/www/images# ssh google.com -p 80 -v
OpenSSH_7.9p1 Debian-10+deb10u2, OpenSSL 1.1.1g  21 Apr 2020
debug1: Connecting to google.com [172.217.169.206] port 80.
debug1: Connection established.
debug1: identity file /root/.ssh/id_rsa type 0
debug1: identity file /root/.ssh/id_rsa-cert type -1
debug1: identity file /root/.ssh/id_dsa type -1
debug1: identity file /root/.ssh/id_dsa-cert type -1
debug1: identity file /root/.ssh/id_ecdsa type -1
debug1: identity file /root/.ssh/id_ecdsa-cert type -1
debug1: identity file /root/.ssh/id_ed25519 type -1
debug1: identity file /root/.ssh/id_ed25519-cert type -1
debug1: identity file /root/.ssh/id_xmss type -1
debug1: identity file /root/.ssh/id_xmss-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_7.9p1 Debian-10+deb10u2
debug1: ssh_exchange_identification: HTTP/1.0 400 Bad Request

 


debug1: ssh_exchange_identification: Content-Type: text/html; charset=UTF-8


debug1: ssh_exchange_identification: Referrer-Policy: no-referrer


debug1: ssh_exchange_identification: Content-Length: 1555


debug1: ssh_exchange_identification: Date: Wed, 30 Dec 2020 14:13:25 GMT


debug1: ssh_exchange_identification:


debug1: ssh_exchange_identification: <!DOCTYPE html>

debug1: ssh_exchange_identification: <html lang=en>

debug1: ssh_exchange_identification:   <meta charset=utf-8>

debug1: ssh_exchange_identification:   <meta name=viewport content="initial-scale=1, minimum-scale=1, width=device-width">

debug1: ssh_exchange_identification:   <title>Error 400 (Bad Request)!!1</title>

debug1: ssh_exchange_identification:   <style>

debug1: ssh_exchange_identification:     *{margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px}* > body{background:url(//www.google.com/images/errors/robot.png) 10
debug1: ssh_exchange_identification: 0% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}@media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.g
debug1: ssh_exchange_identification: oogle.com/images/branding/googlelogo/1x/googlelogo_color_150x54dp.png) no-repeat;margin-left:-5px}@media only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat 0
debug1: ssh_exchange_identification: % 0%/100% 100%;-moz-border-image:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) 0}}@media only screen and (-webkit-min-device-pixel-ratio:2){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_
debug1: ssh_exchange_identification: color_150x54dp.png) no-repeat;-webkit-background-size:100% 100%}}#logo{display:inline-block;height:54px;width:150px}

debug1: ssh_exchange_identification:   </style>

debug1: ssh_exchange_identification:   <a href=//www.google.com/><span id=logo aria-label=Google></span></a>

debug1: ssh_exchange_identification:   <p><b>400.</b> <ins>That\342\200\231s an error.</ins>

debug1: ssh_exchange_identification:   <p>Your client has issued a malformed or illegal request.  <ins>That\342\200\231s all we know.</ins>

ssh_exchange_identification: Connection closed by remote host

 

Here is another example on how to test remote host whether a certain service such as DNS (bind) or telnetd is enabled and listening on remote local network  IP with ssh

[root@pciserver: /home ]# ssh 192.168.1.200 -p 53 -v -oConnectTimeout=5
OpenSSH_7.9p1 Debian-10+deb10u2, OpenSSL 1.1.1g  21 Apr 2020
debug1: Connecting to 192.168.1.200 [192.168.1.200] port 53.
debug1: connect to address 192.168.1.200 port 53: Connection timed out
ssh: connect to host 192.168.1.200 port 53: Connection timed out

[root@server: /home ]# ssh 192.168.1.200 -p 23 -v -oConnectTimeout=5
OpenSSH_7.9p1 Debian-10+deb10u2, OpenSSL 1.1.1g  21 Apr 2020
debug1: Connecting to 192.168.1.200 [192.168.1.200] port 23.
debug1: connect to address 192.168.1.200 port 23: Connection timed out
ssh: connect to host 192.168.1.200 port 23: Connection timed out


But what if Linux server you have tow work on is so paranoid that you even the ssh client is absent? Well you can use anything else that is capable of doing a connectivity to remote port such as wget or curl. Some web servers or application servers usually have wget or curl as it is integral part for some local shell scripts doing various operation needed for proper services functioning or simply to test locally a local or remote listener services, if that's the case we can use curl to connect and get output of a remote service simulating a normal telnet connection like this:

host:~# curl -vv 'telnet://remote-server-host5:22'
* About to connect() to remote-server-host5 port 22 (#0)
*   Trying 10.52.67.21… connected
* Connected to aflpvz625 (10.52.67.21) port 22 (#0)
SSH-2.0-OpenSSH_5.3

Now lets test whether we can connect remotely to a local net remote IP's Qmail mail server with curls telnet simulation mode:

host:~#  curl -vv 'telnet://192.168.0.200:25'
* Expire in 0 ms for 6 (transfer 0x56066e5ab900)
*   Trying 192.168.0.200…
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x56066e5ab900)
* Connected to 192.168.0.200 (192.168.0.200) port 25 (#0)
220 This is Mail Pc-Freak.NET ESMTP

Fine it works, lets now test whether a remote server who has MySQL listener service on standard MySQL port TCP 3306 is reachable with curl

host:~#  curl -vv 'telnet://192.168.0.200:3306'
* Expire in 0 ms for 6 (transfer 0x5601fafae900)
*   Trying 192.168.0.200…
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x5601fafae900)
* Connected to 192.168.0.200 (192.168.0.200) port 3306 (#0)
Warning: Binary output can mess up your terminal. Use "–output -" to tell
Warning: curl to output it to your terminal anyway, or consider "–output
Warning: <FILE>" to save to a file.
* Failed writing body (0 != 107)
* Closing connection 0
root@pcfreak:/var/www/images#  curl -vv 'telnet://192.168.0.200:3306'
* Expire in 0 ms for 6 (transfer 0x5598ad008900)
*   Trying 192.168.0.200…
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x5598ad008900)
* Connected to 192.168.0.200 (192.168.0.200) port 3306 (#0)
Warning: Binary output can mess up your terminal. Use "–output -" to tell
Warning: curl to output it to your terminal anyway, or consider "–output
Warning: <FILE>" to save to a file.
* Failed writing body (0 != 107)
* Closing connection 0

As you can see the remote connection is returning binary data which is unknown to a standard telnet terminal thus to get the output received we need to pass curl suggested arguments.

host:~#  curl -vv 'telnet://192.168.0.200:3306' –output –
* Expire in 0 ms for 6 (transfer 0x55b205c02900)
*   Trying 192.168.0.200…
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x55b205c02900)
* Connected to 192.168.0.200 (192.168.0.200) port 3306 (#0)
g


The curl trick used to troubleshoot remote port to remote host from a Windows OS host which does not have telnet installed by default but have curl instead.

Also When troubleshooting vSphere Replication, it is often necessary to troubleshoot port connectivity as common Windows utilities are not available.
As Curl is available in the VMware vCenter Server Appliance command line interface.

On servers where curl is not there but you have wget is installed you can use it also to test a remote port

 

# wget -vv -O /dev/null http://google.com:554 –timeout=5
–2020-12-30 16:54:22–  http://google.com:554/
Resolving google.com (google.com)… 172.217.169.206, 2a00:1450:4017:80b::200e
Connecting to google.com (google.com)|172.217.169.206|:554… failed: Connection timed out.
Connecting to google.com (google.com)|2a00:1450:4017:80b::200e|:554… failed: Cannot assign requested address.
Retrying.

–2020-12-30 16:54:28–  (try: 2)  http://google.com:554/
Connecting to google.com (google.com)|172.217.169.206|:554… ^C

As evident from output the port 554 is filtered in google which is pretty normal.

If curl or wget is not there either as a final alternative you can either install some perl, ruby, python or bash script etc. that can opens a remote socket to the remote IP.

How to fix rkhunter checking dev for suspisiocus files, solve rkhunter checking if SSH root access is allowed warning

Friday, November 20th, 2020

rkhunter-logo

On a server if you have a rkhunter running and you suddenly you get some weird Warnings for suspicious files under dev, like show in in the screenshot and you're puzzled how comes this happened as so far it was not reported before the regular package patching update conducted …

root@haproxy-server ~]# rkhunter –check

rkhunter-warn-screenshot

To investigate further I've checked rkhunter produced log /var/log/rkhunter.log for a verobose message and found more specifics there on what is the exact files which rkhunter finds suspicious.
To further investigate what exactly are this suspicious files for or where, they're used for something on the system or in reality it is a hacker who hacked our supposibly PCI compliant system,
I've used the good old fuser command which is capable to show which system process is actively using a file. To have fuser report for each file from /var/log/rkhunter.log with below shell loop:

[root@haproxy-server ~]#  for i in $(tail -n 50 /var/log/rkhunter/rkhunter.log|grep -i /dev/shm|awk '{ print $2 }'|sed -e 's#:##g'); do fuser -v $i; done
                     BEN.        PID ZUGR.  BEFEHL
/dev/shm/qb-1783-1851-27-f1sTlC/qb-request-cpg-header:
                     root       1783 ….m corosync
                     hacluster   1851 ….m attrd
                     BEN.        PID ZUGR.  BEFEHL
/dev/shm/qb-1783-1844-26-Znk1UM/qb-event-quorum-data:
                     root       1783 ….m corosync
                     root       1844 ….m pacemakerd
                     BEN.        PID ZUGR.  BEFEHL
/dev/shm/qb-1783-1844-26-Znk1UM/qb-event-quorum-header:
                     root       1783 ….m corosync
                     root       1844 ….m pacemakerd
                     BEN.        PID ZUGR.  BEFEHL
/dev/shm/qb-1783-1844-26-Znk1UM/qb-response-quorum-data:
                     root       1783 ….m corosync
                     root       1844 ….m pacemakerd
                     BEN.        PID ZUGR.  BEFEHL
/dev/shm/qb-1783-1844-26-Znk1UM/qb-response-quorum-header:
                     root       1783 ….m corosync
                     root       1844 ….m pacemakerd
                     BEN.        PID ZUGR.  BEFEHL
/dev/shm/qb-1783-1844-26-Znk1UM/qb-request-quorum-data:
                     root       1783 ….m corosync
                     root       1844 ….m pacemakerd
                     BEN.        PID ZUGR.  BEFEHL
/dev/shm/qb-1783-1844-26-Znk1UM/qb-request-quorum-header:
                     root       1783 ….m corosync
                     root       1844 ….m pacemakerd
                     BEN.        PID ZUGR.  BEFEHL
/dev/shm/qb-1783-1844-25-oCdaKX/qb-event-cpg-data:
                     root       1783 ….m corosync
                     root       1844 ….m pacemakerd
                     BEN.        PID ZUGR.  BEFEHL
/dev/shm/qb-1783-1844-25-oCdaKX/qb-event-cpg-header:
                     root       1783 ….m corosync
                     root       1844 ….m pacemakerd
                     BEN.        PID ZUGR.  BEFEHL
/dev/shm/qb-1783-1844-25-oCdaKX/qb-response-cpg-data:
                     root       1783 ….m corosync
                     root       1844 ….m pacemakerd
                     BEN.        PID ZUGR.  BEFEHL
/dev/shm/qb-1783-1844-25-oCdaKX/qb-response-cpg-header:
                     root       1783 ….m corosync
                     root       1844 ….m pacemakerd
                     BEN.        PID ZUGR.  BEFEHL
/dev/shm/qb-1783-1844-25-oCdaKX/qb-request-cpg-data:
                     root       1783 ….m corosync
                     root       1844 ….m pacemakerd
                     BEN.        PID ZUGR.  BEFEHL
/dev/shm/qb-1783-1844-25-oCdaKX/qb-request-cpg-header:
                     root       1783 ….m corosync
                     root       1844 ….m pacemakerd
                     BEN.        PID ZUGR.  BEFEHL
/dev/shm/qb-1783-1844-24-GKyj3l/qb-event-cfg-data:
                     root       1783 ….m corosync
                     root       1844 ….m pacemakerd
                     BEN.        PID ZUGR.  BEFEHL
/dev/shm/qb-1783-1844-24-GKyj3l/qb-event-cfg-header:
                     root       1783 ….m corosync
                     root       1844 ….m pacemakerd
                     BEN.        PID ZUGR.  BEFEHL
/dev/shm/qb-1783-1844-24-GKyj3l/qb-response-cfg-data:
                     root       1783 ….m corosync
                     root       1844 ….m pacemakerd
                     BEN.        PID ZUGR.  BEFEHL
/dev/shm/qb-1783-1844-24-GKyj3l/qb-response-cfg-header:
                     root       1783 ….m corosync
                     root       1844 ….m pacemakerd
                     BEN.        PID ZUGR.  BEFEHL
/dev/shm/qb-1783-1844-24-GKyj3l/qb-request-cfg-data:
                     root       1783 ….m corosync
                     root       1844 ….m pacemakerd
                     BEN.        PID ZUGR.  BEFEHL
/dev/shm/qb-1783-1844-24-GKyj3l/qb-request-cfg-header:
                     root       1783 ….m corosync
                     root       1844 ….m pacemakerd


As you see from the output all the /dev/shm/qb/ files in question are currently opened by the corosync / pacemaker and necessery for proper work of the haproxy cluster processes running on the machines.
 

How to solve the /dev/ suspcisios files rkhunter warning?

To solve we need to tell rkhunter not check against this files this is done via  /etc/rkhunter.conf first I thought this is done by EXISTWHITELIST= but then it seems there is  a special option for rkhunter whitelisting /dev type of files only ALLOWDEVFILE.

Hence to resolve the warning for the upcoming planned early PCI audit and save us troubles we had to add on running OS which is CentOS Linux release 7.8.2003 (Core) in /etc/rkhunter.conf

ALLOWDEVFILE=/dev/shm/qb-*/qb-*

Re-run

# rkhunter –check

and Voila, the warning should be no more.

rkhunter-check-output

Another thing is on another machine the warnings produced by rkhunter were a bit different as rkhunter has mistakenly detected the root login is enabled where in reality PermitRootLogin was set to no in /etc/ssh/sshd_config

rkhunter-warning

As the problem was experienced on some machines and on others it was not.
I've done the standard boringconfig comparison we sysadmins do to tell
why stuff differs.
The result was on first machine where we had everything working as expected and
PermitRootLogin no was recognized the correct configuration was:

— SNAP —
#ALLOW_SSH_ROOT_USER=no
ALLOW_SSH_ROOT_USER=unset
— END —

On the second server where the problem was experienced the values was:

— SNAP —
#ALLOW_SSH_ROOT_USER=unset
ALLOW_SSH_ROOT_USER=no
— END —

Note that, the warning produced regarding the rsyslog remote logging is allowed is perfectly fine as, we had enabled remote logging to a central log server on the machines, this is done with:

This is done with config options under /etc/rsyslog.conf

# Configure Remote rsyslog logging server
*.* @remote-logging-server.com:514
*.* @remote-logging-server.com:514