Posts Tagged ‘DNS’

« Older Entries

How to Run Your Own Windows Domain Authentication on Linux

Thursday, October 2nd, 2025

samba-active-directory-win-tux-logo

 

Run Your Own Domain Authentication on Linux

Running your own domain authentication system on Linux can significantly enhance security and manageability in your IT environment. Whether you're setting up centralized login for a small network or a more complex domain environment, Linux provides powerful tools to become your own domain controller using open-source software.

In this guide, we’ll walk you through setting up Samba as an Active Directory (AD) Domain Controller on a Linux server.
These tutorial should work fine on Debian 12 (Bookworm), though it should work with minor modifications on pretty much most of recent Debs and deb based distros.

What is Domain Authentication?

Domain authentication allows users to log in to any authorized machine within a network using the same set of credentials. It provides centralized management of:

  • Users and groups
  • Computer accounts
  • Group policies
  • File and printer sharing
  • Access control

Microsoft's Active Directory is the most well-known implementation, but you can achieve similar functionality using Samba on Linux.

Pre-requirements

  • A fresh Linux installation (Ubuntu Server 22.04 LTS or Debian 12 recommended)
  • Static IP address
  • Root or sudo access
  • Domain name (e.g., mydomain.local)
 

1. Update System and Set proper Hostname

# apt update && sudo apt upgrade -y

# hostnamectl set-hostname dc1.mydomain.local


Add the hostname to /etc/hosts:

# vim /etc/hosts

Add the local network IP the SMB Domain controller will have locally on the machine:

192.168.1.100  dc1.mydomain.local dc1

 

2. Install Samba and Required Packages

# apt install samba krb5-config krb5-user winbind smbclient dnsutils -y

During the installation, you may be prompted for Kerberos configuration:

  • Default realm: MYDOMAIN.LOCAL
  • KDC: dc1.mydomain.local
  • Admin server: dc1.mydomain.local


samba-active-directory-raw-illustration

 

3. Provision Samba as a Domain Controller

First, stop any running Samba services:
 

# systemctl stop smbd nmbd winbind

# systemctl disable smbd nmbd winbind

Move default config:

# mv /etc/samba/smb.conf /etc/samba/smb.conf.bak

Now provision the domain:

# samba-tool domain provision –use-rfc2307 –interactive

Answer prompts:

  • Realm: MYDOMAIN.LOCAL
  • Domain: MYDOMAIN
  • Server role: dc
  • DNS backend: SAMBA_INTERNAL
  • Admin password: (choose a strong one)

Once done, configure Kerberos using the samba krb5.conf template file:

# mv /etc/krb5.conf /etc/krb5.conf.bak

# cp /var/lib/samba/private/krb5.conf /etc/

 

4. Start and Enable Samba AD Services

# systemctl unmask samba-ad-dc

# systemctl enable samba-ad-dc –now

Verify it’s working by running:

# samba-tool domain level show

Check Kerberos authentication is OK:

# kinit administrator

# klist

You should see a valid Kerberos ticket.

5. Configure DNS (Optional but Recommended)

If using SAMBA_INTERNAL DNS backend:

Check DNS resolution is OK:

# host -t A dc1.mydomain.local

# host -t SRV _kerberos._udp.mydomain.local

If you want clients to resolve domain names, configure them to use the Samba DC's IP as their DNS server.

6. Add Users and Join Client Machines

Add a new user:

# samba-tool user add your.samba.user

Join a Windows client:

  1. Go to System Properties → Computer Name → Change settings
  2. Click Domain, enter MYDOMAIN
  3. Authenticate with Administrator and the password you set
  4. Reboot

7. Managing the Domain

You can manage users, groups, and policies simply via commands or GUI interface or LDAP tools:

  • samba-tool (CLI)
  • RSAT tools on Windows (for GUI management)
  • via LDAP tools (if you have to stick to RFC2307)

Example commands:

# samba-tool user list

# samba-tool group list

# samba-tool user setpassword your.samba.user

8. Managing Samba AD Samba Linux Domain easily with UI
 

You can manage a Samba domain (especially when it's running as an Active Directory Domain Controller) via a web interface — but not directly through Samba itself, since it doesn't come with a built-in web UI.

Instead, you can integrate Samba with third-party web-based tools that provide management interfaces for:

  • Users and groups
  • Computer accounts
  • LDAP directory entries
  • Domain policies (to a limited extent)

Popular Web Interfaces to Manage a Samba Domain

Here are the most reliable options:

8.1. [Cockpit + 389 Directory Server or FreeIPA (for LDAP-based domains)]

  • Cockpit is a modern web admin interface for Linux servers.
  • When paired with FreeIPA, you can manage users, groups, policies, and more.
  • However, this is more suited for FreeIPA-based domains, not Samba AD.

✅ Great for: Linux-native domains
❌ Not compatible with Windows-style Samba AD

 

8.2. [LDAP Account Manager (LAM)] – RECOMMENDED FOR SAMBA + AD

Website: https://www.ldap-account-manager.org/

LDAP Account Manager (LAM) is one of the best tools to manage a Samba domain via LDAP, especially when:

  • You use Samba in AD DC mode with RFC2307 extensions (for Unix attributes)
  • Or, you're using Samba as a member server with an external LDAP backend

Features:

  • Web-based GUI to manage:

     

     

    • Users and groups
    • Samba-specific attributes (like SID, RID, home directories)
    • POSIX and Windows-compatible accounts
  • Can bind directly to the Samba LDAP directory

Authentication: Admin binds via LDAP (either over plain or TLS)

✅ Works with Samba AD (with some config)
✅ Handles Samba3/4 user schemas
✅ Active development and documentation

 

8.3. Samba Web Administration Tool (SWAT) ❌ Deprecated

SWAT was the original web interface for Samba but:

  • It was deprecated and removed from Samba after version 4.1
  • It's no longer secure or maintained
  • Not suitable for Samba AD DC environments

Recommendation: Do not use SWAT

8.4. Webmin (Partial Support)

  • Webmin is a general Linux web admin tool
  • It has a Samba module, but:

     

     

    • Designed for traditional Samba file sharing (not AD/DC mode)
    • Cannot manage Samba AD users/groups
    • Doesn’t interact with samba-tool or the AD schema

✅ Works for standalone Samba file servers
❌ Not suitable for Samba AD DCs

Can You really Use RSAT Instead ?

If you want full Active Directory-style control (like Group Policy, OU structure, DNS, etc.), the best GUI tool is actually RSAT (Remote Server Administration Tools) on Windows
but for that of course you will have to have an own Windows Server setup especailly for it.

  • Connects to your Samba AD DC
  • Fully supports:

     

     

    • Users and groups
    • Group Policy Objects (GPO)
    • DNS management (if using internal Samba DNS)

Install RSAT on a Windows machine and run dsa.msc (Active Directory Users and Computers).

✅ Officially supported
✅ Full compatibility with Samba AD
❌ Requires a Windows machine

Summary: Web UI for Samba Domain Management

 

Tool

Works with Samba AD DC?

Features

Notes

LDAP Account Manager (LAM)

Yes

User/group management

Best web option

Cockpit + FreeIPA

❌ No (not Samba AD)

Excellent for FreeIPA domains

Not compatible with Samba AD

Webmin

❌ Not fully

File shares only

No AD/DC management

RSAT (Windows)

✅ Yes

Full AD management

Not web-based

Recommendation

If you're running a Samba AD DC and want a web-based interface:

  • Use LAM (LDAP Account Manager) for basic account management
  • Use RSAT tools on Windows for full domain administration
  • Avoid SWAT and Webmin for this purpose

Security Considerations

  • Ensure firewall allows relevant ports (e.g., 53, 88, 389, 445, etc.) with Iptables / firewalld or whatever firewall solution you have present on the server and in the Network in which you hosted the server
  • Keep the system updated
  • Use secure passwords and rotate them regularly
  • Consider setting up replication if high availability is needed

Conclusion

Running your own domain authentication system on Linux using Samba is a powerful way to control user access in a centralized manner. It’s ideal for small to mid-sized networks, homelabs, or even enterprise environments looking for a cost-effective alternative to Windows Server.

With Samba acting as your domain controller, you can enjoy the benefits of centralized authentication, integrated DNS, and a high degree of compatibility with Windows clients — all while staying in the open-source ecosystem.

 

References

  • Samba Wiki: Setting up Samba as an AD Domain Controller
  • man samba-tool
  • man smb.conf


Notes and things to consider:

/var/lib/samba/private/krb5.conf file is generated only after you provision Samba as an Active Directory (AD) Domain Controller using:

# samba-tool domain provision

After provisioning, Samba creates a custom Kerberos config at:

/var/lib/samba/private/krb5.conf

 

This is true for both Debian and Ubuntu because it's handled by the Samba package itself, not the distro.

Why use that krb5.conf instead of Debian's default?

Well because:

The default /etc/krb5.conf on Debian isn't tailored for Samba AD.
The one Samba generates includes correct realm, KDC, and admin server settings.
It avoids subtle issues like failed kinit or broken Kerberos trust.

So you copy it over Debian’s default:

 

Gotchas on Debian to be aware of

Do not install samba via tasksel (like tasksel's “Samba file server” role), as it sets up a traditional SMB server, not AD.

Only use samba-tool domain provision if you're setting up AD DC.

Debian sometimes separates systemd services (e.g., samba-ad-dc might not be enabled by default). So make sure to enable samba-ad-dc instead of smbd/nmbd.

 

Tags: , , , , , , , , , ,
Posted in Everyday Life, Linux, Samba, System Administration, Various | No Comments »

How to Install and use FreeIPA forcentralized SSO authention on Linux computer domain

Wednesday, October 1st, 2025

freeipa-gnu-linux-free-sso-solution-logo

FreeIPA is a popular open-source identity management solution that centralizes user, host, and service authentication for Linux environments. It combines LDAP, Kerberos, DNS, and certificate management into a single platform, making it easier to manage large Linux deployments securely.

In this article, we’ll cover how to install FreeIPA on a Linux server, perform initial configuration, and start using it for basic user management.

Prerequisites

  • A clean Linux server (CentOS, RHEL, Fedora, or similar)
  • Root or sudo access
  • A fully qualified domain name (FQDN) for your server (e.g., ipa.example.com)
  • Proper DNS setup (recommended but can be configured during installation)
     

1. Update system to the latest

Start by updating your system to ensure all packages are current.
 

# dnf update -y


2. Install FreeIPA Server Packages

Install the FreeIPA server and its dependencies:

# dnf install -y ipa-server ipa-server-dns

  • ipa-server-dns is optional but recommended if you want FreeIPA to manage DNS for your domain.

3. Configure FreeIPA server

Run the FreeIPA installation script to configure the server. Replace ipa.example.com with your actual server hostname.

sudo ipa-server-install

You will be prompted for:

  • Realm name: Usually uppercase of your domain, e.g., EXAMPLE.COM
  • Directory Manager password: LDAP admin password
  • IPA admin password: FreeIPA admin user password
  • DNS configuration: Enable if you want FreeIPA to manage DNS

Sample configuration flow:

Realm name: EXAMPLE.COM

DNS domain name: example.com

Server host name: ipa.example.com

Directory Manager password: [choose a strong password]

IPA admin password: [choose a strong password]

Do you want to configure integrated DNS (BIND)? [yes/no]: yes

The installer will set up Kerberos, LDAP, the CA, DNS (if chosen), and the Web UI.

4. Start and Enable FreeIPA Services

The installer usually starts services automatically, but you can verify with:

# systemctl status ipa

Enable the service to start on boot:
 

# systemctl enable ipa


5. Access FreeIPA Web Interface

Open your browser and navigate to:

https://ipa.example.com/ipa/ui/

Log in using the admin username and the password you set during installation.

6. Add Users and Groups

You can manage users and groups either via the Web UI or the CLI.

Using the CLI:

Add a new user:

# ipa user-add johndoe –first=John –last=Doe –email=johndoe@example.com

Set a password for the new user:

# ipa passwd johndoe


Add a new group:

# ipa group-add developers –desc="Development Team"


Add user to the group:

# ipa group-add-member developers –users=johndoe


7. Join Client Machines to the FreeIPA Domain
 

On a client machine, install the client packages:

# dnf install -y ipa-client

Run the client setup:

# ipa-client-install –mkhomedir

Follow the prompts to join the client to the FreeIPA domain.

8. Test Authentication
 

Try logging into the client machine with the FreeIPA user you created:
 

# ssh username@client-machine-host.com

You should be able to authenticate using the FreeIPA credentials.
 

Conclusion


You now have a basic FreeIPA server up and running, managing users and authentication across your Linux network. FreeIPA simplifies identity management by providing a centralized, secure, and integrated solution. From here, you can explore advanced features like role-based access control, host-based access control, and certificate management.

 

Here's a practical example of how FreeIPA can be used in a real-world Linux environment.

Scenario: Centralized Authentication in a DevOps Environment
 

Tech Problem

Lets say you are managing a growing team of DevOps engineers and developers across multiple Linux servers (e.g., for CI/CD, staging, and production). Manually creating and maintaining user accounts, SSH keys, and sudo permissions on each server is:

  • Time-consuming
  • Error-prone
  • A security risk (inconsistent policies, orphaned accounts)

Solution: Use FreeIPA to Centralize Identity & Access Management

By deploying FreeIPA, you can:

  • Create user accounts once and manage them centrally
  • Enforce SSO across servers using Kerberos
  • Automatically apply sudo rules, group permissions, and access control policies
  • Easily revoke access for offboarded employees
  • Use host-based access control (HBAC) to control who can log in to what
     

Solution Walkthrough
 

1. Set up FreeIPA server

  • Installed on: ipa.internal.example.com
  • Domain: internal.example.com
  • Realm: INTERNAL.EXAMPLE.COM


2. Add User Accounts

Let's add two users: alice (developer) and bob (DevOps).
 

# ipa user-add alice –first=Alice –last=Smith –email=alice@internal.example.com

# ipa user-add bob –first=Bob –last=Jones –email=bob@internal.example.com

# ipa passwd alice

# ipa passwd bob


3. Create Groups and Roles necessery

Create functional groups for managing permissions.
 

# ipa group-add developers –desc="Developers Team"

# ipa group-add devops –desc="DevOps Team"

# ipa group-add-member developers –users=alice

# ipa group-add-member devops –users=bob

4. Configure Sudo Rules

Let’s allow DevOps team members to use sudo on all servers:
 

# ipa sudorule-add devops-sudo –cmdcat=all

# ipa sudorule-add-user devops-sudo –groups=devops

# ipa sudorule-add-host devops-sudo –hostgroups=all

5. Control Access with HBAC Rules

Let’s say:

  • Developers can access dev and staging servers
  • DevOps can access all servers

# Create host groups
 

# ipa hostgroup-add dev-servers –desc="Development Servers"

# ipa hostgroup-add staging-servers –desc="Staging Servers"

 

# Add hosts to groups
 

# ipa hostgroup-add-member dev-servers –hosts=dev1.internal.example.com

# ipa hostgroup-add-member staging-servers –hosts=staging1.internal.example.com

 

# HBAC rule for developers

# ipa hbacrule-add allow-developers

# ipa hbacrule-add-user allow-developers –groups=developers

# ipa hbacrule-add-host allow-developers –hostgroups=dev-servers

# ipa hbacrule-add-host allow-developers –hostgroups=staging-servers

# ipa hbacrule-add-service allow-developers –hbacsvcs=sshd

 

# HBAC rule for DevOps (all access)

# ipa hbacrule-add allow-devops

# ipa hbacrule-add-user allow-devops –groups=devops

# ipa hbacrule-add-host allow-devops –hostgroups=all

# ipa hbacrule-add-service allow-devops –hbacsvcs=sshd


6. Join Client Servers to FreeIPA

On each Linux server (e.g., dev1, staging1, prod1), run:

 

# ipa-client-install –mkhomedir –server=ipa.internal.example.com –domain=internal.example.com

 

Now, user alice can log in to dev1 and staging1, but not prod1. bob can log in to all servers and use sudo.

7. What Happens When Alice Leaves the Company?

Just disable the user in FreeIPA:

# ipa user-disable alice

This immediately revokes her access across all servers — no need to touch individual machines.

Benefits in This Example

Feature

Outcome

Centralized user management

No need to manually create accounts on every server

Group-based sudo

DevOps has privileged access, others don’t

Access control

Developers only access dev/staging, not prod

Kerberos SSO

Secure, passwordless SSH with ticketing

Auditing

Central logs of who accessed what and when

Quick offboarding

Instant account disablement from a single location

Summary

FreeIPA is not just a replacement for LDAP — it's a full-blown identity and access management solution tailored for Linux systems. In this practical example, it brings enterprise-grade access control, authentication, and user management to a DevOps workflow with minimal friction.

Tags: , , , , , , , , , , , , , , , , ,
Posted in Computer Security, Linux, System Administration, Various | No Comments »

DNS Monitoring: Check and Alert if DNS nameserver resolver of Linux machine is not properly resolving shell script. Monitor if /etc/resolv.conf DNS runs Okay

Thursday, March 14th, 2024

linux-monitor-check-dns-is-resolving-fine

If you happen to have issues occasionally with DNS resolvers and you want to keep up an eye on it and alert if DNS is not properly resolving Domains, because sometimes you seem to have issues due to network disconnects, disturbances (modifications), whatever and you want to have another mean to see whether a DNS was reachable or unreachable for a time, here is a little bash shell script that does the "trick".

Script work mechacnism is pretty straight forward as you can see we check what are the configured nameservers if they properly resolve and if they're properly resolving we write to log everything is okay, otherwise we write to the log DNS is not properly resolvable and send an ALERT email to preconfigured Email address.

Below is the check_dns_resolver.sh script:

 

#!/bin/bash
# Simple script to Monitor DNS set resolvers hosts for availability and trigger alarm  via preset email if any of the nameservers on the host cannot resolve
# Use a configured RESOLVE_HOST to try to resolve it via available configured nameservers in /etc/resolv.conf
# if machines are not reachable send notification email to a preconfigured email
# script returns OK 1 if working correctly or 0 if there is issue with resolving $RESOLVE_HOST on $SELF_HOSTNAME and mail on $ALERT_EMAIL
# output of script is to be kept inside DNS_status.log

ALERT_EMAIL='your.email.address@email-fqdn.com';
log=/var/log/dns_status.log;
TIMEOUT=3; DNS=($(grep -R nameserver /etc/resolv.conf | cut -d ' ' -f2));  

SELF_HOSTNAME=$(hostname –fqdn);
RESOLVE_HOST=$(hostname –fqdn);

for i in ${DNS[@]}; do dns_status=$(timeout $TIMEOUT nslookup $RESOLVE_HOST  $i); 

if [[ “$?” == ‘0’ ]]; then echo "$(date "+%y.%m.%d %T") $RESOLVE_HOST $i on host $SELF_HOST OK 1" | tee -a $log; 
else 
echo "$(date "+%y.%m.%d %T")$RESOLVE_HOST $i on host $SELF_HOST NOT_OK 0" | tee -a $log; 

echo "$(date "+%y.%m.%d %T") $RESOLVE_HOST $i DNS on host $SELF_HOST resolve ERROR" | mail -s "$RESOLVE_HOST /etc/resolv.conf $i DNS on host $SELF_HOST resolve ERROR";

fi

 done

Download check_dns_resolver.sh here set the script to run via a cron job every lets say 5 minutes, for example you can set a cronjob like this:
 

# crontab -u root -e
*/5 * * * *  check_dns_resolver.sh 2>&1 >/dev/null

 

Then Voila, check the log /var/log/dns_status.log if you happen to run inside a service downtime and check its output with the rest of infrastructure componets, network switch equipment, other connected services etc, that should keep you in-line to proof during eventual RCA (Root Cause Analysis) if complete high availability system gets down to proof your managed Linux servers was not the reason for the occuring service unavailability.

A simplified variant of the check_dns_resolver.sh can be easily integrated to do Monitoring with Zabbix userparameter script and DNS Check Template containing few Triggers, Items and Action if I have time some time in the future perhaps, I'll blog a short article on how to configure such DNS zabbix monitoring, the script zabbix variant of the DNS monitor script is like this:

[root@linux-server bin]# cat check_dns_resolver.sh 
#!/bin/bash
TIMEOUT=3; DNS=($(grep -R nameserver /etc/resolv.conf | cut -d ' ' -f2));  for i in ${DNS[@]}; do dns_status=$(timeout $TIMEOUT nslookup $(hostname –fqdn) $i); if [[ “$?” == ‘0’ ]]; then echo "$i OK 1"; else echo "$i NOT OK 0"; fi; done
[root@linux-server bin]#


Hope this article, will help someone to improve his Unix server Infrastucture monitoring.

Enjoy and Cheers !

Tags: , , , , , , , , , , , , , , , , , , , , , , , , , ,
Posted in Bash Scripting, DNS, Linux, Monitoring, System Administration | No Comments »

Hack: Using ssh / curl or wget to test TCP port connection state to remote SSH, DNS, SMTP, MySQL or any other listening service in PCI environment servers

Wednesday, December 30th, 2020

using-curl-ssh-wget-to-test-tcp-port-opened-or-closed-for-web-mysql-smtp-or-any-other-linstener-in-pci-linux-logo

If you work on PCI high security environment servers in isolated local networks where each package installed on the Linux / Unix system is of importance it is pretty common that some basic stuff are not there in most cases it is considered a security hole to even have a simple telnet installed on the system. I do have experience with such environments myself and thus it is pretty daunting stuff so in best case you can use something like a simple ssh client if you're lucky and the CentOS / Redhat / Suse Linux whatever distro has openssh-client package installed.
If you're lucky to have the ssh onboard you can use telnet in same manner as netcat or the swiss army knife (nmap) network mapper tool to test whether remote service TCP / port is opened or not. As often this is useful, if you don't have access to the CISCO / Juniper or other (networ) / firewall equipment which is setting the boundaries and security port restrictions between networks and servers.

Below is example on how to use ssh client to test port connectivity to lets say the Internet, i.e.  Google / Yahoo search engines.
 

[root@pciserver: /home ]# ssh -oConnectTimeout=3 -v google.com -p 23
OpenSSH_7.9p1 Debian-10+deb10u2, OpenSSL 1.1.1g  21 Apr 2020
debug1: Connecting to google.com [172.217.169.206] port 23.
debug1: connect to address 172.217.169.206 port 23: Connection timed out
debug1: Connecting to google.com [2a00:1450:4017:80b::200e] port 23.
debug1: connect to address 2a00:1450:4017:80b::200e port 23: Cannot assign requested address
ssh: connect to host google.com port 23: Cannot assign requested address
root@pcfreak:/var/www/images# ssh -oConnectTimeout=3 -v google.com -p 80
OpenSSH_7.9p1 Debian-10+deb10u2, OpenSSL 1.1.1g  21 Apr 2020
debug1: Connecting to google.com [172.217.169.206] port 80.
debug1: connect to address 172.217.169.206 port 80: Connection timed out
debug1: Connecting to google.com [2a00:1450:4017:807::200e] port 80.
debug1: connect to address 2a00:1450:4017:807::200e port 80: Cannot assign requested address
ssh: connect to host google.com port 80: Cannot assign requested address
root@pcfreak:/var/www/images# ssh google.com -p 80
ssh_exchange_identification: Connection closed by remote host
root@pcfreak:/var/www/images# ssh google.com -p 80 -v -oConnectTimeout=3
OpenSSH_7.9p1 Debian-10+deb10u2, OpenSSL 1.1.1g  21 Apr 2020
debug1: Connecting to google.com [172.217.169.206] port 80.
debug1: connect to address 172.217.169.206 port 80: Connection timed out
debug1: Connecting to google.com [2a00:1450:4017:80b::200e] port 80.
debug1: connect to address 2a00:1450:4017:80b::200e port 80: Cannot assign requested address
ssh: connect to host google.com port 80: Cannot assign requested address
root@pcfreak:/var/www/images# ssh google.com -p 80 -v -oConnectTimeout=5
OpenSSH_7.9p1 Debian-10+deb10u2, OpenSSL 1.1.1g  21 Apr 2020
debug1: Connecting to google.com [142.250.184.142] port 80.
debug1: connect to address 142.250.184.142 port 80: Connection timed out
debug1: Connecting to google.com [2a00:1450:4017:80c::200e] port 80.
debug1: connect to address 2a00:1450:4017:80c::200e port 80: Cannot assign requested address
ssh: connect to host google.com port 80: Cannot assign requested address
root@pcfreak:/var/www/images# ssh google.com -p 80 -v
OpenSSH_7.9p1 Debian-10+deb10u2, OpenSSL 1.1.1g  21 Apr 2020
debug1: Connecting to google.com [172.217.169.206] port 80.
debug1: Connection established.
debug1: identity file /root/.ssh/id_rsa type 0
debug1: identity file /root/.ssh/id_rsa-cert type -1
debug1: identity file /root/.ssh/id_dsa type -1
debug1: identity file /root/.ssh/id_dsa-cert type -1
debug1: identity file /root/.ssh/id_ecdsa type -1
debug1: identity file /root/.ssh/id_ecdsa-cert type -1
debug1: identity file /root/.ssh/id_ed25519 type -1
debug1: identity file /root/.ssh/id_ed25519-cert type -1
debug1: identity file /root/.ssh/id_xmss type -1
debug1: identity file /root/.ssh/id_xmss-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_7.9p1 Debian-10+deb10u2
debug1: ssh_exchange_identification: HTTP/1.0 400 Bad Request

 


debug1: ssh_exchange_identification: Content-Type: text/html; charset=UTF-8


debug1: ssh_exchange_identification: Referrer-Policy: no-referrer


debug1: ssh_exchange_identification: Content-Length: 1555


debug1: ssh_exchange_identification: Date: Wed, 30 Dec 2020 14:13:25 GMT


debug1: ssh_exchange_identification:


debug1: ssh_exchange_identification: <!DOCTYPE html>

debug1: ssh_exchange_identification: <html lang=en>

debug1: ssh_exchange_identification:   <meta charset=utf-8>

debug1: ssh_exchange_identification:   <meta name=viewport content="initial-scale=1, minimum-scale=1, width=device-width">

debug1: ssh_exchange_identification:   <title>Error 400 (Bad Request)!!1</title>

debug1: ssh_exchange_identification:   <style>

debug1: ssh_exchange_identification:     *{margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px}* > body{background:url(//www.google.com/images/errors/robot.png) 10
debug1: ssh_exchange_identification: 0% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}@media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.g
debug1: ssh_exchange_identification: oogle.com/images/branding/googlelogo/1x/googlelogo_color_150x54dp.png) no-repeat;margin-left:-5px}@media only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat 0
debug1: ssh_exchange_identification: % 0%/100% 100%;-moz-border-image:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) 0}}@media only screen and (-webkit-min-device-pixel-ratio:2){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_
debug1: ssh_exchange_identification: color_150x54dp.png) no-repeat;-webkit-background-size:100% 100%}}#logo{display:inline-block;height:54px;width:150px}

debug1: ssh_exchange_identification:   </style>

debug1: ssh_exchange_identification:   <a href=//www.google.com/><span id=logo aria-label=Google></span></a>

debug1: ssh_exchange_identification:   <p><b>400.</b> <ins>That\342\200\231s an error.</ins>

debug1: ssh_exchange_identification:   <p>Your client has issued a malformed or illegal request.  <ins>That\342\200\231s all we know.</ins>

ssh_exchange_identification: Connection closed by remote host

 

Here is another example on how to test remote host whether a certain service such as DNS (bind) or telnetd is enabled and listening on remote local network  IP with ssh

[root@pciserver: /home ]# ssh 192.168.1.200 -p 53 -v -oConnectTimeout=5
OpenSSH_7.9p1 Debian-10+deb10u2, OpenSSL 1.1.1g  21 Apr 2020
debug1: Connecting to 192.168.1.200 [192.168.1.200] port 53.
debug1: connect to address 192.168.1.200 port 53: Connection timed out
ssh: connect to host 192.168.1.200 port 53: Connection timed out

[root@server: /home ]# ssh 192.168.1.200 -p 23 -v -oConnectTimeout=5
OpenSSH_7.9p1 Debian-10+deb10u2, OpenSSL 1.1.1g  21 Apr 2020
debug1: Connecting to 192.168.1.200 [192.168.1.200] port 23.
debug1: connect to address 192.168.1.200 port 23: Connection timed out
ssh: connect to host 192.168.1.200 port 23: Connection timed out


But what if Linux server you have tow work on is so paranoid that you even the ssh client is absent? Well you can use anything else that is capable of doing a connectivity to remote port such as wget or curl. Some web servers or application servers usually have wget or curl as it is integral part for some local shell scripts doing various operation needed for proper services functioning or simply to test locally a local or remote listener services, if that's the case we can use curl to connect and get output of a remote service simulating a normal telnet connection like this:

host:~# curl -vv 'telnet://remote-server-host5:22'
* About to connect() to remote-server-host5 port 22 (#0)
*   Trying 10.52.67.21… connected
* Connected to aflpvz625 (10.52.67.21) port 22 (#0)
SSH-2.0-OpenSSH_5.3

Now lets test whether we can connect remotely to a local net remote IP's Qmail mail server with curls telnet simulation mode:

host:~#  curl -vv 'telnet://192.168.0.200:25'
* Expire in 0 ms for 6 (transfer 0x56066e5ab900)
*   Trying 192.168.0.200…
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x56066e5ab900)
* Connected to 192.168.0.200 (192.168.0.200) port 25 (#0)
220 This is Mail Pc-Freak.NET ESMTP

Fine it works, lets now test whether a remote server who has MySQL listener service on standard MySQL port TCP 3306 is reachable with curl

host:~#  curl -vv 'telnet://192.168.0.200:3306'
* Expire in 0 ms for 6 (transfer 0x5601fafae900)
*   Trying 192.168.0.200…
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x5601fafae900)
* Connected to 192.168.0.200 (192.168.0.200) port 3306 (#0)
Warning: Binary output can mess up your terminal. Use "–output -" to tell
Warning: curl to output it to your terminal anyway, or consider "–output
Warning: <FILE>" to save to a file.
* Failed writing body (0 != 107)
* Closing connection 0
root@pcfreak:/var/www/images#  curl -vv 'telnet://192.168.0.200:3306'
* Expire in 0 ms for 6 (transfer 0x5598ad008900)
*   Trying 192.168.0.200…
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x5598ad008900)
* Connected to 192.168.0.200 (192.168.0.200) port 3306 (#0)
Warning: Binary output can mess up your terminal. Use "–output -" to tell
Warning: curl to output it to your terminal anyway, or consider "–output
Warning: <FILE>" to save to a file.
* Failed writing body (0 != 107)
* Closing connection 0

As you can see the remote connection is returning binary data which is unknown to a standard telnet terminal thus to get the output received we need to pass curl suggested arguments.

host:~#  curl -vv 'telnet://192.168.0.200:3306' –output –
* Expire in 0 ms for 6 (transfer 0x55b205c02900)
*   Trying 192.168.0.200…
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x55b205c02900)
* Connected to 192.168.0.200 (192.168.0.200) port 3306 (#0)
g


The curl trick used to troubleshoot remote port to remote host from a Windows OS host which does not have telnet installed by default but have curl instead.

Also When troubleshooting vSphere Replication, it is often necessary to troubleshoot port connectivity as common Windows utilities are not available.
As Curl is available in the VMware vCenter Server Appliance command line interface.

On servers where curl is not there but you have wget is installed you can use it also to test a remote port

 

# wget -vv -O /dev/null http://google.com:554 –timeout=5
–2020-12-30 16:54:22–  http://google.com:554/
Resolving google.com (google.com)… 172.217.169.206, 2a00:1450:4017:80b::200e
Connecting to google.com (google.com)|172.217.169.206|:554… failed: Connection timed out.
Connecting to google.com (google.com)|2a00:1450:4017:80b::200e|:554… failed: Cannot assign requested address.
Retrying.

–2020-12-30 16:54:28–  (try: 2)  http://google.com:554/
Connecting to google.com (google.com)|172.217.169.206|:554… ^C

As evident from output the port 554 is filtered in google which is pretty normal.

If curl or wget is not there either as a final alternative you can either install some perl, ruby, python or bash script etc. that can opens a remote socket to the remote IP.

Tags: , , , , , , , , , , , , , , , , , , , , ,
Posted in Curious Facts, Hacks, System Administration, Various | No Comments »

Improve DNS lookup domain resolve speed on Linux / UNIX servers through /etc/resolv.conf timeout, attempts, rorate options

Thursday, February 27th, 2020

improve-dns-lookup-speed-on-Linux-UNIX-servers-resolv.conf-change-dns-settings-linux
If you're an performance optimization freak and you want to optimize your Linux servers to perform better in terms of DNS resolve slowness because of failing DNS resolve queries due to Domain Name Server request overload or due to Denial of Service attack towards it. It might be interesting to mention about some little known functionalities of /etc/resolv.conf described in the manual page.

The defined nameservers under /etc/resolv.conf are queried one by one waiting for responce of the sent DNS resolve request if it is not replied from the first one for some time, the 2nd one is queried until a responce is received by any of the defined nameserver IPs

A default /etc/resolv.conf on a new Linux server install looks something like this:
 

nameserver      10.10.8.1
nameserver      10.10.8.2
nameserver      10.10.8.3
search          sub.subdomain.com subdom.dom.domain.com


However one thing is that defined if NS1 dies out due to anything, it takes timeout time until the second or 3rd working one takes over to resolve the query.
This is controlled by the timeout value.

Below is description from man page

timeout:n
sets the amount of time the resolver will wait for a
response from a remote name server before retrying the
query via a different name server.  Measured in
seconds, the default is RES_TIMEOUT (currently 5, see
<resolv.h>).  The value for this option is silently
capped to 30.
 

  • In other words Timeout value is time to resolving IP address from hostname through DNS server,timeout option is to reduce hostname lookup time

As you see from manual default is 5 seconds which is quite high, thus reducing the value to 3 secs or even 1 seconds is a good sysadmin practice IMHO.

Another value that could be tuned in /etc/resolv.conf is attempts value below is what the manual says about it: 
 

attempts:n
                     Sets the number of times the resolver will send a query to its name servers before giving up and returning an error to the calling application.  The default is RES_DFLRETRY (cur‐
                     rently 2, see <resolv.h>).  The value for this option is silently capped to 5.

 

 

  • This means default behaviour on a failing DNS query resolve is to try to resend the DNS resolve request to the failing nameserver 5 more times, that is quite high thus it is a good practice from my experience to reduce it to something as 2 or 1


Another very useful resolv.conf value is rotate
The default behavior of how DNS outgoing Domain requests are handled is to use only the primary defined DNS, instead if you need to do a load balancing in a round-robin manner add to conf rotate option.

The final /etc/resolv.conf optimized would look like so:

 

linux# cat /etc/resolv.conf

nameserver      10.10.8.1
nameserver      10.10.8.2
nameserver      10.10.8.3
search          sub.subdomain.com subdom.dom.domain.com
options ndots:1
options timeout:1
options attempts:1
options rotate


The search opt. placement is also important to be placed in the right location in the file. The correct placement is after the nameservers defined, I have to say in older Linux distributions the correct placement of search option was to be on top of resolv.conf.

Note that this configuration is good and fits not only Linux but also is a good DNS lookup optimization speed on other UNIX derivatives such as FreeBSD / NetBSD as well as other Proprietary OS UNIX machines running IBM AIX etc.

On Linux it is also possible to place the options given in one single line like so, below is the config I have on my www.pc-freak.net running Lenovo server:

 

domain www.pc-freak.net
search www.pc-freak.net
#nameserver 192.168.0.1
nameserver 127.0.0.1
nameserver 83.228.92.2
nameserver 8.8.8.8
nameserver 83.228.92.1
nameserver 208.67.222.222
nameserver 208.67.220.220
options timeout:2 attempts:1 rotate

 

When is /etc/hosts record venerated and when is /etc/resolv.conf DNS defined queried for a defined DNS host?

 

One important thing to know when dealing with /etc/resolv.conf  is what happens if a Name domain is defined in both /etc/hosts and /etc/resolv.conf.
For example you have a www.pc-freak.net domain record in /etc/hosts to a certain domain but the DNS nameserver 8.8.8.8 in Google has a record to an IP that is the real IP 83.228.93.76

 

83.228.93.75 irc.www.pc-freak.net www.pc-freak.net pcfreak.biz www.pc-freak.net pcfreak.us services.www.pc-freak.net jabber.www.pc-freak.net

 

# dig @8.8.8.8 www.pc-freak.net

; <<>> DiG 9.11.5-P4-5.1-Debian <<>> @8.8.8.8 www.pc-freak.net
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 54656
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;www.pc-freak.net.                  IN      A

;; ANSWER SECTION:
www.pc-freak.net.           3599    IN      A       83.228.93.76

;; Query time: 40 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: чт фев 27 18:04:23 EET 2020
;; MSG SIZE  rcvd: 57
 

 

  • Which of the 2 different IPs will the applications installed on the server such as Apache / Squid / MySQL / tinyproxy for their DNS resolve operations?

 


Now it is time to say few words about /etc/nsswitch.conf (The Nameserver switching configuration file). This file defines the DNS resolve file used order in which the Operationg System does IP to domain translation and backwards.
 

# grep -i hosts: /etc/nsswitch.conf

hosts:          files dns myhostname

As you can see first the local defined in files like /etc/hosts record is venerated when resolving, then it is the externally configured DNS resolver IPs from /etc/resolv.conf.

nsswitch.conf  is used also for defining where the OS will look up for user / passwd (e.g. login credentials) on login, on systems which are having an LDAP authentication via the sssd (system security services daemon) via definitions like:

 

passwd:     files sss
shadow:     files sss
group:      files sss


E.g. the user login will be first try to read from local /etc/passwd , /etc/shadow , /etc/groups and if no matched record is found then the LDAP service the sssd is queried.

Tags: , , , , , ,
Posted in Everyday Life, FreeBSD, Linux, System Administration | No Comments »

Mail send from command line on Linux and *BSD servers – useful for scripting

Monday, September 10th, 2018

mail-send-email-from-command-line-on-linux-and-freebsd-operating-systems-logo

Historically Email sending has been very different from what most people use it in the Office, there was no heavy Email clients such as Outlook Express no MX Exchange, no e-mail client capabilities for Calendar and Meetings schedule as it is in most of the modern corporate offices that depend on products such as Office 365 (I would call it a connectedHell 365 days a year !).

There was no free webmail and pop3 / imap providers such as Mail.Yahoo.com, Gmail.com, Hotmail.com, Yandex.com, RediffMail, Mail.com the innumerous lists goes and on.
Nope back in the day emails were doing what they were originally supposed to like the post services in real life simply send and receive messages.

For those who remember that charming times, people used to be using BBS-es (which were basicly a shared set-up home system as a server) or some of the few University Internal Email student accounts or by crazy sysadmins who received their notification and warnings logs about daemon (services) messages via local DMZ-ed network email servers and it was common to read the email directly with mail (mailx) text command or custom written scripts … It was not uncommon also that mailx was used heavily to send notification messages on triggered events from logs. Oh life was simple and clear back then, and even though today the email could be used in a similar fashion by hard-core old school sysadmins and Dev Ops / simple shell scriptings tasks or report cron jobs such usage is already in the deep history.

The number of ways one could send email in text format directly from the GNU / Linux / *BSD server to another remote mail MTA node (assuming it had properly configured Relay server be it Exim or Postifix) were plenty.

In this article I will try to rewind back some of the UNIX history by pinpointing a few of the most common ways, one used to send quick emails directly from a remote server connection terminal or lets say a cheap VPS few cents server, through something like (SSH or Telnet) etc.
 

1. Using the mail command client (part of bsd-mailx on Debian).
 

In my previous article Linux: "bash mail command not found" error fix
I ended the article with a short explanation on how this is done but I will repeat myself one more time here for the sake of clearness of this article.

root@linux:~# echo "Your Sample Message Body" | mail -s "Whatever … Message Subject" remote_receiver@remote-server-email-address.com
…


The mail command will connect to local server TCP PORT 25 on local configured MTA and send via it. If the local MTA is misconfigured or it doesn't have a proper MX / PTR DNS records etc. or not configure as a relay SMTP remote mail will not get delivered. Sent Email should be properly delivered at remote recipient address.

How to send HTML formatted emails using mailx command on Linux console / terminal shell using remote server through SSH ?

Connect to remote SSH server (VPS), dedicated server, home Linux router etc. and run:

 

root@linux:~# mailx -a 'Content-Type: text/html'
      -s "This is advanced mailx indeed!" < email_content.html
      "first_email_to_send_to@gmail.com, mail_recipient_2@yahoo.com"

 


email_content.html should be properly formatted (at best w3c standard compliant) HTML.

Here is an example email_content.html (skeleton file)

 

    To: your_customer@gmail.com
    Subject: This is an HTML message
    From: marketing@your_company.com
    Content-Type: text/html; charset="utf8"

    <html>
    <body>
    <div style="
        background-color:
        #abcdef; width: 300px;
        height: 300px;
        ">
    </div>
Whatever text mixed with valid email HTML tags here.
    </body>
    </html>


Above command sends to two email addresses however if you have a text formatted list of recipients you can easily use that file with a bash shell script for loop and send to multiple addresses red from lets say email_addresses_list.txt .

To further advance the one liner you can also want to provide an email attachment, lets say the file email_archive.rar by using the -A email_archive.rar argument.

 

root@linux:~# mailx -a 'Content-Type: text/html'
      -s "This is advanced mailx indeed!" -A ~/email_archive.rar < email_content.html
      "first_email_to_send_to@gmail.com, mail_recipient_2@yahoo.com"

 

For those familiar with Dan Bernstein's Qmail MTA (which even though a bit obsolete is still a Security and Stability Beast across email servers) – mailx command had to be substituted with a custom qmail one in order to be capable to send via qmail MTA daemon.
 

2. Using sendmail command to send email
 

Do you remember that heavy hard to configure MTA monster sendmail ? It was and until this very day is the default Mail Transport Agent for Slackware Linux.

Here is how we were supposed to send mail with it:

 

[root@sendmail-host ~]# vim email_content_to_be_delivered.txt

 

Content of file should be something like:

Subject: This Email is sent from UNIX Terminal Email

Hi this Email was typed in a file and send via sendmail console email client
(part of the sendmail mail server)

It is really fun to go back in the pre-history of Mail Content creation 🙂

 

[root@sendmail-host ~]# sendmail -v user_name@remote-mail-domain.com  < /tmp/email_content_to_be_delivered.txt

 

-v argument provided, will make the communication between the mail server and your mail transfer agent visible.
 

3. Using ssmtp command to send mail
 

ssmtp MTA and its included shell command was used historically as it was pretty straight forward you just launch it on the command line type on one line all your email and subject and ship it (by pressing the CTRL + D key combination).

To give it a try you can do:

 

root@linux:~# apt-get install ssmtp
Reading package lists… Done
Building dependency tree       
Reading state information… Done
The following additional packages will be installed:
  libgnutls-openssl27
The following packages will be REMOVED:
  exim4-base exim4-config exim4-daemon-heavy
The following NEW packages will be installed:
  libgnutls-openssl27 ssmtp
0 upgraded, 2 newly installed, 3 to remove and 1 not upgraded.
Need to get 239 kB of archives.
After this operation, 3,697 kB disk space will be freed.
Do you want to continue? [Y/n] Y
Get:1 http://ftp.us.debian.org/debian stretch/main amd64 ssmtp amd64 2.64-8+b2 [54.2 kB]
Get:2 http://ftp.us.debian.org/debian stretch/main amd64 libgnutls-openssl27 amd64 3.5.8-5+deb9u3 [184 kB]
Fetched 239 kB in 2s (88.5 kB/s)         
Preconfiguring packages …
dpkg: exim4-daemon-heavy: dependency problems, but removing anyway as you requested:
 mailutils depends on default-mta | mail-transport-agent; however:
  Package default-mta is not installed.
  Package mail-transport-agent is not installed.
  Package exim4-daemon-heavy which provides mail-transport-agent is to be removed.

 

(Reading database … 169307 files and directories currently installed.)
Removing exim4-daemon-heavy (4.89-2+deb9u3) …
dpkg: exim4-config: dependency problems, but removing anyway as you requested:
 exim4-base depends on exim4-config (>= 4.82) | exim4-config-2; however:
  Package exim4-config is to be removed.
  Package exim4-config-2 is not installed.
  Package exim4-config which provides exim4-config-2 is to be removed.
 exim4-base depends on exim4-config (>= 4.82) | exim4-config-2; however:
  Package exim4-config is to be removed.
  Package exim4-config-2 is not installed.
  Package exim4-config which provides exim4-config-2 is to be removed.

Removing exim4-config (4.89-2+deb9u3) …
Selecting previously unselected package ssmtp.
(Reading database … 169247 files and directories currently installed.)
Preparing to unpack …/ssmtp_2.64-8+b2_amd64.deb …
Unpacking ssmtp (2.64-8+b2) …
(Reading database … 169268 files and directories currently installed.)
Removing exim4-base (4.89-2+deb9u3) …
Selecting previously unselected package libgnutls-openssl27:amd64.
(Reading database … 169195 files and directories currently installed.)
Preparing to unpack …/libgnutls-openssl27_3.5.8-5+deb9u3_amd64.deb …
Unpacking libgnutls-openssl27:amd64 (3.5.8-5+deb9u3) …
Processing triggers for libc-bin (2.24-11+deb9u3) …
Setting up libgnutls-openssl27:amd64 (3.5.8-5+deb9u3) …
Setting up ssmtp (2.64-8+b2) …
Processing triggers for man-db (2.7.6.1-2) …
Processing triggers for libc-bin (2.24-11+deb9u3) …

 

As you see from above output local default Debian Linux Exim is removed …

Lets send a simple test email …

 

hipo@linux:~# ssmtp user@remote-mail-server.com
Subject: Simply Test SSMTP Email
This Email was send just as a test using SSMTP obscure client
via SMTP server.
^d

 

What is notable about ssmtp is that even though so obsolete today it supports of STARTTLS (email communication encryption) that is done via its config file

 

/etc/ssmtp/ssmtp.conf

 

4. Send Email from terminal using Mutt client
 

Mutt was and still is one of the swiff army of most used console text email clients along with Alpine and Fetchmail to know more about it read here

Mutt supports reading / sending mail from multiple mailboxes and capable of reading IMAP and POP3 mail fetch protocols and was a serious step forward over mailx. Its syntax pretty much resembles mailx cmds.

 

root@linux:~# mutt -s "Test Email" user@example.com < /dev/null

 

Send email including attachment a 15 megabytes MySQL backup of Squirrel Webmail

 

root@linux:~# mutt  -s "This is last backup small sized database" -a /home/backups/backup_db.sql user@remote-mail-server.com < /dev/null

 


5. Using simple telnet to test and send email (verify existence of email on remote SMTP)
 

As a Mail Server SysAdmin this is one of my best ways to test whether I had a server properly configured and even sometimes for the sake of fun I used it as a hack to send my mail 🙂
telnet is and will always be a great tool for doing SMTP issues troubleshooting.
 

It is very useful to test whether a remote SMTP TCP port 25 is opened or a local / remote server firewall prevents connections to MTA.

Below is an example connect and send example using telnet to my local SMTP on www.pc-freak.net (QMail powered (R) 🙂 )

sending-email-using-telnet-command-howto-screenshot

 

root@pcfreak:~# telnet localhost 25
Trying 127.0.0.1…
Connected to localhost.
Escape character is '^]'.
220 This is Mail Pc-Freak.NET ESMTP
HELO mail.www.pc-freak.net
250 This is Mail Pc-Freak.NET
MAIL FROM:<hipo@www.pc-freak.net>
250 ok
RCPT TO:<roots_bg@yahoo.com>
250 ok
DATA
354 go ahead
Subject: This is a test subject

 

This is just a test mail send through telnet
.
250 ok 1536440787 qp 28058
^]
telnet>

 

Note that the returned messages are native to qmail, a postfix would return a slightly different content, here is another test example to remote SMTP running sendmail or postfix.

 

root@pcfreak:~# telnet mail.servername.com 25
Trying 127.0.0.1…
Connected to localhost.localdomain (127.0.0.1).
Escape character is '^]'.
220 mail.servername.com ESMTP Sendmail 8.13.8/8.13.8; Tue, 22 Oct 2013 05:05:59 -0400
HELO yahoo.com
250 mail.servername.com Hello mail.servername.com [127.0.0.1], pleased to meet you
mail from: systemexec@gmail.com
250 2.1.0 hipo@www.pc-freak.net… Sender ok
rcpt to: hip0d@yandex.ru
250 2.1.5 hip0d@yandex.ru… Recipient ok
data
354 Enter mail, end with "." on a line by itself
Hey
This is test email only

 

Thanks
.
250 2.0.0 r9M95xgc014513 Message accepted for delivery
quit
221 2.0.0 mail.servername.com closing connection
Connection closed by foreign host.


It is handy if you want to know whether remote MTA server has a certain Emailbox existing or not with telnet by simply trying to send to a certian email and checking the Email server returned output (note that the message returned depends on the remote MTA version and many qmails are configured to not give information on the initial SMTP handshake but returns instead a MAILER DAEMON failure error sent back to your sender address. Some MX servrers are still vulnerable to this attack yet, historically dreamhost.com. Below attack screenshot is made at the times before dreamhost.com fixed the brute force email issue.

Terminal-Verify-existing-Email-with-telnet

6. Using simple netcat TCP/IP Swiss Army Knife to test and send email in console

netcat-logo-a-swiff-army-knife-of-the-hacker-and-security-expert-logo
Other tool besides telnet of testing remote / local SMTP is netcat tool (for reading and writting data across TCP and UDP connections).

The way to do it is analogous but since netcat is not present on most Linux OSes by default you need to install it through the package manager first be it apt or yum etc.

# apt-get –yes install netcat
…


 

First lets create a new file test_email_content.txt using bash's echo cmd.
 

 

# echo 'EHLO hostname
MAIL FROM: hip0d@yandex.ru
RCPT TO:   solutions@www.pc-freak.net
DATA
From: A tester <hip0d@yandex.ru>
To:   <solutions@www.pc-freak.net>
Date: date
Subject: A test message from test hostname

 

Delete me, please
.
QUIT
' >>test_email_content.txt

 

# netcat -C localhost 25 < test_email_content.txt

 

220 This is Mail Pc-Freak.NET ESMTP
250-This is Mail Pc-Freak.NET
250-STARTTLS
250-SIZE 80000000
250-PIPELINING
250 8BITMIME
250 ok
250 ok
354 go ahead
451 See http://pobox.com/~djb/docs/smtplf.html.

Because of its simplicity and the fact it has a bit more capabilities in reading / writing data over network it was no surprise it was among the favorite tools not only of crackers and penetration testers but also a precious debug tool for the avarage sysadmin. netcat's advantage over telnet is you can push-pull over the remote SMTP port (25) a non-interactive input.


7. Using openssl to connect and send email via encrypted channel

 

root@linux:~# openssl s_client -connect smtp.gmail.com:465 -crlf -ign_eof

    ===
               Certificate negotiation output from openssl command goes here
        ===

        220 smtp.gmail.com ESMTP j92sm925556edd.81 – gsmtp
            EHLO localhost
        250-smtp.gmail.com at your service, [78.139.22.28]
        250-SIZE 35882577
        250-8BITMIME
        250-AUTH LOGIN PLAIN XOAUTH2 PLAIN-CLIENTTOKEN OAUTHBEARER XOAUTH
        250-ENHANCEDSTATUSCODES
        250-PIPELINING
        250-CHUNKING
        250 SMTPUTF8
            AUTH PLAIN *passwordhash*
        235 2.7.0 Accepted
            MAIL FROM: <hipo@pcfreak.org>
        250 2.1.0 OK j92sm925556edd.81 – gsmtp
            rcpt to: <systemexec@gmail.com>
        250 2.1.5 OK j92sm925556edd.81 – gsmtp
            DATA
        354  Go ahead j92sm925556edd.81 – gsmtp
            Subject: This is openssl mailing

            Hello nice user
            .
        250 2.0.0 OK 1339757532 m46sm11546481eeh.9
            quit
        221 2.0.0 closing connection m46sm11546481eeh.9
        read:errno=0


8. Using CURL (URL transfer) tool to send SSL / TLS secured crypted channel emails via Gmail / Yahoo servers and MailGun Mail send API service


Using curl webpage downloading advanced tool for managing email send might be  a shocking news to many as it is idea is to just transfer data from a server.
curl is mostly used in conjunction with PHP website scripts for the reason it has a Native PHP implementation and many PHP based websites widely use it for download / upload of user data.
Interestingly besides support for HTTP and FTP it has support for POP3 and SMTP email protocols as well
If you don't have it installed on your server and you want to give it a try, install it first with apt:
 

root@linux:~# apt-get install curl

 


To learn more about curl capabilities make sure you check cURL –manual arg.
 

root@linux:~# curl –manual

 

a) Sending Emails via Gmail and other Mail Public services

Curl is capable to send emails from terminal using Gmail and Yahoo Mail services, if you want to give that a try.

gmail-settings-google-allow-less-secure-apps-sign-in-to-google-screenshot

Go to myaccount.google.com URL and login from the web interface choose Sign in And Security choose Allow less Secure Apps to be -> ON and turn on access for less secure apps in Gmail. Though I have not tested it myself so far with Yahoo! Mail, I suppose it should have a similar security settings somewhere.

Here is how to use curl to send email via Gmail.

Gmail-password-Allow-less-secure-apps-ON-screenshot-howto-to-be-able-to-send-email-with-text-commands-with-encryption-and-outlook

 

 

root@linux:~# curl –url 'smtps://smtp.gmail.com:465' –ssl-reqd \
  –mail-from 'your_email@gmail.com' –mail-rcpt 'remote_recipient@mail.com' \
  –upload-file mail.txt –user 'your_email@gmail.com:your_accout_password'


b) Sending Emails using Mailgun.com (Transactional Email Service API for developers)

To use Mailgun to script sending automated emails go to Mailgun.com and create account and generate new API key.

Then use curl in a similar way like below example:

 

curl -sv –user 'api:key-7e55d003b…f79accd31a' \
    https://api.mailgun.net/v3/sandbox21a78f824…3eb160ebc79.mailgun.org/messages \
    -F from='Excited User <developer@yourcompany.com>' \
    -F to=sandbox21a78f824…3eb160ebc79.mailgun.org \
    -F to=user_acc@gmail.com \
    -F subject='Hello' \
    -F text='Testing Mailgun service!' \
   –form-string html='<h1>EDMdesigner Blog</h1><br /><cite>This tutorial helps me understand email sending from Linux console</cite>' \
    -F attachment=@logo_picture.jpg

 

The -F option that is heavy present in above command lets curl (Emulate a form filled in button in which user has pressed the submit button).
For more info of the options check out man curl.
 

 

9. Using swaks command to send emails from

 

root@linux:~# apt-cache show swaks|grep "Description" -B 10
Package: swaks
Version: 20170101.0-1
Installed-Size: 221
Maintainer: Andreas Metzler <ametzler@debian.org>
Architecture: all
Depends: perl
Recommends: libnet-dns-perl, libnet-ssleay-perl
Suggests: perl-doc, libauthen-sasl-perl, libauthen-ntlm-perl
Description-en: SMTP command-line test tool
 swaks (Swiss Army Knife SMTP) is a command-line tool written in Perl
 for testing SMTP setups; it supports STARTTLS and SMTP AUTH (PLAIN,
 LOGIN, CRAM-MD5, SPA, and DIGEST-MD5). swaks allows one to stop the
 SMTP dialog at any stage, e.g to check RCPT TO: without actually
 sending a mail.
 .
 If you are spending too much time iterating "telnet foo.example 25"
 swaks is for you.
Description-md5: f44c6c864f0f0cb3896aa932ce2bdaa8
 

 

 

root@linux:~# apt-get instal –yes swaks

root@linux:~# swaks –to mailbox@example.com -s smtp.gmail.com:587
      -tls -au <user-account> -ap <account-password>

 


The -tls argument (in order to use gmail encrypted TLS channel on port 587)

If you want to hide the password not to provide the password from command line so (in order not to log it to user history) add the -a options.

10. Using qmail-inject on Qmail mail servers to send simple emails

Create new file with content like:
 

root@qmail:~# vim email_file_content.text
To: user@mail-example.com
Subject: Test


This is a test message.
 

root@qmail:~# cat email_file_content.text | /var/qmail/bin/qmail-inject


qmail-inject is part of ordinary qmail installation so it is very simple it even doesn't return error codes it just ships what ever given as content to remote MTA.
If the linux host where you invoke it has a properly configured qmail installation the email will get immediately delivered. The advantage of qmail-inject over the other ones is it is really lightweight and will deliver the simple message more quickly than the the prior heavy tools but again it is more a Mail Delivery Agent (MDA) for quick debugging, if MTA is not working, than for daily email writting.

It is very useful to simply test whether email send works properly without sending any email content by (I used qmail-inject to test local email delivery works like so).
 

root@linux:~# echo 'To: mailbox_acc@mail-server.com' | /var/qmail/bin/qmail-inject

 

11. Debugging why Email send with text tool is not being send properly to remote recipient

If you use some of the above described methods and email is not delivered to remote recipient email addresses check /var/log/mail.log (for a general email log and postfix MTAs – the log is present on many of the Linux distributions) and /var/log/messages or /var/log/qmal (on Qmail installations) /var/log/exim4 (on servers running Exim as MTA).

https://www.pc-freak.net/images/linux-email-log-debug-var-log-mail-output

 Closure

The ways to send email via Linux terminal are properly innumerous as there are plenty of scripted tools in various programming languages, I am sure in this article,  also missing a lot of pre-bundled installable distro packages. If you know other interesting ways / tools to send via terminal I would like to hear it.

Hope you enjoyed, happy mailing !

Tags: , , , , , , , , , , , , , , , , , , , , , , , ,
Posted in Curious Facts, Email clients, Linux, System Administration | No Comments »

How to connect to WiFi network using console or terminal on GNU / Linux

Wednesday, November 2nd, 2011

iwlist-screenshot-howto-connect-to-wifi-in-text-console-linux-wpasupplicant

Sometimes its useful to connect to Wireless Networks using console . The reasons for that might be many, one possible reason is to be able to debug, Wireless connection failures or simply omit the use of the many available GUI wifi connection programs.

As a first step before connecting in terminal is to look up for the wifi networks available for connection, this is done with cmd:

linux:~# iwlist wlan0 scanning
wlan0 Scan completed :
Cell 01 - Address: 00:24:01:90:8F:38
Channel:7
Frequency:2.442 GHz (Channel 7)
Quality=70/70 Signal level=-39 dBm
Encryption key:on
ESSID:"magdanoz"
Bit Rates:1 Mb/s; 2 Mb/s; 5.5 Mb/s; 11 Mb/s
Bit Rates:6 Mb/s; 9 Mb/s; 12 Mb/s; 48 Mb/s; 18 Mb/s
24 Mb/s; 36 Mb/s; 54 Mb/s
Mode:Master
Extra:tsf=000000034f5c786b
Extra: Last beacon: 68ms ago
IE: Unknown: 00086D616764616E6F7A
IE: Unknown: 010482848B96
IE: Unknown: 030107
IE: Unknown: 32080C1218602430486C
IE: Unknown: CC0700CC020000018A
IE: Unknown: CC0700CC0300000100
IE: WPA Version 1
Group Cipher : TKIP
Pairwise Ciphers (2) : TKIP CCMP
Authentication Suites (1) : PSK
IE: IEEE 802.11i/WPA2 Version 1
Group Cipher : TKIP
Pairwise Ciphers (2) : TKIP CCMP
Authentication Suites (1) : PSK
Cell 02 - Address: 00:1E:2A:60:5E:DC
Channel:1
...

To just list the ESSID s of the wifi networks:

linux:~# iwlist wlan0 scanning|grep -i 'essid'
ESSID:"magdanoz"
ESSID:"default"
ESSID:"todorov"
ESSID:"BTC-ADSL"
ESSID:"Zahari"
ESSID:"Drago"

1. Connecting to Open Wireless Network

Now from the above output it is clear 6 wifi networks are available for connection. The default wifi network from the list is an Open network (e.g. without pass). To connect to it I use cmd:

linux:~# /sbin/iwconfig wlan0 essid 'default'
linux:~# /sbin/iwconfig wlan0 key open

After connected to configure IP, Gateway and DNS from a DHCP server running on the WIFI router, dhclient cmd is used:

linux:~# /sbin/dhclient wlan0

2. Connecting to WEP 64bit / 128bit encrypted network

linux:~# /sbin/iwconfig wlan0 key 1234-5678-9101-1213

3. Connecting to WPA / WPA2 encrypted wifi network

To connect to WPA or WPA2 encrypted network its necessery to have installed wpasupplicant package. The name of the package might vary in different distributions on Debian and Ubuntu, the name of the package is wpasupplicant, on Fedora, CentOS and RHEL the package that has to be in is wpa_supplicant :
After having installed the wpa_supplicant to connect to the network with ESSID name magdanoz , wpa_passphrase is used first:

linux:~# /usr/bin/wpa_passphrase magdanoz Secret_Wifi_Password | tee -a /etc/wpa_supplicant.conf
network={
ssid="magdanoz"
#psk="Secret_Wifi_Password"
psk=6f7590250c4496ff7bf8dd25199ac3bb5a614d4bc069975aa884bcf084da73bc
}

As you see in above command the secret password key is generated printed on the screen and then added to /etc/wpa_supplicant.conf , necessery to establish the wireless connection with wpa_supplicant with cmd:

linux:~# /sbin/wpa_supplicant wpa_supplicant -d wext -i wlan0 -c /etc/wpa_supplicant.conf -B

-d wext instructs wpa_supplicant to use (Linux wireless extension driver).
-B tells wpa_supplicant to background the connection to prevent the wireless connection to drop off, if the console / terminal from which it is launched gets closed.

In case of succesful connection with wpa_supplicant , once again IP, Gateway and DNS is configured fetching the settings from the wifi hotspot dhcp server:

linux:~# /sbin/dhclient wlan0

General information about the wireless network and info related to the established connection can be obtained with /usr/bin/iwconfig :

linux:~# /sbin/iwconfig
lo no wireless extensions.
eth0 no wireless extensions.
wlan0 IEEE 802.11abg ESSID:"magdanoz"
Mode:Managed Frequency:2.442 GHz Access Point: 00:24:00:90:8F:38
Bit Rate=54 Mb/s Tx-Power=15 dBm
Retry long limit:7 RTS thr:off Fragment thr:off
Encryption key:off
Power Management:off
Link Quality=70/70 Signal level=-39 dBm
Rx invalid nwid:0 Rx invalid crypt:0 Rx invalid frag:0
Tx excessive retries:0 Invalid misc:0 Missed beacon:0

To configure the exact channel over which the wireless connection will be established again is done with iwconfig, for instance to configure wlan0 wifi connection established to be on wifi channel 7:

linux:~# /sbin/iwconfig wlan0 channel 11

By default iwconfig is set to automatically set the channel based on connected network ESSID , if the channel is modified to some specific number to revert it back use:

linux:~# /sbin/iwconfig wlan0 channel auto
 

Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,
Posted in System Administration | 4 Comments »

Thursday, July 14th, 2016

use-remote-dns-on-mozilla-firefox-howto-windows-linux-logo.svg

If you're using Mozilla Firefox browser to browse the Web with Traffic Tunneling via SSH Tunnel to your own Linux server like I do in order to prevent yourself traffic to be sniffed from your Work corporate computer (as most of the corporations such as IBM / Hewlett Packard / Concentrix etc. are forcing all employee PC traffic to be  to be transported via default set Windows Corporate Proxy active for all browsers.

Then you will certainly also want to prevent the DNS requests to be not logged somewhere in your Corporate IT department thus the question arises:

How to force DNS requests to be made through the Proxy server (SSH host)?

Nomatter where you're using Firefox browser with advanced proxying plugin such as FoxyProxy FF add-on or the default Proxy FF features the DNS lookups might end up in Corporate set DNS servers often forced for the computer / notebook and impossible to be changed to a custom ones as many of the Corporation internal Sharepoints and domains are only visible from their internal networks.

Thanksfully in newer versions there is an easy way to do it directly from Visual menus via:

Tools -> Options -> Advanced -> Network -> Settings

You will get a screen like below:
 

firefox-use-proxy-remote-dns-howto-screenshot

Just tick the Remote DNS and that will force Firefox to query remote Proxy server proxy DNS

 

If you happen to be running older Firefox which doesn't have the Remote DNS tick you can also try to set the setting manually:

 

  1. In firefox type this in your address bar:

    about:config

  2. Click I'll be careful I  promise.

  3. In the filter textbox, type: proxy

  4. Find the preference name called *network.proxy.socks_remote_dns*. Double click it to set it to true.

    i-will-be-careful-i-promise-firefox-windows-screenshot-warranty


network-proxy-socks-remote_dns-firefox-screenshot

Enjoy ! 🙂

Tags: , , , , , , , , ,
Posted in Curious Facts, Everyday Life, Firefox, Various, Windows | No Comments »

How to install and configure djbdns from source as a Cachening Localhost Proxy resolver to increase resolving efficiency on Debian 6 Squeeze

Monday, August 1st, 2011

djbdns-logo-install-configure-djbdns-from-source-on-gnu-linux-to-accelerate-server-dns-resolving
It seems DjbDNS on Debian Squeeze has been not included as a Debian package. There is still possibility to install djbdns from an older deb package or install it from source. I however decided to install it from source as finding the old Debian package for Lenny and Etch takes time, plus I'm running an amd64 version of Debian and this might even more complicate the situation.
Installing it from source is not really a Debian way but at least it works.

In this article I assume that daemontools and ucspi-tcp are preliminary installed, if not one needs to install them with:

debian:~# apt-get install ucspi-tcp daemontools daemontools-run
...

The above two ones are required as DJBDNS is originally made to run through djb's daemontools.

Here is the exact step I took to have it installed as local caching DNS server on a Debian Squeeze server:

1. Download and untar DjbDNS

debian:~# wget -q http://cr.yp.to/djbdns/djbdns-1.05.tar.gz debian:~# tar -zxvvf djbdns-1.05.tar.gz
...

2. Add DjbDNS users to /etc/passwd

Creating the below two users is not arbitrary but it's recommendable.

echo 'dnscache:*:54321:54321:dnscache:/dev/null:/dev/null' >> /etc/passwd
echo 'dnslog:*:54322:54322:dnslog:/dev/null:/dev/null' >> /etc/passwd

3. Compile DJBDNS nameserver

First it's necessery to use the below echo command to work around a common Linux bug:

debian:~# cd djbdns-1.05
debian:/root/djbdns-1.05# echo gcc -O2 -include /usr/include/errno.h > conf-cc

Next let's make it:

debian:/root/djbdns-1.05# make

4. Install the compiled djbdns binaries

debian:/root/djbdns-1.05# make setup check
# here comes some long install related output

If no errors are produced by make setup check this means that the djbdns should have installed itself fine.

As installation is compileted it's a good idea to report about the newly installed DjbDNS server if running a mail server. This info is used by Dan Bernstein to gather statistical data about the number of installations of djbdns servers throughout the world.

5. Do some general configurations to the newly installed DJBDNS

Now let's copy the list of the IP addresses of the global DNS root servers in /etc/.

debian:/root/djbdns-1.05# cp -rpf dnsroots.global /etc/ debian:/root/djbdns-1.05# ./dnscache-conf dnscache dnslog /etc/dnscache 0.0.0.0

dnscache-conf will generate some default configuration files for djbdns in /etc/dnscache

Next allow the networks which should be able to use the just installed djbdns server as a caching server:

debian:/root/djbdns-1.05# cd /etc/dnscache/root/ip
debian:/etc/dnscache/root# touch 192.168.1
debian:/root/djbdns-1.05# touch 123.123

First command will allow all ips in range 192.168.1.* to be able to access the DNS server and the second command will allow all ips from 123.123.1-255.1-255 to be able to query the server.

Some further fine tunning can be done from the files:

/etc/dnscache/env/CACHESIZE and /etc/dnscache/env/DATALIMIT

As a last step, before it's running, we have to link the /etc/dnscache to daemontools like so:

debian:/root/djbdns-1.05# ln -sf /etc/dnscache /etc/service/dnscache

If the daemontools is not linked to be accessible via /etc/service it's also a good to link it there:

debian:~# ln -sf /etc/service /

Now the DJBDNS should be running fine, to test if it's running without errors through daemontools I used:

debian:~# ps ax|grep -i readproc
5358 pts/18 R+ 0:00 grep -i readproc
11824 ? S 0:00 readproctitle service errors: ...........

If no errors are displayed it's configured and running to also test if it's capable of resolving I used the host command:

debian:~# host www.pc-freak.net localhost
Using domain server:
Name: localhost
Address: 127.0.0.1#53
Aliases:

www.pc-freak.net has address 83.228.93.76
www.pc-freak.net mail is handled by 0 mail.www.pc-freak.net.

Now the DJBDNS is properly installed and if you test it for a while with time host somehost.com localhost , you will see how quick it is in resolving.

The advantage of running DJBDNS is it does not require almost no maintance, its rock solid and great just like all other Dan Bernstein's written software.
Enjoy 😉

Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,
Posted in Linux, Performance Tuning, System Administration, Web and CMS | 4 Comments »

« Older Entries