Posts Tagged ‘connections’

How to Create Secure Stateful Firewall Rules with nftables on Linux

Monday, October 6th, 2025

nftables-logo-linux-mastering-stateful-firewall-rules-with-nftables-firewall

Firewalls are the frontline defense of any server or network. While many sysadmins are familiar with iptables, nftables is the modern Linux firewall framework offering more power, flexibility, and performance.

One of the key features of nftables is stateful packet inspection, which lets you track the state of network connections and write precise rules that dynamically accept or reject packets based on connection status.

In this guide, we’ll deep dive into stateful firewall rules using nftables — what they are, why they matter, and how to master them for a robust, secure network.

What is Stateful Firewalling?

A stateful firewall keeps track of all active connections passing through it. It monitors the state of a connection (new, established, related, invalid) and makes decisions based on that state rather than just static IP or port rules.

This allows:

  • Legitimate traffic for existing connections to pass freely
  • Blocking unexpected or invalid packets
  • Better security and less manual rule writing

Understanding Connection States in nftables

nftables uses the conntrack subsystem to track connections. Common states are:

State

Description

new

Packet is trying to establish a new connection

established

Packet belongs to an existing connection

related

Packet related to an existing connection (e.g. FTP data)

invalid

Packet that does not belong to any connection or is malformed

Basic Stateful Rule Syntax in nftables

The key keyword is ct state. For example:

nft add rule inet filter input ct state established,related accept

This means: allow any incoming packets that are part of an established or related connection.

Step-by-Step: Writing a Stateful Firewall with nftables

  1. Create the base table and chains


nft add table inet filter

nft add chain inet filter input { type filter hook input priority 0 \; }

nft add chain inet filter forward { type filter hook forward priority 0 \; }

nft add chain inet filter output { type filter hook output priority 0 \; }

  1. Allow loopback traffic

nft add rule inet filter input iif lo accept

  1. Allow established and related connections

nft add rule inet filter input ct state established,related accept

  1. Drop invalid packets

nft add rule inet filter input ct state invalid drop

  1. Allow new SSH connections

nft add rule inet filter input tcp dport ssh ct state new accept

  1. Drop everything else

nft add rule inet filter input drop

Why Use Stateful Filtering?

  • Avoid writing long lists of rules for each connection direction
  • Automatically handle protocols with dynamic ports (e.g. FTP, SIP)
  • Efficient resource usage and faster lookups
  • Better security by rejecting invalid or unexpected packets
nftables-tcpip-model-diagram-logo

Advanced Tips for Stateful nftables Rules

  • Use ct helper for protocols requiring connection tracking helpers (e.g., FTP)
  • Combine ct state with interface or user match for granular control
  • Use counters with rules to monitor connection states
  • Rate-limit new connections using limit rate with ct state new

Real-World Example: Preventing SSH Brute Force with Stateful Rules

nft add rule inet filter input tcp dport ssh ct state new limit rate 5/minute accept

nft add rule inet filter input tcp dport ssh drop

This allows only 5 new SSH connections per minute.

Troubleshooting Stateful Rules

  • Use conntrack -L to list tracked connections
  • Logs can help; enable logging on dropped packets temporarily
  • Check if your firewall blocks ICMP (important for some connections)
  • Remember some protocols may require connection helpers

Making Your nftables Rules Permanent

By default, any rules you add using nft commands are temporary — they live in memory and are lost after a reboot.

To make your nftables rules persistent, you need to save them to a configuration file and ensure they're loaded at boot.

Option 1. Using the nftables Service (Preferred on Most Distros)

Most modern Linux distributions (Debian ≥10, Ubuntu ≥20.04, CentOS/RHEL ≥8) come with a systemd service called nftables.service that automatically loads rules from /etc/nftables.conf at boot.

 Do the following to make nftables load on boot:

Dump your current rules into a file:

# nft list ruleset > /etc/nftables.conf

Enable the nftables service to load them at boot:

# systemctl enable nftables

(Optional) Start the service immediately if it’s not running:

# systemctl start nftables

Check status:

# systemctl status nftables

Now your rules will survive reboots.

Alternative way to load nftables on network UP, Use Hooks in
/etc/network/if-pre-up.d/ or Custom Scripts (Advanced)

If your distro doesn't use nftables.service or you're on a minimal setup (e.g., Alpine, Slackware, older Debian), you can load the rules manually at boot:

Save your rules:

# nft list ruleset > /etc/nftables.rules

Create a script to load them (e.g., /etc/network/if-pre-up.d/nftables):

#!/bin/sh

nft -f /etc/nftables.rules

Make it executable:

chmod +x /etc/network/if-pre-up.d/nftables

This method works on systems without systemd.

Sample /etc/nftables.conf config

We first define variables which we can use later on in our ruleset:

 

define NIC_NAME = "eth0"

define NIC_MAC_GW = "DE:AD:BE:EF:01:01"

define NIC_IP = "192.168.1.12"

define LOCAL_INETW = { 192.168.0.0/16 }

define LOCAL_INETWv6 = { fe80::/10 }

define DNS_SERVERS = { 1.1.1.1, 8.8.8.8 }

define NTP_SERVERS = { time1.google.com, time2.google.com, time3.google.com, time4.google.com }

define DHCP_SERVER = "192.168.1.1"

Next code block shows ip filter and ip6 filter sample:

We first create an explicit deny rule (policy drop;) for the chain input and chain output.
This means all network traffic is dropped unless its explicitly allowed later on.

Next we have to define these exceptions based on network traffic we want to allow.
Loopback network traffic is only allowed from the loopback interface and within RFC loopback network space.

nftables automatically maps network protocol names to port numbers (e.g. HTTPS 443).
In this example, we only allow incoming sessions which we initiated (ct state established accept) from ephemeral ports (dport 32768-65535). Be aware an app or web server should allow newly initiated sessions (ct state new).

Certain network sessions initiated by this host (ct state new,established accept) in the chain output are explicitly allowed in the output chain. We also allow outgoing ping requests (icmp type echo-request), but do not want others to ping this host, hence ct state established in the icmp type input chain. 

table ip filter {

    chain input {

       type filter hook input priority 0; policy drop;

       iifname "lo" accept

       iifname "lo" ip saddr != 127.0.0.0/8 drop

       iifname $NIC_NAME ip saddr 0.0.0.0/0 ip daddr $NIC_IP tcp sport { ssh, http, https, http-alt } tcp dport 32768-65535 ct state established accept

       iifname $NIC_NAME ip saddr $NTP_SERVERS ip daddr $NIC_IP udp sport ntp udp dport 32768-65535 ct state established accept

       iifname $NIC_NAME ip saddr $DHCP_SERVER ip daddr $NIC_IP udp sport bootpc udp dport 32768-65535 ct state established log accept

       iifname $NIC_NAME ip saddr $DNS_SERVERS ip daddr $NIC_IP udp sport domain udp dport 32768-65535 ct state established accept

       iifname $NIC_NAME ip saddr $LOCAL_INETW ip daddr $NIC_IP icmp type echo-reply ct state established accept

    }

 

    chain output {

       type filter hook output priority 0; policy drop;

       oifname "lo" accept

       oifname "lo" ip daddr != 127.0.0.0/8 drop

       oifname $NIC_NAME ip daddr 0.0.0.0/0 ip saddr $NIC_IP tcp dport { ssh, http, https, http-alt } tcp sport 32768-65535 ct state new,established accept

       oifname $NIC_NAME ip daddr $NTP_SERVERS ip saddr $NIC_IP udp dport ntp udp sport 32768-65535 ct state new,established accept

       oifname $NIC_NAME ip daddr $DHCP_SERVER ip saddr $NIC_IP udp dport bootpc udp sport 32768-65535 ct state new,established log accept

       oifname $NIC_NAME ip daddr $DNS_SERVERS ip saddr $NIC_IP udp dport domain udp sport 32768-65535 ct state new,established accept

       oifname $NIC_NAME ip daddr $LOCAL_INETW ip saddr $NIC_IP icmp type echo-request ct state new,established accept

    }

 

    chain forward {

       type filter hook forward priority 0; policy drop;

    }

}

 

The next code block is used to block incoming and outgoing IPv6 traffic, except ping requests (icmpv6 type echo-request) and IPv6 network discovery (nd-router-advert, nd-neighbor-solicit, nd-neighbor-advert).

vNICs are often automatically provisioned with IPv6 addresses and left untouched. These interfaces can be abused by malicious entities to tunnel out confidential data or even a shell.

table ip6 filter {

    chain input {

       type filter hook input priority 0; policy drop;

       iifname "lo" accept

       iifname "lo" ip6 saddr != ::1/128 drop

       iifname $NIC_NAME ip6 saddr $LOCAL_INETWv6 icmpv6 type { destination-unreachable, packet-too-big, time-exceeded, parameter-problem, echo-reply, nd-router-advert, nd-neighbor-solicit, nd-neighbor-advert } ct state established accept

    }

 

    chain output {

       type filter hook output priority 0; policy drop;

       oifname "lo" accept

       oifname "lo" ip6 daddr != ::1/128 drop

       oifname $NIC_NAME ip6 daddr $LOCAL_INETWv6 icmpv6 type echo-request ct state new,established accept

    }

 

    chain forward {

       type filter hook forward priority 0; policy drop;

    }

}

 Last code block is used for ARP traffic which limits ARP broadcast network frames:

table arp filter {

   chain input {

       type filter hook input priority 0; policy accept;

       iif $NIC_NAME limit rate 1/second burst 2 packets accept

   }

 

   chain output {

       type filter hook output priority 0; policy accept;

   }

}

To load up nftables rules

# systemctl restart nftables && systemctl status nftables && nft list ruleset

Test Before Save and Apply

NB !!! Always test your rules before saving them permanently. A typo can lock you out of your server !!!

Try:

# nft flush ruleset

# nft -f /etc/nftables.conf


!!! Make sure to test your ports are truly open or closed. You can use nc, telnet or tcpdump for this. !!!

Or use a screen or tmux session and set a watchdog timer (e.g., at now +2 minutes reboot) so you can recover if something goes wrong.

Conclusion

In the ever-evolving landscape of network security, relying on static firewall rules is no longer enough. Stateful filtering with nftables gives sysadmins the intelligence and flexibility needed to deal with real-world traffic — allowing good connections, rejecting bad ones, and keeping things efficient.

With just a few lines, you can build a firewall that’s not only more secure but also easier to manage and audit over time.Whether you're protecting a personal server, a VPS, or a corporate gateway, understanding ct state is a critical step in moving from "good enough" security to proactive, intelligent defense.
If you're still relying on outdated iptables chains with hundreds of line-by-line port filters, maybe it's time to embrace the modern way.
nftables isn’t just the future — it’s the present. Further on log, monitor, and learn from your own traffic.

Start with the basics, then layer on your custom rules and monitoring and enjoy your system services and newtork being a bit more secure than before.


Cheers ! 🙂

Windows 10 install local Proxy server to Save bandwidth on a slow and limited Mobile Phone HotSpot network Shared connections

Wednesday, August 20th, 2025

https://pc-freak.net/images/how-to-use-local-proxy-to-speed-up-internet-speed-connectivity-on-windows-os-with-squid-and-privoxy

If you're running on Internet ISP that is providing via a Internet / Wifi Router device with a 3G / 4G / 5G etc. but your receiving point location is situated somewhere very far in a places like High mountains lets say Rila Mountain or  Alps on a very distant places where Internet coverate of Inetner Service Provider is low or very low but you need still to Work / Play / Entertain on the Net frequently.
Hence you will cenrtainly be looking for a ways to Speed Up / Optimize the Internet connectivity somehow.
You cannot do miracles but certainly the daily operations and a pack up of repeating traffic can be achieved by using installing and using simple local proxy server.

The advantages of using a proxy are even more besides the speed up of Internet connection lines, here is the Pros you get by using the proxy:
 

  • Using Caches frequently accessed content (e.g., images, scripts, web pages).
  • Blocks ads and trackers (reduces bandwidth).
  • Compresses data (if needed)
  • Can serve multiple local devices if needed.
     

To save bandwidth on a slow and limited connectivity Internet router or mobile phone hotspot using Windows 10, you can install a local proxy server that:

Here’s a step-by-step guide to set this up:
 

Install a local caching proxy server on Windows 10 to reduce bandwidth usage over a mobile hotspot.


1. Install Squid (Caching Proxy Server)

Squid is a powerful and widely used open-source caching proxy.

Download Squid for Windows

Download Squid for Windows from:

https://squid.acmeconsulting.it/download (Unofficial, stable build)

or compile it manually (if you're having an own Linux or BSD router that is passing on the traffic)

2. Install Squid Proxy sever on Windows


2.1. Extract or install the downloaded Squid package.


 

2.2. Install it as a Windows Service

Open Command Prompt (Admin) and run:

C:\\Users\\hipo\\Downloads> squid -i

Initialize cache directories:
 

C:\\Users\\hipo\\Downloads> squid -z

 

3. Configure Squid Proxy via squid.conf


3.1. Open squid.conf

usually in

C:\\Squid\\etc\\squid\\squid.conf
 

3.2. Edit key lines:  

http_port 3128
cache_dir ufs c:/squid/var/cache 100 16 256
access_log c:/squid/var/logs/access.log
cache_log c:/squid/var/logs/cache.log
maximum_object_size 4096 KB
cache_mem 64 MB

 

 

3.3. Allow local access:

 

acl localnet src 192.168.0.0/16
http_access allow localnet

(Adjust IP ranges according to your network.)

 

Here's a ready-to-use Squid configuration file optimized for Running on Windows 10:

  • Caching web content to save bandwidth
  • Blocking ads and trackers
  • Allowing local device connections

 

Location for the squid Config File
 

The Windows squid installer should have setup the Squid proxy by default inside C:\Squid so the full path to squid.conf should be:
Place this as

squid.conf

in:

C:\\Squid\\etc\\squid\\squid.conf

 

# BASIC CONFIGURATION
http_port 3128
visible_hostname localhost

# CACHE SETTINGS
cache_mem 128 MB
maximum_object_size 4096 KB
maximum_object_size_in_memory 512 KB
cache_dir ufs c:/squid/var/cache 100 16 256
cache_log c:/squid/var/logs/cache.log
access_log c:/squid/var/logs/access.log

# DNS
dns_nameservers 8.8.8.8 1.1.1.1

# ACLs (Access Control Lists)
acl localhost src 127.0.0.1/32
acl localnet src 192.168.0.0/16
acl Safe_ports port 80      # HTTP
acl Safe_ports port 443     # HTTPS
acl Safe_ports port 21      # FTP
acl CONNECT method CONNECT

# BLOCKED DOMAINS (Ad/Tracking)
acl ads dstdomain .doubleclick.net .googlesyndication.com .googleadservices.com
acl ads dstdomain .ads.yahoo.com .adnxs.com .track.adform.net
http_access deny ads

# SECURITY & ACCESS CONTROL
http_access allow localhost
http_access allow localnet
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access deny all

# REFRESH PATTERNS (Cache aggressively)
refresh_pattern ^ftp:           1440    20%     10080
refresh_pattern ^gopher:        1440    0%      1440
refresh_pattern -i \.jpg$       10080   90%     43200
refresh_pattern -i \.png$       10080   90%     43200
refresh_pattern -i \.gif$       10080   90%     43200
refresh_pattern -i \.css$       10080   90%     43200
refresh_pattern -i \.js$        10080   90%     43200
refresh_pattern -i \.html$      1440    90%     10080
refresh_pattern .               0       20%     4320

# LOGGING
logfile_rotate 10

 

 

4. Start the Squid Win Service from Admin command prompt

C:\Users\hipo> net start squid


5. Test the Proxy

 

Set the proxy server in your Windows proxy settings:
 

  • Go to Settings > Network & Internet > Proxy
     
  • Enable Manual proxy setup:

Address: 127.0.0.1

Port: 3128

Browse the web — Squid will now cache content locally.

Make sure

C:\Squid\var\cache

and

C:\Squid\var\logs

exist.

You can expand the ad block list by importing public blocklists. Let me know if you want help with that.

To share this proxy with other local devices, ensure they’re on the same network and allowed via ACL.
 

6. Block Ads and Save More Bandwidth with the Proxy

You can modify Squid to:

Block ad domains (using

acl

rules or a blacklist)

Limit download sizes

Restrict background updates or telemetry

Example rule to block a domain:

acl ads dstdomain .doubleclick.net .ads.google.com http_access deny ads


7. Use Aternative lightweight Proxy Privoxy (Lightweight filtering proxy) 

What is Privoxy?

Privoxy is a lightweight, highly customizable proxy server focused on privacy protection, content filtering, and web page optimization.

Unlike caching proxies (like Squid), Privoxy doesn’t store data locally—but it filters and blocks unnecessary traffic before it even reaches your browser.

7.1. Why Use Privoxy to Speed Up Internet?

Here's how Privoxy helps:

Feature Benefit
 Blocks Ads & Banners Reduces page load size and clutter
 Stops Trackers Prevents background data requests
Filters Pop-ups Improves usability and safety
Speeds Up Web Browsing By stripping unwanted content
Low Resource Usage Works on older or low-spec systems

 

Privoxy is easier to set up than Squid and usually much more simple and fits well if you want something simpler and more light weight and is also great for ad/tracker blocking.
To install and use it it comes to 4 simple steps

  1. Download from: https://www.privoxy.org/

  2. Install and run it.

  3. Configure browser/system to use proxy lets say on:

    127.0.0.1:8118

  4. Customize

    config.txt

    to add block rules.

7.2. Configure Your Web Browser or System Proxy

Set your browser/system to use the local Privoxy proxy:

Proxy address:

127.0.0.1

Port:

8118

On Windows:

Go to Settings > Network & Internet > Proxy

Enable Manual Proxy Setup

Enter:

Address:

127.0.0.1

Port:

8118

Save

7.3: Enable Privoxy Filtering and Blocking Rules

Privoxy comes with built-in rules for:

  • Ad blocking
  • Tracker blocking
  • Cookie management
  • Script filtering

You can customize filters in the configuration files via following configs:

Main config:

C:\\Program Files (x86)\\Privoxy\\config

 

Action files:

C:\\Program Files (x86)\\Privoxy\\default.action

 

Filter files:

C:\\Program Files (x86)\\Privoxy\\default.filter

 

7.4. Example to Block All Ads with Privoxy

Look in

default.action

and ensure these are uncommented:

 

{ +block }


Or add specific ad server domains:

{ +block{Ad Servers} }
.com.doubleclick.net
.ads.google.com
.adnxs.com

 

You can further use community-maintained blocklists for stronger Ads filtering.

 

Privoxy does not compress traffic, so to speed up even further with privoxy you might Compress traffic to do so use ziproxy (the http traffic compressor).

Now all your HTTP traffic is routed through Privoxy and you will notice search engines and repeatingly accessed websites pictures and Internet resources such as css / javscript / htmls etc. will give a boost !

Improve haproxy logging with custom log-format for better readiability

Friday, April 12th, 2024

Haproxy logging is a very big topic, worthy of many articles, but unfortunately not enough is written on the topic, perhaps for the reason haproxy is free software and most people who use it doesn't follow the philosophy of free software sharing but want to keep, the acquired knowledge on the topic for their own and if possible in the capitalist world most of us live to use it for a Load Balancer haproxy consultancy, consultancy fee or in their daily job as system administrators (web and middleware) or cloud specialist etc. 🙂

Having a good haproxy logging is very important as you need to debug issues with backend machines or some other devices throwing traffic to the HA Proxy.
Thus it is important to build a haproxy logging in a way that it provides most important information and the information is as simple as possible, so everyone can understand what is in without much effort and same time it contains enough debug information, to help you if you want to use the output logs with Graylog filters or process data with some monitoring advanced tool as Prometheus etc.

In our effort to optimize the way haproxy logs via a configured handler that sends the haproxy output to logging handler configured to log through rsyslog, we have done some experiments with logging arguments and came up with few variants, that we liked. In that article the idea is I share this set of logging  parameters with hope to help some other guy that starts with haproxy to build a good logging readable and easy to process with scripts log output from haproxy.

The criterias for a decent haproxy logging used are:

1. Log should be simple but not dumb
2. Should be concrete (and not too much complicated)
3. Should be easy to read for the novice and advanced sysadmin

Before starting, have to say that building the logging format seems tedious task but to make it fit your preference could take a lot of time, especially as logging parameters naming is hard to remember, thus the haproxy logging documentation log-format description table comes really handy:

Haproxy log-format paremeters ASCII table
 

 Please refer to the table for log-format defined variables :
 

+---+------+-----------------------------------------------+-------------+
| R | var  | field name (8.2.2 and 8.2.3 for description)  | type        |
+---+------+-----------------------------------------------+-------------+
|   | %o   | special variable, apply flags on all next var |             |
+---+------+-----------------------------------------------+-------------+
|   | %B   | bytes_read           (from server to client)  | numeric     |
| H | %CC  | captured_request_cookie                       | string      |
| H | %CS  | captured_response_cookie                      | string      |
|   | %H   | hostname                                      | string      |
| H | %HM  | HTTP method (ex: POST)                        | string      |
| H | %HP  | HTTP request URI without query string (path)  | string      |
| H | %HQ  | HTTP request URI query string (ex: ?bar=baz)  | string      |
| H | %HU  | HTTP request URI (ex: /foo?bar=baz)           | string      |
| H | %HV  | HTTP version (ex: HTTP/1.0)                   | string      |
|   | %ID  | unique-id                                     | string      |
|   | %ST  | status_code                                   | numeric     |
|   | %T   | gmt_date_time                                 | date        |
|   | %Ta  | Active time of the request (from TR to end)   | numeric     |
|   | %Tc  | Tc                                            | numeric     |
|   | %Td  | Td = Tt - (Tq + Tw + Tc + Tr)                 | numeric     |
|   | %Tl  | local_date_time                               | date        |
|   | %Th  | connection handshake time (SSL, PROXY proto)  | numeric     |
| H | %Ti  | idle time before the HTTP request             | numeric     |
| H | %Tq  | Th + Ti + TR                                  | numeric     |
| H | %TR  | time to receive the full request from 1st byte| numeric     |
| H | %Tr  | Tr (response time)                            | numeric     |
|   | %Ts  | timestamp                                     | numeric     |
|   | %Tt  | Tt                                            | numeric     |
|   | %Tw  | Tw                                            | numeric     |
|   | %U   | bytes_uploaded       (from client to server)  | numeric     |
|   | %ac  | actconn                                       | numeric     |
|   | %b   | backend_name                                  | string      |
|   | %bc  | beconn      (backend concurrent connections)  | numeric     |
|   | %bi  | backend_source_ip       (connecting address)  | IP          |
|   | %bp  | backend_source_port     (connecting address)  | numeric     |
|   | %bq  | backend_queue                                 | numeric     |
|   | %ci  | client_ip                 (accepted address)  | IP          |
|   | %cp  | client_port               (accepted address)  | numeric     |
|   | %f   | frontend_name                                 | string      |
|   | %fc  | feconn     (frontend concurrent connections)  | numeric     |
|   | %fi  | frontend_ip              (accepting address)  | IP          |
|   | %fp  | frontend_port            (accepting address)  | numeric     |
|   | %ft  | frontend_name_transport ('~' suffix for SSL)  | string      |
|   | %lc  | frontend_log_counter                          | numeric     |
|   | %hr  | captured_request_headers default style        | string      |
|   | %hrl | captured_request_headers CLF style            | string list |
|   | %hs  | captured_response_headers default style       | string      |
|   | %hsl | captured_response_headers CLF style           | string list |
|   | %ms  | accept date milliseconds (left-padded with 0) | numeric     |
|   | %pid | PID                                           | numeric     |
| H | %r   | http_request                                  | string      |
|   | %rc  | retries                                       | numeric     |
|   | %rt  | request_counter (HTTP req or TCP session)     | numeric     |
|   | %s   | server_name                                   | string      |
|   | %sc  | srv_conn     (server concurrent connections)  | numeric     |
|   | %si  | server_IP                   (target address)  | IP          |
|   | %sp  | server_port                 (target address)  | numeric     |
|   | %sq  | srv_queue                                     | numeric     |
| S | %sslc| ssl_ciphers (ex: AES-SHA)                     | string      |
| S | %sslv| ssl_version (ex: TLSv1)                       | string      |
|   | %t   | date_time      (with millisecond resolution)  | date        |
| H | %tr  | date_time of HTTP request                     | date        |
| H | %trg | gmt_date_time of start of HTTP request        | date        |
| H | %trl | local_date_time of start of HTTP request      | date        |
|   | %ts  | termination_state                             | string      |
| H | %tsc | termination_state with cookie status          | string      |
+---+------+-----------------------------------------------+-------------+
R = Restrictions : H = mode http only ; S = SSL only


Our custom log-format built in order to fulfill our needs is as this:

log-format %ci:%cp\ %H\ [%t]\ [%f\ %fi:%fp]\ [%b/%s\ %si:%sp]\ %Tw/%Tc/%Tt\ %B\ %ts\ %ac/%fc/%bc/%sc/%sq/%bq


Once you place the log-format as a default for all haproxy frontend / backends or for a custom defined ones, the output you will get when tailing the log is:

# tail -f /var/log/haproxy.log

Apr  5 21:47:19  10.42.73.83:23262 haproxy-fqdn-hostname.com [05/Apr/2024:21:46:23.879] [ft_FRONTEND_NAME 10.46.108.6:61310] [bk_BACKEND_NAME/bk_appserv3 10.75.226.88:61310] 1/0/55250 55 sD 4/2/1/0/0/0
Apr  5 21:48:14  10.42.73.83:57506 haproxy-fqdn-hostname.com [05/Apr/2024:21:47:18.925] [ft_FRONTEND_NAME 10.46.108.6:61310] [bk_BACKEND_NAME//bk_appserv1 10.35.242.134:61310] 1/0/55236 55 sD 4/2/1/0/0/0
Apr  5 21:49:09  10.42.73.83:46520 haproxy-fqdn-hostname.com [05/Apr/2024:21:48:13.956] [ft_FRONTEND_NAME 10.46.108.6:61310] [bk_BACKEND_NAME//bk_appserv2 10.75.226.89:61310] 1/0/55209 55 sD 4/2/1/0/0/0


If you don't care about extra space and logs being filled with more naming, another variant of above log-format, that makes it even more readable even for most novice sys admin or programmer would look like this:

log-format [%t]\ %H\ [IN_IP]\ %ci:%cp\ [FT_NAME]\ %f:%fp\ [FT_IP]\ %fi:%fp\ [BK_NAME]\ [%b/%s:%sp]\ [BK_IP]\ %si:%sp\ [TIME_WAIT]\ {%Tw/%Tc/%Tt}\ [CONN_STATE]\ {%B\ %ts}\ [STATUS]\ [%ac/%fc/%bc/%sc/%sq/%bq]

Once you apply the config test the haproxy.cfg to make sure no syntax errors during copy / paste from this page

haproxy-serv:~# haproxy -c -f /etc/haproxy/haproxy.cfg
Configuration file is valid


Next restart graceously haproxy 

haproxy-serv:~# /usr/sbin/haproxy -D -f /etc/haproxy/haproxy.cfg -p /var/run/haproxy.pid -sf $(cat /var/run/haproxy.pid)


Once you reload haproxy graceously without loosing the established connections in stead of restarting it completely via systemd sysctl restart haproxy:

 

2024-04-05T21:46:03+02:00 localhost haproxy[1897731]: 193.200.198.195:50714 haproxy-fqdn-hostname.com [05/Apr/2024:21:46:03.012] [FrotnendProd 10.55.0.20:27800] [BackendProd/<NOSRV> -:-] -1/-1/0 0 — 4/1/0/0/0/0
2024-04-05T21:46:03+02:00 localhost haproxy[1897731]: 193.100.193.189:54290 haproxy-fqdn-hostname.com
[05/Apr/2024:21:46:03.056] [FrotnendProd 10.55.0.20:27900] [BackendProd/<NOSRV> -:-] -1/-1/0 0 — 4/4/3/0/0/0
2024-04-05T21:46:03+02:00 localhost haproxy[1897731]: 193.100.193.190:26778 haproxy-fqdn-hostname.com
[05/Apr/2024:21:46:03.134] [FrotnendProd 10.55.0.20:27900] [BackendProd/tsefas02s 10.35.242.134:27900] 1/-1/0 0 CC 4/4/3/0/0/0

Note that in that log localhost haproxy[pid] is written by rsyslog, you can filter it out by modifying rsyslogd configurations

The only problem with this log-format is not everyone wants to have to much repeating information pointer on which field is what, but I personally liked this one as well because using it even though occuping much more space, makes the log much easier to process with perl or python scripting for data visualize and very for programs that does data or even "big data" analysis.

How to count number of ESTABLISHED state TCP connections to a Windows server

Wednesday, March 13th, 2024

count-netstat-established-connections-on-windows-server-howto-windows-logo-debug-network-issues-windows

Even if you have the background of a Linux system administrator, sooner or later you will have have to deal with some Windows hosts, thus i'll blog in this article shortly on how the established TCP if it happens you will have to administarte a Windows hosts or help a windows sysadmin noobie 🙂

In Linux it is pretty easy to check the number of established conenctions, because of the wonderful command wc (word count). with a simple command like:
 

$ netstat -etna |wc -l


Then you will get the number of active TCP connections to the machine and based on that you can get an idea on how busy the server is.

But what if you have to deal with lets say a Microsoft Windows 2012 /2019 / 2020 or 2022 Server, assuming you logged in as Administrator and you see the machine is quite loaded and runs multiple Native Windows Administrator common services such as IIS / Active directory Failover Clustering, Proxy server etc.
How can you identify the established number of connections via a simple command in cmd.exe?

1.Count ESTABLISHED TCP connections from Windows Command Line

Here is the answer, simply use netstat native windows command and combine it with find, like that and use the /i (ignores the case of characters when searching the string) /c (count lines containing the string) options

C:\Windows\system32>netstat -p TCP -n|  find /i "ESTABLISHED" /c
1268

Voila, here are number of established connections, only 1268 that is relatively low.
However if you manage Windows servers, and you get some kind of hang ups as part of the monitoring, it is a good idea to setup a script based on this simple command for at least Windows Task Scheduler (the equivallent of Linux's crond service) to log for Peaks in Established connections to see whether Server crashes are not related to High Rise in established connections.
Even better if company uses Zabbix / Nagios, OpenNMS or other  old legacy monitoring stuff like Joschyd even as of today 2024 used in some big of the TOP IT companies such as SAP (they were still using it about 4 years ago for their SAP HANA Cloud), you can set the script to run and do a Monitoring template or Alerting rules to draw you graphs and Trigger Alerts if your connections hits a peak, then you at least might know your Windows server is under a "Hackers" Denial of Service attack or there is something happening on the network, like Cisco Network Infrastructure Switch flappings or whatever.

Perhaps an example script you can use if you decide to implement the little nestat established connection checks Monitoring in Zabbix is the one i've writen about in the previous article "Calculate established connection from IP address with shell script and log to zabbix graphic".

2. Few Useful netstat options for the Windows system admin
 

C:\Windows\System32> netstat -bona


netstat-useful-arguments-for-the-windows-system-administrator

Cmd.exe will lists executable files, local and external IP addresses and ports, and the state in list form. You immediately see which programs have created connections or are listening so that you can find offenders quickly.

b – displays the executable involved in  creating the connection.
o – displays the owning process ID.
n – displays address and port numbers.
a – displays all connections and listening ports.

As you can see in the screenshot, by using netstat -bona you get which process has binded to which local address and the Process ID PID of it, that is pretty useful in debugging stuff.

3. Use a Third Party GUI tool to debug more interactively connection issues

If you need to keep an eye in interactive mode, sometimes if there are issues CurrPorts tool can be of a great help

currports-windows-network-connections-diagnosis-cports

CurrPorts Tool own Description

CurrPorts is network monitoring software that displays the list of all currently opened TCP/IP and UDP ports on your local computer. For each port in the list, information about the process that opened the port is also displayed, including the process name, full path of the process, version information of the process (product name, file description, and so on), the time that the process was created, and the user that created it.
In addition, CurrPorts allows you to close unwanted TCP connections, kill the process that opened the ports, and save the TCP/UDP ports information to HTML file , XML file, or to tab-delimited text file.
CurrPorts also automatically mark with pink color suspicious TCP/UDP ports owned by unidentified applications (Applications without version information and icons).

Sum it up

What we learned is how to calculate number of established TCP connections from command line, useful for scripting, how you can use netstat to display the process ID and Process name that relates to a used Local / Remote TCP connections, and how eventually you can use this to connect it to some monitoring tool to periodically report High Peaks with TCP established connections (usually an indicator of servere system issues).
 

How to calculate connections from IP address with shell script and log to Zabbix graphic

Thursday, March 11th, 2021

We had to test the number of connections incoming IP sorted by its TCP / IP connection state.

For example:

TIME_WAIT, ESTABLISHED, LISTEN etc.


The reason behind is sometimes the IP address '192.168.0.1' does create more than 200 connections, a Cisco firewall gets triggered and the connection for that IP is filtered out. To be able to know in advance that this problem is upcoming. a Small userparameter script is set on the Linux servers, that does print out all connections from IP by its STATES sorted out.

 

The script is calc_total_ip_match_zabbix.sh is below:

#!/bin/bash
#  check ESTIMATED / FIN_WAIT etc. netstat output for IPs and calculate total
# UserParameter=count.connections,(/usr/local/bin/calc_total_ip_match_zabbix.sh)
CHECK_IP='192.168.0.1';
f=0; 

 

for i in $(netstat -nat | grep "$CHECK_IP" | awk '{print $6}' | sort | uniq -c | sort -n); do

echo -n "$i ";
f=$((f+i));
done;
echo
echo "Total: $f"

 

root@pcfreak:/bashscripts# ./calc_total_ip_match_zabbix.sh 
1 TIME_WAIT 2 ESTABLISHED 3 LISTEN 

Total: 6

 

root@pcfreak:/bashscripts# ./calc_total_ip_match_zabbix.sh 
2 ESTABLISHED 3 LISTEN 
Total: 5


images/zabbix-webgui-connection-check1

To make process with Zabbix it is necessery to have an Item created and a Depedent Item.

 

webguiconnection-check1

webguiconnection-check1
 

webgui-connection-check2-item

images/webguiconnection-check1

Finally create a trigger to trigger alarm if you have more than or eqaul to 100 Total overall connections.


images/zabbix-webgui-connection-check-trigger

The Zabbix userparameter script should be as this:

[root@host: ~]# cat /etc/zabbix/zabbix_agentd.d/userparameter_webgui_conn.conf
UserParameter=count.connections,(/usr/local/bin/webgui_conn_track.sh)

 

Some collleagues suggested more efficient shell script solution for suming the overall number of connections, below is less time consuming version of script, that can be used for the calculation.
 

#!/bin/bash -x
# show FIN_WAIT2 / ESTIMATED etc. and calcuate total
count=$(netstat -n | grep "192.168.0.1" | awk ' { print $6 } ' | sort -n | uniq -c | sort -nr)
total=$((${count// /+}))
echo "$count"
echo "Total:" "$total"

      2 ESTABLISHED
      1 TIME_WAIT
Total: 3

 


Below is the graph built with Zabbix showing all the fluctuations from connections from monitored IP. ebgui-check_ip_graph

 

How to calculate connections from IP address with shell script and log to Zabbix graphic

Thursday, March 11th, 2021

We had to test the number of connections incoming IP sorted by its TCP / IP connection state.

For example:

TIME_WAIT, ESTABLISHED, LISTEN etc.


The reason behind is sometimes the IP address '192.168.0.1' does create more than 200 connections, a Cisco firewall gets triggered and the connection for that IP is filtered out. To be able to know in advance that this problem is upcoming. a Small userparameter script is set on the Linux servers, that does print out all connections from IP by its STATES sorted out.

 

The script is calc_total_ip_match_zabbix.sh is below:

#!/bin/bash
#  check ESTIMATED / FIN_WAIT etc. netstat output for IPs and calculate total
# UserParameter=count.connections,(/usr/local/bin/calc_total_ip_match_zabbix.sh)
CHECK_IP='192.168.0.1';
f=0; 

 

for i in $(netstat -nat | grep "$CHECK_IP" | awk '{print $6}' | sort | uniq -c | sort -n); do

echo -n "$i ";
f=$((f+i));
done;
echo
echo "Total: $f"

 

root@pcfreak:/bashscripts# ./calc_total_ip_match_zabbix.sh 
1 TIME_WAIT 2 ESTABLISHED 3 LISTEN 

Total: 6

 

root@pcfreak:/bashscripts# ./calc_total_ip_match_zabbix.sh 
2 ESTABLISHED 3 LISTEN 
Total: 5


To make process with Zabbix it is necessery to have an Item created and a Depedent Item.

images/zabbix-webgui-connection-check1

 

 

 

 

webguiconnection-check1

webguiconnection-check1
 

webgui-connection-check2-item

images/webguiconnection-check1

Finally create a trigger to trigger alarm if you have more than or eqaul to 100 Total overall connections.


images/zabbix-webgui-connection-check-trigger

The Zabbix userparameter script should be as this:
cat /etc/zabbix/zabbix_agentd.d/userparameter_webgui_conn.conf
UserParameter=count.connections,(/usr/local/bin/webgui_conn_track.sh)
 

Some collleagues suggested more efficient shell script solution for suming the overall number of connections, below is less time consuming version of script, that can be used for the calculation.
 

#!/bin/bash -x
# show FIN_WAIT2 / ESTIMATED etc. and calcuate total
count=$(netstat -n | grep "192.168.0.1" | awk ' { print $6 } ' | sort -n | uniq -c | sort -nr)
total=$((${count// /+}))
echo "$count"
echo "Total:" "$total"

 

      2 ESTABLISHED
      1 TIME_WAIT
Total: 3

 


Below is the graph built with Zabbix showing all the fluctuations from connections from monitored IP.
ebgui-check_ip_graph

Check the count and monitor of established / time_wait TCP, UDP connections on Linux and Windows with netstat command

Wednesday, February 6th, 2019

netstat-windows-linux-commands-to-better-understand-your-server-type-of-networrk-tcp-udp-connections

For me as a GNU / Linux sysadmin it is intuitive to check on a server the number of established connections / connections in time_wait state and so on .

I will not explain why this is necessery as every system administrator out there who had a performance or network issues due to server / applications connection overload or have been a target of Denial of Service (DoS)
or Distributed Denial of Service attacks (DDoS)  
is well aware that a number of connections in different states such as SYN_ACK /  TIME_WAIT or ESTABLISHED state could be very nasty thing and could cause a productive application or Infrastructure service to be downed for some time causing from thousands of Euros to even millions to some bussinesses as well as some amount of data loss …

To prevent this therefore sysadmins should always take a look periodically on the Connection states on the adminned server (and in this number I say not only sys admins but DevOps guys who are deploying micro-services for a customer in the Cloud – yes I believe Richard Stallman is right here they're clouding your minds :).

Even though cloud services could provide a very high amount of Hardware (CPU / Memory / Storage) resources, often for custom applications migrating the application in the Cloud does not solve it's design faults or even problems on a purely classical system administration level.

 

1. Get a statistic for FIN_WAIT1, FOREIGN, SYNC_RECV, LAST_ACK, TIME_WAIT, LISTEN and ESTABLISHED  Connections on GNU / Linux

 

On GNU / Linux and other Linux like UNIXes the way to do it is to grep out the TCP / UDP connection type you need via netstat a very useful cmd in that case is:

 

root@pcfreak:~# netstat -nat | awk '{print $6}' | sort | uniq -c | sort -n
      1 established)
      1 FIN_WAIT1
      1 Foreign
      1 SYN_RECV
      3 LAST_ACK
      4 FIN_WAIT2
      8 TIME_WAIT
     45 LISTEN
    147 ESTABLISHED

 

2. Netstat 1 liner to Get only established and time_wait connections state 

 

Other ways to check only TCP ESTABLISHED connections on Linux I use frequently are:

 

root@pcfreak:~# netstat -etna|grep -i establi|wc -l
145

 

netstat-connection-types-statistics-linux-established-time-wait-check-count

Or to get whole list of connections including the ones who are about to be esatablished in FIN_WAIT2, TIME_WAIT, SYN_RECV state:

 

root@pcfreak:~# netstat -tupen |wc -l
164

 

3. Other Linux useful one liner commands to track your connection types
 

netstat -n -p | grep SYN_REC | sort -u

List out the all IP addresses involved instead of just count.

netstat -n -p | grep SYN_REC | awk '{print $5}' | awk -F: '{print $1}'

 

List all the unique IP addresses of the node that are sending SYN_REC connection status.

netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n

 

Use netstat command to calculate and count the number of connections each IP address makes to the server.

netstat -anp |grep 'tcp\|udp' | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n

 

List count of number of connections the IPs are connected to the server using TCP or UDP protocol.

netstat -ntu | grep ESTAB | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -nr

 

Check on ESTABLISHED connections instead of all connections, and displays the connections count for each IP.

 

netstat -plan|grep :80|awk {'print $5'}|cut -d: -f 1|sort|uniq -c|sort -nk 1

 

Show and list IP address and its connection count that connect to port 80 on the server. Port 80 is used mainly by HTTP web page request.

Examples are taken from this nice blog post

 

4. Check the count of esatblished connections on M$ Windows

 

As I'm forced to optimize a couple of Microsoft Windows DNS servers which are really slow to resolve the
The logical question for me was how the Established and TIME_WAIT state connections then could be checked on Windows OS, after a quick investigation online I've come up with this:

 

C:\Users\admin> netstat -nao | find /i "estab" /c
78

 

netstat-check-number-of-established-ports-connections-windows
 

 

C:\Users\admin> netsatt -nao | find /i "time_wait" /c
333

 

 

If you're used to Linux watch command, then to do same on Windows OS (e.g. check the output of netstat) command every second
and print output use:

 

netstat –an 1 | find “3334”

 

Below commands will show stats for services listening on TCP port 3334

To find out which process on system sends packets to remote destination:

 

netstat –ano 1 | find “Dest_IP_Addr”

 

The -o parameter outputs the process ID (PID) responsible for the connection.
then if you need further you can find the respective process name with tasklist< cmd.
Another handy Windows netstat option is -b which will show EXE file running as long as
the related used DLL Libraries which use TCP / UDP .

Other useful netsatat Win example is to grep for a port and show all established connections for it with:

 

netstat –an 1 | find “8080” | find “ESTABLISHED”

 

5. Closure


Hopefully this article will give you some idea on what is eating your bandwidth connections or overloading your GNU / Linux – Windows systems. And will point you to the next in line logical thing to do optimization / tuning
settings to be made on your system for example if Linux with sysctl – see my previous relater article here

I'll be intested to hear from sysadm colleagoes for other useful ways to track connections perhaps with something like ss tool (a utility to investigate sockets).
Also any optimization hints that would cause servers less downtime and improve network / performance thouroughput is mostly welcome.