Posts Tagged ‘address’

How to install and configure AIDE ( Advanced Intrusion Detection Environment ) on Debian GNU / Linux 11 to monitor files for changes

Thursday, March 9th, 2023

aide-logo-linux

How to install and configure AIDE ( Advanced Intrusion Detection Environment ) on Debian GNU / Linux 11 to monitor files for changes

Having a intrusion detection system is essential to keeping a server security to good level and being compliant with PCI (Payment Card Industry) DSS Standards. It is a great thing for the sake to protect oneself from hackers assaults. 

There is plenty of Intrusion Detection systems available all around since many years, in the past one of main ones for Linux as older system administrators should remember was Tripwire – integrity tool for monitoring and alerting on specific file change(s) on a range of systems. 

Tripwire is still used today but many today prefer to use AIDE that is a free software replacement for Tripwire under GPL (General Public License), that is starting to become like a "standard"  for many Unix-like systems as an inexpensive baseline control and rootkit detection system.

In this article I'll explain shortly how to Install / Configure and Use AIDE to monitor, changes with files on the system.

But before proceeding it is worthy to mention on some of the alternatives companies and businesses choose to as an IDS (Intrusion Detection Systems), that is useful to give a brief idea of the sysadmins that has to deal with Security, on what is some of the main Intrusion Detection Systems adopted on UNIX OSes today:
 

  • Samhain

    An integrity checker and host intrusion detection system that can be used on single hosts as well as large, UNIX-based networks. It supports central monitoring as well as powerful (and new) stealth features to run undetected in memory, using steganography. Samhain is an open-source multiplatform application for POSIX systems (Unix, Linux, Cygwin/Windows).

  • OSSEC 
    OSSEC uses a centralized, cross-platform architecture allowing multiple systems to be monitored and managed.
     
  • Snort
    IDS which has the capabilities to prevent attacks. By taking a particular action based on traffic patterns, it can become an intrusion prevention system (IPS). – written in Pure C.
     
  • Zeek (Bro)
    Zeek helps to perform security monitoring by looking into the network's activity. It can find suspicious data streams. Based on the data, it alert, react, and integrate with other tools – written in C++.
  • Maltrail (Maltrail monitors for traffic on the network that might indicate system compromise or other bad behavior. It is great for intrusion detection and monitoring. – written in Python).

1. Install aide deb package

# apt -y install aide
…

root@haproxy2:~# aide -v
Aide 0.17.3

Compiled with the following options:

WITH_MMAP
WITH_PCRE
WITH_POSIX_ACL
WITH_SELINUX
WITH_XATTR
WITH_CAPABILITIES
WITH_E2FSATTRS
WITH_ZLIB
WITH_MHASH
WITH_AUDIT

Default config values:
config file: <none>
database_in: <none>
database_out: <none>

Available hashsum groups:
md5: yes
sha1: yes
sha256: yes
sha512: yes
rmd160: yes
tiger: yes
crc32: yes
crc32b: yes
haval: yes
whirlpool: yes
gost: yes
stribog256: no
stribog512: no

Default compound groups:
R: l+p+u+g+s+c+m+i+n+md5+acl+selinux+xattrs+ftype+e2fsattrs+caps
L: l+p+u+g+i+n+acl+selinux+xattrs+ftype+e2fsattrs+caps
>: l+p+u+g+i+n+acl+S+selinux+xattrs+ftype+e2fsattrs+caps
H: md5+sha1+rmd160+tiger+crc32+haval+gost+crc32b+sha256+sha512+whirlpool
X: acl+selinux+xattrs+e2fsattrs+caps

2. Prepare AIDE configuration and geenrate (initialize) database

Either you can use the default AIDE configuration which already has a preset rules for various files and directories to be monitored,
or you might add up additional ones.
 

  • For details on configuration of aide.conf accepted options "man aide.conf"

The rules and other configurations resides lays under  /etc/aide/ directory
 

The AIDE database is located under /var/lib/aide

root@server:~# ls -al /var/lib/aide/
общо 33008
drwxr-xr-x  2 root root     4096  9 мар 12:38 ./
drwxr-xr-x 27 root root     4096  9 мар 12:01 ../
-rw——-  1 root root 16895467  9 мар 16:03 aide.db
-rw——-  1 root root 16895467  9 мар 18:49 aide.db.new


Also, details about major setting rules config regarding how AIDE will run via cronjob as with most debian services are into /etc/default/aide

Default aide.conf config is in /etc/aide/aide.conf if you need custom stuff to do with it simply edit it.

Here is an Example:
Lets say you want to omit some directory to not be monitored by aide, which would otherwise do, i.e.
omit /var/log/* from monitoring

# At the end of file /etc/aide/aide.conf

add:

!/var/log
!/home/
!/var/lib
!/proc

  • Initialize the aide database first time

Run aideinit command, aideinit will create a new baseline database –  /var/lib/aide/aide.db.new (a baseline)
Note that, /var/lib/aide/aide.db is the old database that aide uses to check against for any changes of files / directories on the configured monitored filesystem objects.

root@server:~# aideinit
Running aide –init…

debug1: client_input_channel_req: channel 0 rtype keepalive@openssh.com reply 1
debug1: client_input_channel_req: channel 0 rtype keepalive@openssh.com reply 1
debug1: client_input_channel_req: channel 0 rtype keepalive@openssh.com reply 1
debug1: client_input_channel_req: channel 0 rtype keepalive@openssh.com reply 1
debug1: client_input_channel_req: channel 0 rtype keepalive@openssh.com reply 1
Start timestamp: 2023-03-09 12:06:16 +0200 (AIDE 0.17.3)
AIDE initialized database at /var/lib/aide/aide.db.new

Number of entries:      66971

—————————————————
The attributes of the (uncompressed) database(s):
—————————————————

/var/lib/aide/aide.db.new
 SHA256    : nVrYljiBFM/KaKCTjbaJtR2w6N8vc8qN
             DPObbo2UMVo=
 SHA512    : S1ZNB0DCqb4UTmuqaalTgiQ3UAltTOzO
             YNfEQJldp32q5ahplBo4/65uwgtGusMy
             rJC8nvxvYmh+mq+16kfrKA==
 RMD160    : xaUnfW1+/DJV/6FEm/nn1k1UKOU=
 TIGER     : nGYEbX281tsQ6T21VPx1Hr/FwBdwF4cK
 CRC32     : fzf7cg==
 HAVAL     : yYQw/87KUmRiRLSu5JcEIvBUVfsW/G9H
             tVvs6WqL/0I=
 WHIRLPOOL : 6b5y42axPjpUxWFipUs1PtbgP2q0KJWK
             FwFvAGxHXjZeCBPEYZCNkj8mt8MkXBTJ
             g83ZELK9GQBPLea7UF3tng==
 GOST      : sHAzx7hkr5H3q8TCSGCKjndEiZgcvCEL
             E45qcRb25tM=


End timestamp: 2023-03-09 12:38:30 +0200 (run time: 32m 14s)


Be patient now, go grab a coffee / tea or snack as the command might take up to few minutes for the aide to walk through the whole monitored filesystems and built its database.

root@server:~# echo cp /var/lib/aide/aide.db{.new,}
cp /var/lib/aide/aide.db.new /var/lib/aide/aide.db

 

root@server:~# cp /var/lib/aide/aide.db{.new,}

root@server:~# aide –check –config /etc/aide/aide.conf

Start timestamp: 2023-03-09 13:01:32 +0200 (AIDE 0.17.3)
AIDE found differences between database and filesystem!!

Summary:
  Total number of entries:      66972
  Added entries:                1
  Removed entries:              0
  Changed entries:              7

—————————————————
Added entries:
—————————————————

f+++++++++++++++++: /var/lib/aide/aide.db

—————————————————
Changed entries:
—————————————————

d =…. mc.. .. . : /etc/aide
d =…. mc.. .. . : /root
f <…. mci.H.. . : /root/.viminfo
f =…. mc..H.. . : /var/lib/fail2ban/fail2ban.sqlite3
d =…. mc.. .. . : /var/lib/vnstat
f =…. mc..H.. . : /var/lib/vnstat/vnstat.db
f >b… mc..H.. . : /var/log/sysstat/sa09

—————————————————
Detailed information about changes:
—————————————————

Directory: /etc/aide
 Mtime     : 2023-03-09 12:04:03 +0200        | 2023-03-09 12:51:11 +0200
 Ctime     : 2023-03-09 12:04:03 +0200        | 2023-03-09 12:51:11 +0200

Directory: /root
 Mtime     : 2023-03-09 12:06:13 +0200        | 2023-03-09 12:51:11 +0200
 Ctime     : 2023-03-09 12:06:13 +0200        | 2023-03-09 12:51:11 +0200

File: /root/.viminfo
 Size      : 18688                            | 17764
 Mtime     : 2023-03-09 12:06:13 +0200        | 2023-03-09 12:51:11 +0200
 Ctime     : 2023-03-09 12:06:13 +0200        | 2023-03-09 12:51:11 +0200
 Inode     : 133828                           | 133827
 SHA256    : aV54gi33aA/z/FuBj2ZioU2cTa9H16TT | dnFdLVQ/kx3UlTah09IgEMrJ/aYgczHe
             TzkLSxBDSB4=                     | DdxDAmPOSAM=

…

3. Test aide detects file changes

Create a new file and append some text and rerun the aide check

 

root@server:~# touch /root/test.txt
root@server:~# echo aaa > /root/test.txt
root@server:~# aide –check –config /etc/aide/aide.conf

 

Start timestamp: 2023-03-09 13:07:21 +0200 (AIDE 0.17.3)
AIDE found differences between database and filesystem!!

Summary:
  Total number of entries:      66973
  Added entries:                2
  Removed entries:              0
  Changed entries:              7

—————————————————
Added entries:
—————————————————

f+++++++++++++++++: /root/test.txt
f+++++++++++++++++: /var/lib/aide/aide.db

—————————————————
Changed entries:
—————————————————

d =…. mc.. .. . : /etc/aide
d =…. mc.. .. . : /root
f <…. mci.H.. . : /root/.viminfo
f =…. mc..H.. . : /var/lib/fail2ban/fail2ban.sqlite3
d =…. mc.. .. . : /var/lib/vnstat
f =…. mc..H.. . : /var/lib/vnstat/vnstat.db
f >b… mc..H.. . : /var/log/sysstat/sa09

….


The same command can be shortened for the lazy typist:

root@server:~# aide -c /etc/aide/aide.conf -C

The command will basically try to check the deviation between the AIDE database and the filesystem.

4. Limiting AIDES Integrity Checks to Specific Files / Directories

In order to limit the integrity checks to a specific entries for example /etc, pass the –limit REGEX option to AIDE check command where REGEX is the entry to check.

For example, check and update the database entries matching /etc, you would run aide command as shown below;
 

root@server:~# aide -c /etc/aide/aide.conf –limit /etc –check

 

AIDE found differences between database and filesystem!!
Limit: /etc

Summary:
  Total number of entries:      66791
  Added entries:                0
  Removed entries:              0
  Changed entries:              2

—————————————————
Changed entries:
—————————————————

d =…. mc.. .. . : /etc/aide
d =…. mc.. .. . : /etc/default

—————————————————
Detailed information about changes:
—————————————————

Directory: /etc/aide
 Mtime     : 2023-03-09 15:59:53 +0200        | 2023-03-09 16:43:03 +0200
 Ctime     : 2023-03-09 15:59:53 +0200        | 2023-03-09 16:43:03 +0200

Directory: /etc/default
 Mtime     : 2023-03-09 12:06:13 +0200        | 2023-03-09 18:42:12 +0200
 Ctime     : 2023-03-09 12:06:13 +0200        | 2023-03-09 18:42:12 +0200


—————————————————
The attributes of the (uncompressed) database(s):
—————————————————

/var/lib/aide/aide.db
 SHA256    : sjCxyIkr0nC/gTkNmn7DNqAQWttreDF6
             vSUV4jBoFY4=
 SHA512    : vNMpb54qxrbOk6S1Z+m9r0UwGvRarkWY
             0m50TfMvGElfZWR1I3SSaeTdORAZ4rQe
             17Oapo5+Sc0E2E+STO93tA==
 RMD160    : anhm5E6UlKmPYYJ4WYnWXk/LT3A=
 TIGER     : 5e1wycoF35/ABrRf7FNypZ45169VTuV4
 CRC32     : EAJlFg==
 HAVAL     : R5imONWRYgNGEfhBTc096K+ABnMFkMmh
             Hsqe9xt20NU=
 WHIRLPOOL : c6zySLliXNgnOA2DkHUdLTCG2d/T18gE
             4rdAuKaC+s7gqAGyA4p2bnDHhdd0v06I
             xEGY7YXCOXiwx8BM8xHAvQ==
 GOST      : F5zO2Ovtvf+f7Lw0Ef++ign1znZAQMHM
             AApQOiB9CqA=


End timestamp: 2023-03-09 20:02:18 +0200 (run time: 1m 32s)

5. Add the modified /root/test.txt to AIDE list of known modified files database
 

root@server:~# aide –update –config /etc/aide/aide.
  ERROR: cannot open config file '/etc/aide/aide.': No such file or directory

 

root@server:~# ​ aide –update –config /etc/aide/aide.conf
 

Start timestamp: 2023-03-09 18:45:17 +0200 (AIDE 0.17.3)
AIDE found differences between database and filesystem!!
New AIDE database written to /var/lib/aide/aide.db.new

Summary:
  Total number of entries:      66791
  Added entries:                0
  Removed entries:              0
  Changed entries:              8

—————————————————
Changed entries:
—————————————————

d =…. mc.. .. . : /etc/aide
d =…. mc.. .. . : /etc/default
d =…. mc.. .. . : /root
f >…. mci.H.. . : /root/.viminfo
f >…. mci.H.. . : /root/test.txt
f =…. mc..H.. . : /var/lib/fail2ban/fail2ban.sqlite3
d =…. mc.. .. . : /var/lib/vnstat
f =…. mc..H.. . : /var/lib/vnstat/vnstat.db

—————————————————
Detailed information about changes:
—————————————————

Directory: /etc/aide
 Mtime     : 2023-03-09 15:59:53 +0200        | 2023-03-09 16:43:03 +0200
 Ctime     : 2023-03-09 15:59:53 +0200        | 2023-03-09 16:43:03 +0200

Directory: /etc/default
 Mtime     : 2023-03-09 12:06:13 +0200        | 2023-03-09 18:42:12 +0200
 Ctime     : 2023-03-09 12:06:13 +0200        | 2023-03-09 18:42:12 +0200

Directory: /root
 Mtime     : 2023-03-09 15:59:53 +0200        | 2023-03-09 18:44:34 +0200
 Ctime     : 2023-03-09 15:59:53 +0200        | 2023-03-09 18:44:34 +0200

File: /root/.viminfo
 Size      : 16706                            | 16933
 Mtime     : 2023-03-09 15:59:53 +0200        | 2023-03-09 18:44:34 +0200
 Ctime     : 2023-03-09 15:59:53 +0200        | 2023-03-09 18:44:34 +0200
 Inode     : 136749                           | 133828
 SHA256    : KMHGoMVJo10BtafVrWIOLt3Ht9gK8bc+ | rrp8S3VftzZzvjBP1JC+PBpODv9wPKGw
             9uHh/z7iJWA=                     | TA+hyhTiY+U=
 SHA512    : ieDHy7ObSTfYm5d8DtYcHKxHya13CS65 | PDAJjyZ39uU3kKFo2lHBduTqxMDq4i01
             ObMYIRAre6IgvLslEs0ZodQFyrczMyRt | 1Kvm/h6xzFhHtFgjidtcemG8wDcjtfNF
             +d6SrW0gn3skKn2B7G09eQ==         | Z7LO230fgGeO7UepqtxZjQ==
 RMD160    : nUgg/G4zsVGKzVmmrqltuYUDvtg=     | jj61KAFONK92mj+u66RDJmxFhmI=
 TIGER     : 3vPSOrla5k+k2br1E2ES4eNiSZ2novFX | mn4kNCzd8SQr2ID2VSe4f4l0ta7pO/xo
 CRC32     : NDnMgw==                         | AyzVUQ==
 HAVAL     : Q9/KozxRiPbLEkaIfnBUZdEWftaF52Mw | 6jADKV6jg7ZVr/A/oMhR4NXc8TO1AOGW
             7tiR7DXhl0o=                     | NrYe+j6UcO0=
 WHIRLPOOL : vB/ZMCul4hN0aYd39gBu+HmZT/peRUI8 | mg6c1lYYVNZcy4mVzGojwraim8e3X2/R
             KDkaslNb8+YleoFWx0mbhAbkGurc0+jh | urVvEmbsgTuUCJOuf9+OrEACiF0fbe/x
             YPBviZIKcxUbTc2nGthTWw==         | t+BXnSQWk08OL9EI6gMGqA==
 GOST      : owVGTgU9BH3b0If569wQygw3FAbZIZde | ffx29GV2jaCB7XzuNjdiRzziIiZYnbi3
             eAfQfzlRPGY=                     | Ar7jyNMUutk=

File: /root/test.txt
 Size      : 4                                | 8
 Mtime     : 2023-03-09 13:07:12 +0200        | 2023-03-09 18:44:34 +0200
 Ctime     : 2023-03-09 13:07:12 +0200        | 2023-03-09 18:44:34 +0200
 Inode     : 133828                           | 136751
 SHA256    : F+aC8GC1+OR+oExcSFWQiwpa1hICImD+ | jUIZMGfiMdAlWFHu8mmmlml4qAGNQNL5
             UOEeywzAq3Y=                     | 6NhzJ1sYFZE=
 SHA512    : d+UmFKFBzvGadt5hk+nIRbjP//7PSXNl | ixn20lcEMDEtsJo3hO90Ea/wHWLCHcrz
             Pl16XRIUUPq2FCiQ4PeUcVciukJX7ijL | seBWunbBysY0z3BWcfgnN2vH05WfRfvA
             D045ZvGOEcnmL6a6vwp0jw==         | QiNtQS1tStuEdB3Voq54zQ==
 RMD160    : I6waxKN3rMx4WTz4VCUQXoNoxUg=     | urTh1j1t3UHchnJGnBG4lUZnjI4=
 TIGER     : cwUYgfKHcJnWXcA0pr/OKuxuoxh+b9lA | prstKqCfMXL39aVGFPA0kX4Q9x7a+hUn
 CRC32     : UD78Dw==                         | zoYiEA==
 HAVAL     : bdbKR9LvPgsYClViKiHx48fFixfIL/jA | ZdpdeMhw4MvKBgWsM4EeyUgerO86Rt82
             F3tjdc2Gm8Y=                     | W94fJFRWbrM=
 WHIRLPOOL : OLP0Y4oKcqW2yEvme8z419N1KE4TB9GJ | Xk8Ujo3IU2SzSqbJFegq7p1ockmrnxJF
             biHn/9XgrBz4fQiDJ8eHpx+0exA9hXmY | R3Rfstd1jWSwLFNTEwfbRRw+TARtRK50
             EbbakMJJdzLt1ipKWiV9gg==         | iWJeHLsD5dZ+CzV0tf4sUg==
 GOST      : ystISzoeH/ZznYrrXmxe4rwmybWMpGuE | GhMWNxg7Is0svJ+5LP+DVWbgt+CDQO+3
             0PzRnVEqnR8=                     | 08dwBuVAwB8=

File: /var/lib/fail2ban/fail2ban.sqlite3
 Mtime     : 2023-03-09 15:55:01 +0200        | 2023-03-09 18:45:01 +0200
 Ctime     : 2023-03-09 15:55:01 +0200        | 2023-03-09 18:45:01 +0200
 SHA256    : lLilXNleqSgHIP1y4o7c+oG5XyUPGzgi | NCJJ2H6xgCw/NYys1LMA7hOWwoOoxI8Y
             RHYH+zvlAL4=                     | 4SJygfqEioE=
 SHA512    : iQj2pNT4NES4fBcujzdlEEGZhDnkhKgc | ClQZ5HMOSayUNb//++eZc813fiMJcXnj
             QDlGFSAn6vi+RXesFCjCABT7/00eEm5/ | vTGs/2tANojoe6cqpsT/LaJ3QZXpmrfh
             ILcaqlQtBSLJgHjMQehzdg==         | syVak1I4n9yg8cDKEkZUvw==
 RMD160    : Xg4YU8YI935L+DLvkRsDanS4DGo=     | SYrQ27n+/1fvIZ7v+Sar/wQHulI=
 TIGER     : 2WhhPq9kuyeNJkOicDTDeOeJB8HR8zZe | o1LDZtRclri2KfZBe5J3D4YhM05UaP4E
 CRC32     : NQmi4A==                         | tzIsqg==
 HAVAL     : t1ET+84+8WgfwqlLy4R1Qk9qGZQRUbJI | MwVnjtM3dad/RuN2BfgsySX2DpfYq4qi
             z2J0ROGduXc=                     | H1pq6RYsA6o=
 WHIRLPOOL : xKSn71gFIVhk5rWJIBaYQASl0V+pGn+3 | m5LEXfhBbhWFg/d8CFJhklOurmRSkDSG
             N85R0tiCKsTZ2+LRkxDrzcVQdss2k8+z | LC/vICnbEWzLwrCuMwBi1/e5wDNIY8gK
             oqExhoXtPsMaREjpCugd3Q==         | mvGn40x+G4cCYNZ6lGT9Zg==
 GOST      : WptpUlfooIlUjzDHU8XGuOU2waRud5SR | i6K4COXU0nyZ1mL3ZBuGUPz/ZXTj8KKQ
             E/tnoBqk+q0=                     | L6VNyS8/X2Y=

Directory: /var/lib/vnstat
 Mtime     : 2023-03-09 16:00:00 +0200        | 2023-03-09 18:45:01 +0200
 Ctime     : 2023-03-09 16:00:00 +0200        | 2023-03-09 18:45:01 +0200

File: /var/lib/vnstat/vnstat.db
 Mtime     : 2023-03-09 16:00:00 +0200        | 2023-03-09 18:45:00 +0200
 Ctime     : 2023-03-09 16:00:00 +0200        | 2023-03-09 18:45:00 +0200
 SHA256    : X/lnJuuSo4jX4HRzxMBodnKHAjQFvugi | oqtY3HTNds/qDNFCRAEsfN5SuO0U5LRg
             2sh2c0u69x8=                     | otc5z1y+eGY=
 SHA512    : U/g8O6G8cuhsqCUCbrElxgiy+naJKPkI | y+sw4LX8mlDWkRJMX38TsYSo1DQzxPOS
             hG7vdH9rBINjakL87UWajT0s6WSy0pvt | 068otnzw2FSSlM5X5j5EtyJiY6Hd5P+A
             ALaTcDFKHBAmmFrl8df2nQ==         | jFiWStMbx+dQidXYZ4XFAw==
 RMD160    : F6YEjIIQu2J3ru7IaTvSemA9e34=     | bmVSaRKN2qU7qpEWkzfXFoH4ZK4=
 TIGER     : UEwLoeR6Qlf2oOI58pUCEDaWk0pHDkcY | 0Qb4nUqe3cKh/g5CQUnOXGfjZwJHjeWa
 CRC32     : Bv3/6A==                         | jvW6mg==
 HAVAL     : VD7tjHb8o8KTUo5xUH7eJEmTWgB9zjft | rumfiWJvy/sTK/09uj7XlmV3f7vj6KBM
             kOkzKxFWqqU=                     | qeOuKvu0Zjc=
 WHIRLPOOL : wR0qt8u4N8aQn8VQ+bmfrxB7CyCWVwHi | FVWDRE3uY6qHxLlJQLU9i9QggLW+neMj
             ADHpMTUxBEKOpOBlHTWXIk13qYZiD+o/ | Wt+Dj9Rz92BG9EomgLUgUkxfiVFO8cMq
             XtzTB4rMbxS4Z5PAdC/07A==         | WaR/KKq3Z7R8f/50tc9GMQ==
 GOST      : l3ibqMkHMSPpQ+9ok51/xBthET9+JQMd | qn0GyyCg67KRGP13At52tnviZfZDgyAm
             OZtiFGYXmgU=                     | c82NXSzeyV0=


—————————————————
The attributes of the (uncompressed) database(s):
—————————————————

/var/lib/aide/aide.db
 SHA256    : sjCxyIkr0nC/gTkNmn7DNqAQWttreDF6
             vSUV4jBoFY4=
 SHA512    : vNMpb54qxrbOk6S1Z+m9r0UwGvRarkWY
             0m50TfMvGElfZWR1I3SSaeTdORAZ4rQe
             17Oapo5+Sc0E2E+STO93tA==
 RMD160    : anhm5E6UlKmPYYJ4WYnWXk/LT3A=
 TIGER     : 5e1wycoF35/ABrRf7FNypZ45169VTuV4
 CRC32     : EAJlFg==
 HAVAL     : R5imONWRYgNGEfhBTc096K+ABnMFkMmh
             Hsqe9xt20NU=
 WHIRLPOOL : c6zySLliXNgnOA2DkHUdLTCG2d/T18gE
             4rdAuKaC+s7gqAGyA4p2bnDHhdd0v06I
             xEGY7YXCOXiwx8BM8xHAvQ==
 GOST      : F5zO2Ovtvf+f7Lw0Ef++ign1znZAQMHM
             AApQOiB9CqA=

/var/lib/aide/aide.db.new
 SHA256    : QRwubXnz8md/08n28Ek6DOsSQKGkLvuc
             gSZRsw6gRw8=
 SHA512    : 238RmI1PHhd9pXhzcHqM4+VjNzR0es+3
             6eiGNrXHAdDTz7GlAQQ4WfKeQJH9LdyT
             1r5ho/oXRgzfa2BfhKvTHg==
 RMD160    : GJWuX/nIPY05gz62YXxk4tWiH5I=
 TIGER     : l0aOjXlM4/HjyN9bhgBOvvCYeqoQyjpw
 CRC32     : KFz6GA==
 HAVAL     : a//4jwVxF22URf2BRNA612WOOvOrScy7
             OmI44KrNbBM=
 WHIRLPOOL : MBf+NeXElUvscJ2khIuAp+NDu1dm4h1f
             5tBQ0XrQ6dQPNA2HZfOShCBOPzEl/zrl
             +Px3QFV4FqD0jggr5sHK2g==
 GOST      : EQnPh6jQLVUqaAK9B4/U4V89tanTI55N
             K7XqZR9eMG4=


End timestamp: 2023-03-09 18:49:51 +0200 (run time: 4m 34s)
 

6. Substitute old aide database with the new that includes the modified files

As you see AIDE detected the changes in /root/test.txt

To apply the changes be known by AIDE for next time (e.g. this file was authorized and supposed to be written there) simply move the new generated database
to current aide database.

# copy generated DB to master DB
root@dlp:~# cp -p /var/lib/aide/aide.db.new /var/lib/aide/aide.db

7. Check once again to make sure recently modified files are no longer seen as changed by AIDE

Recheck again the database to make sure the files you wanted to omit are no longer mentioned as changed

root@server:~# aide –check –config /etc/aide/aide.conf
Start timestamp: 2023-03-09 16:23:05 +0200 (AIDE 0.17.3)
AIDE found differences between database and filesystem!!

Summary:
  Total number of entries:      66791
  Added entries:                0
  Removed entries:              0
  Changed entries:              3

—————————————————
Changed entries:
—————————————————

f =…. mc..H.. . : /var/lib/fail2ban/fail2ban.sqlite3
d =…. mc.. .. . : /var/lib/vnstat
f =…. mc..H.. . : /var/lib/vnstat/vnstat.db

—————————————————
Detailed information about changes:
—————————————————

File: /var/lib/fail2ban/fail2ban.sqlite3
 Mtime     : 2023-03-09 15:55:01 +0200        | 2023-03-09 16:25:02 +0200
 Ctime     : 2023-03-09 15:55:01 +0200        | 2023-03-09 16:25:02 +0200
 SHA256    : lLilXNleqSgHIP1y4o7c+oG5XyUPGzgi | MnWXC2rBMf7DNJ91kXtHXpM2c2xxF60X
             RHYH+zvlAL4=                     | DfLUQLHiSiY=
 SHA512    : iQj2pNT4NES4fBcujzdlEEGZhDnkhKgc | gxHVBxhGTKi0TjRE8/sn6/gtWsRw7Mfy
             QDlGFSAn6vi+RXesFCjCABT7/00eEm5/ | /wCfPlDK0dkRZEbr8IE2BNUhBgwwocCq
             ILcaqlQtBSLJgHjMQehzdg==         | zuazTy4N4x6X8bwOzRmY0w==
 RMD160    : Xg4YU8YI935L+DLvkRsDanS4DGo=     | +ksl9kjDoSU9aL4tR7FFFOK3mqw=
 TIGER     : 2WhhPq9kuyeNJkOicDTDeOeJB8HR8zZe | 9cvXZNbU+cp5dA5PLiX6sGncXd1Ff5QO
 CRC32     : NQmi4A==                         | y6Oixg==
 HAVAL     : t1ET+84+8WgfwqlLy4R1Qk9qGZQRUbJI | aPnCrHfmZAUm7QjROGEl6rd3776wO+Ep
             z2J0ROGduXc=                     | s/TQn7tH1tY=
 WHIRLPOOL : xKSn71gFIVhk5rWJIBaYQASl0V+pGn+3 | 9Hu6NBhz+puja7uandb21Nt6cEW6zEpm
             N85R0tiCKsTZ2+LRkxDrzcVQdss2k8+z | bTsq4xYA09ekhDHMQJHj2WpKpzZbA+t0
             oqExhoXtPsMaREjpCugd3Q==         | cttMDX8J8M/UadqfL8KZkQ==
 GOST      : WptpUlfooIlUjzDHU8XGuOU2waRud5SR | WUQfAMtye4wADUepBvblvgO+vBodS0Ej
             E/tnoBqk+q0=                     | cIbXy4vpPYc=

Directory: /var/lib/vnstat
 Mtime     : 2023-03-09 16:00:00 +0200        | 2023-03-09 16:25:01 +0200
 Ctime     : 2023-03-09 16:00:00 +0200        | 2023-03-09 16:25:01 +0200

File: /var/lib/vnstat/vnstat.db
 Mtime     : 2023-03-09 16:00:00 +0200        | 2023-03-09 16:25:01 +0200
 Ctime     : 2023-03-09 16:00:00 +0200        | 2023-03-09 16:25:01 +0200
 SHA256    : X/lnJuuSo4jX4HRzxMBodnKHAjQFvugi | N1lzhV3+tkDBud3AVlmIpDkU1c3Rqhnt
             2sh2c0u69x8=                     | YqE8naDicoM=
 SHA512    : U/g8O6G8cuhsqCUCbrElxgiy+naJKPkI | +8B9HvHhOp1C/XdlOORjyd3J2RtTbRBF
             hG7vdH9rBINjakL87UWajT0s6WSy0pvt | b0Moo2Gj+cIxaMCu5wOkgreMp6FloqJR
             ALaTcDFKHBAmmFrl8df2nQ==         | UH4cNES/bAWtonmbj4W7Vw==
 RMD160    : F6YEjIIQu2J3ru7IaTvSemA9e34=     | 8M6TIOHt0NWgR5Mo47DxU28cp+4=
 TIGER     : UEwLoeR6Qlf2oOI58pUCEDaWk0pHDkcY | Du9Ue0JA2URO2tiij31B/+663OaWKefR
 CRC32     : Bv3/6A==                         | v0Ai4w==
 HAVAL     : VD7tjHb8o8KTUo5xUH7eJEmTWgB9zjft | XA+vRnMNdVGFrO+IZtEA0icunWqBGaCf
             kOkzKxFWqqU=                     | leR27LN4ejc=
 WHIRLPOOL : wR0qt8u4N8aQn8VQ+bmfrxB7CyCWVwHi | HG31dNEEcak2zZGR24W7FDJx8mh24MaJ
             ADHpMTUxBEKOpOBlHTWXIk13qYZiD+o/ | BQNhqkuS6R/bmlhx+P+eQ/JimwPAPOaM
             XtzTB4rMbxS4Z5PAdC/07A==         | xWG7cMETIXdT9sUOUal8Sw==
 GOST      : l3ibqMkHMSPpQ+9ok51/xBthET9+JQMd | y6Ek/TyAMGV5egkfCu92Y4qqk1Xge8c0
             OZtiFGYXmgU=                     | 3ONXRveOlr0=


—————————————————
The attributes of the (uncompressed) database(s):
—————————————————

/var/lib/aide/aide.db
 SHA256    : sjCxyIkr0nC/gTkNmn7DNqAQWttreDF6
             vSUV4jBoFY4=
 SHA512    : vNMpb54qxrbOk6S1Z+m9r0UwGvRarkWY
             0m50TfMvGElfZWR1I3SSaeTdORAZ4rQe
             17Oapo5+Sc0E2E+STO93tA==
 RMD160    : anhm5E6UlKmPYYJ4WYnWXk/LT3A=
 TIGER     : 5e1wycoF35/ABrRf7FNypZ45169VTuV4
 CRC32     : EAJlFg==
 HAVAL     : R5imONWRYgNGEfhBTc096K+ABnMFkMmh
             Hsqe9xt20NU=
 WHIRLPOOL : c6zySLliXNgnOA2DkHUdLTCG2d/T18gE
             4rdAuKaC+s7gqAGyA4p2bnDHhdd0v06I
             xEGY7YXCOXiwx8BM8xHAvQ==
 GOST      : F5zO2Ovtvf+f7Lw0Ef++ign1znZAQMHM
             AApQOiB9CqA=


End timestamp: 2023-03-09 16:27:33 +0200 (run time: 4m 28s)

As you can see there are no new added entries for /root/test.txt and some other changed records for vnstat service as well as fail2ban ones, so the Intrusion detection system works just as we expected it.

8. Configure Email AIDE changed files alerting Email recipient address

From here on aide package has set its own cron job which is automatically doing the check operation every day and any new file modifications will be captured and alerts sent to local root@localhost mailbox account, so you can check it out later with mail command.

If you want to sent the Email alert for any files modifications occured to another email, assuming that you have a locally running SMTP server with a mail relay to send to external mails, you can do it via /etc/default/aide via:

MAILTO=root


For example change it to a FQDN email address

MAILTO=external_mail@your-mail.com

9.Force AIDE to run AIDE at specitic more frequent time intervals

You can as well install a cron job to execute AIDE at specific time intervals, as of your choice

Lets say you want to run a custom prepared set of files to monitor in /etc/aide/aide_custom_config.conf configure a new cronjob like below:

root@server:~# crontab -u root -e
*/5 * * * * aide -c /etc/aide/aide_custom_config.conf -u && cp /var/lib/custom-aide/aide.db{.new,}


This will execute AIDE system check every 5 minutse and email the report to ealier configured email username@whatever-your-smtp.com via /etc/default/aide

10. Check the output of AIDE for changes – useful for getting a files changes from aide from scripts

Check the command exit status.

root@server:~# echo $?

According to AIDE man pages, the AIDE’s exit status is normally 0 if no errors occurred. Except when the –check, –compare or –update command was requested, in which case the exit status is defined as:

   1 * (new files detected?)     +

   2 * (removed files detected?) +

   4 * (changed files detected?)


   Since  those three cases can occur together, the respective error codes are added. For example, if there are new files and removed files detected, the exit status will be 1 + 2 = 3.

   Additionally, the following exit codes are defined for generic error conditions in aide help manual:

   14 Error writing error

   15 Invalid argument error

   16 Unimplemented function error

   17 Invalid configureline error

   18 IO error

   19 Version mismatch error

PLEASE CONSIDER

  • That AIDE checks might be resource intensive
    and could cause a peak in CPU use and have a negative effect on lets very loaded application server machines,
    thus causing a performance issuea during integrity checks !
     
  • If you are scanning file system wide and you do it frequent, be sure to provide “enough” resources or schedule the scan at a times that the Linux host will be less used !
     
  • Whenever you made any AIDE configuration changes, remember to initialize the database to create a baseline !

Tags: , , , , , , , , , , , , , , , , , ,
Posted in Computer Security, Debian, Linux | 1 Comment »

Disable VNC on KVM Virtual Machine without VM restart / How to Change VNC listen address

Monday, February 28th, 2022

disable-vnc-port-listener-on-a-KVM-ran-virtual-machine-virsh-libvirt-libvirt-architecture-design

Say you have recently run a new KVM Virtual machine, have connected via VNC on lets say the default tcp port 5900 
installed a brand new Linux OS using a VNC client to connect, such as:
TightVNC / RealVNC if connecting from Windows Client machine or Vncviewer / Remmina if connecting from Linux / BSD  and now 
you want to turn off the VNC VM listener server either for security reasons to make sure some script kiddie random scanner did not manage to connect and take control over your VM or just because, you will be only further using the new configured VM only via SSH console sessions as they call it in modern times to make a buziness buzz out of it a headless UNIX server (server machines connected a network without a Physical monitor attached to it).


The question comes then how can be the KVM VNC listener on TCP port 5900 be completely disabled?

One way of course is to filter out with a firewall 5900 completely either on a Switch Level (lets say on a Cisco equipment catalist in front of the machine) or the worst solution to  locally filter directly on the server with firewalld or iptables chain rules.
 

1. Disable KVM VNC Port listener via VIRSH VM XML edit

The better way of course  is to completely disable the VNC using KVM, that is possible through the virsh command interface.
By editing the XML Virtual Machine configuration and finding the line about vnc confiuguration with:

root@server:/kvm/disk# virsh edit pcfreakweb
Domain pcfreakweb XML configuration not changed.

like:

<graphics type='vnc' port='5900' autoport='yes' listen='0.0.0.0'>
      <listen type='address' address='0.0.0.0'/>


and set value to undefined:

port='-1'


virsh-KVM-disable-VNC-port-listener-virsh-xml-edit-screenshot

Modifying the XML however will require you to reboot the Virtual Machine for which XML was editted. This might be not possible
if you have a running production server already configured with Apache / Proxy / PostgreSQL / Mail or any other Internet public service.

2. Disable VNC KVM TCP port 5900 to a dynamic running VM without a machine reboot


Thus if you want to remove the KVM VNC Port Listener on 5900 without a VM shutdown / reboot you can do it via KVM's virsh client interface.

root@server:/kvm/disk# virsh
Welcome to virsh, the virtualization interactive terminal.

Type:  'help' for help with commands
       'quit' to quit

virsh # qemu-monitor-command pcfreakweb –hmp change  vnc none

 

The virsh management user interface client, can do pretty much more of real time VM changes, it is really useful to use it if you have KVM Hypervisor hosts with 10+ Virtual machines and it if you have to deal with KVM machines on daily, do specific changes to the VMs on how VM networks are configured, information on HV hardware, configure / reconfigure storage volumes to VMs etc, take some time to play with it 🙂

Tags: , , , , , , , , , , , , , , , , ,
Posted in Curious Facts, Linux, Virtual Machines | No Comments »

How to calculate connections from IP address with shell script and log to Zabbix graphic

Thursday, March 11th, 2021

We had to test the number of connections incoming IP sorted by its TCP / IP connection state.

For example:

TIME_WAIT, ESTABLISHED, LISTEN etc.


The reason behind is sometimes the IP address '192.168.0.1' does create more than 200 connections, a Cisco firewall gets triggered and the connection for that IP is filtered out. To be able to know in advance that this problem is upcoming. a Small userparameter script is set on the Linux servers, that does print out all connections from IP by its STATES sorted out.

 

The script is calc_total_ip_match_zabbix.sh is below:

#!/bin/bash
#  check ESTIMATED / FIN_WAIT etc. netstat output for IPs and calculate total
# UserParameter=count.connections,(/usr/local/bin/calc_total_ip_match_zabbix.sh)
CHECK_IP='192.168.0.1';
f=0; 

 

for i in $(netstat -nat | grep "$CHECK_IP" | awk '{print $6}' | sort | uniq -c | sort -n); do

echo -n "$i ";
f=$((f+i));
done;
echo
echo "Total: $f"

 

root@pcfreak:/bashscripts# ./calc_total_ip_match_zabbix.sh 
1 TIME_WAIT 2 ESTABLISHED 3 LISTEN 

Total: 6

 

root@pcfreak:/bashscripts# ./calc_total_ip_match_zabbix.sh 
2 ESTABLISHED 3 LISTEN 
Total: 5


To make process with Zabbix it is necessery to have an Item created and a Depedent Item.

images/zabbix-webgui-connection-check1

 

 

 

 

webguiconnection-check1

webguiconnection-check1
 

webgui-connection-check2-item

images/webguiconnection-check1

Finally create a trigger to trigger alarm if you have more than or eqaul to 100 Total overall connections.


images/zabbix-webgui-connection-check-trigger

The Zabbix userparameter script should be as this:
cat /etc/zabbix/zabbix_agentd.d/userparameter_webgui_conn.conf
UserParameter=count.connections,(/usr/local/bin/webgui_conn_track.sh)
 

Some collleagues suggested more efficient shell script solution for suming the overall number of connections, below is less time consuming version of script, that can be used for the calculation.
 

#!/bin/bash -x
# show FIN_WAIT2 / ESTIMATED etc. and calcuate total
count=$(netstat -n | grep "192.168.0.1" | awk ' { print $6 } ' | sort -n | uniq -c | sort -nr)
total=$((${count// /+}))
echo "$count"
echo "Total:" "$total"

 

      2 ESTABLISHED
      1 TIME_WAIT
Total: 3

 


Below is the graph built with Zabbix showing all the fluctuations from connections from monitored IP.
ebgui-check_ip_graph

Tags: , , , , , ,
Posted in Linux, Various, Web and CMS, Zabbix | No Comments »

Hack: Using ssh / curl or wget to test TCP port connection state to remote SSH, DNS, SMTP, MySQL or any other listening service in PCI environment servers

Wednesday, December 30th, 2020

using-curl-ssh-wget-to-test-tcp-port-opened-or-closed-for-web-mysql-smtp-or-any-other-linstener-in-pci-linux-logo

If you work on PCI high security environment servers in isolated local networks where each package installed on the Linux / Unix system is of importance it is pretty common that some basic stuff are not there in most cases it is considered a security hole to even have a simple telnet installed on the system. I do have experience with such environments myself and thus it is pretty daunting stuff so in best case you can use something like a simple ssh client if you're lucky and the CentOS / Redhat / Suse Linux whatever distro has openssh-client package installed.
If you're lucky to have the ssh onboard you can use telnet in same manner as netcat or the swiss army knife (nmap) network mapper tool to test whether remote service TCP / port is opened or not. As often this is useful, if you don't have access to the CISCO / Juniper or other (networ) / firewall equipment which is setting the boundaries and security port restrictions between networks and servers.

Below is example on how to use ssh client to test port connectivity to lets say the Internet, i.e.  Google / Yahoo search engines.
 

[root@pciserver: /home ]# ssh -oConnectTimeout=3 -v google.com -p 23
OpenSSH_7.9p1 Debian-10+deb10u2, OpenSSL 1.1.1g  21 Apr 2020
debug1: Connecting to google.com [172.217.169.206] port 23.
debug1: connect to address 172.217.169.206 port 23: Connection timed out
debug1: Connecting to google.com [2a00:1450:4017:80b::200e] port 23.
debug1: connect to address 2a00:1450:4017:80b::200e port 23: Cannot assign requested address
ssh: connect to host google.com port 23: Cannot assign requested address
root@pcfreak:/var/www/images# ssh -oConnectTimeout=3 -v google.com -p 80
OpenSSH_7.9p1 Debian-10+deb10u2, OpenSSL 1.1.1g  21 Apr 2020
debug1: Connecting to google.com [172.217.169.206] port 80.
debug1: connect to address 172.217.169.206 port 80: Connection timed out
debug1: Connecting to google.com [2a00:1450:4017:807::200e] port 80.
debug1: connect to address 2a00:1450:4017:807::200e port 80: Cannot assign requested address
ssh: connect to host google.com port 80: Cannot assign requested address
root@pcfreak:/var/www/images# ssh google.com -p 80
ssh_exchange_identification: Connection closed by remote host
root@pcfreak:/var/www/images# ssh google.com -p 80 -v -oConnectTimeout=3
OpenSSH_7.9p1 Debian-10+deb10u2, OpenSSL 1.1.1g  21 Apr 2020
debug1: Connecting to google.com [172.217.169.206] port 80.
debug1: connect to address 172.217.169.206 port 80: Connection timed out
debug1: Connecting to google.com [2a00:1450:4017:80b::200e] port 80.
debug1: connect to address 2a00:1450:4017:80b::200e port 80: Cannot assign requested address
ssh: connect to host google.com port 80: Cannot assign requested address
root@pcfreak:/var/www/images# ssh google.com -p 80 -v -oConnectTimeout=5
OpenSSH_7.9p1 Debian-10+deb10u2, OpenSSL 1.1.1g  21 Apr 2020
debug1: Connecting to google.com [142.250.184.142] port 80.
debug1: connect to address 142.250.184.142 port 80: Connection timed out
debug1: Connecting to google.com [2a00:1450:4017:80c::200e] port 80.
debug1: connect to address 2a00:1450:4017:80c::200e port 80: Cannot assign requested address
ssh: connect to host google.com port 80: Cannot assign requested address
root@pcfreak:/var/www/images# ssh google.com -p 80 -v
OpenSSH_7.9p1 Debian-10+deb10u2, OpenSSL 1.1.1g  21 Apr 2020
debug1: Connecting to google.com [172.217.169.206] port 80.
debug1: Connection established.
debug1: identity file /root/.ssh/id_rsa type 0
debug1: identity file /root/.ssh/id_rsa-cert type -1
debug1: identity file /root/.ssh/id_dsa type -1
debug1: identity file /root/.ssh/id_dsa-cert type -1
debug1: identity file /root/.ssh/id_ecdsa type -1
debug1: identity file /root/.ssh/id_ecdsa-cert type -1
debug1: identity file /root/.ssh/id_ed25519 type -1
debug1: identity file /root/.ssh/id_ed25519-cert type -1
debug1: identity file /root/.ssh/id_xmss type -1
debug1: identity file /root/.ssh/id_xmss-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_7.9p1 Debian-10+deb10u2
debug1: ssh_exchange_identification: HTTP/1.0 400 Bad Request

 


debug1: ssh_exchange_identification: Content-Type: text/html; charset=UTF-8


debug1: ssh_exchange_identification: Referrer-Policy: no-referrer


debug1: ssh_exchange_identification: Content-Length: 1555


debug1: ssh_exchange_identification: Date: Wed, 30 Dec 2020 14:13:25 GMT


debug1: ssh_exchange_identification:


debug1: ssh_exchange_identification: <!DOCTYPE html>

debug1: ssh_exchange_identification: <html lang=en>

debug1: ssh_exchange_identification:   <meta charset=utf-8>

debug1: ssh_exchange_identification:   <meta name=viewport content="initial-scale=1, minimum-scale=1, width=device-width">

debug1: ssh_exchange_identification:   <title>Error 400 (Bad Request)!!1</title>

debug1: ssh_exchange_identification:   <style>

debug1: ssh_exchange_identification:     *{margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px}* > body{background:url(//www.google.com/images/errors/robot.png) 10
debug1: ssh_exchange_identification: 0% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}@media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.g
debug1: ssh_exchange_identification: oogle.com/images/branding/googlelogo/1x/googlelogo_color_150x54dp.png) no-repeat;margin-left:-5px}@media only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat 0
debug1: ssh_exchange_identification: % 0%/100% 100%;-moz-border-image:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) 0}}@media only screen and (-webkit-min-device-pixel-ratio:2){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_
debug1: ssh_exchange_identification: color_150x54dp.png) no-repeat;-webkit-background-size:100% 100%}}#logo{display:inline-block;height:54px;width:150px}

debug1: ssh_exchange_identification:   </style>

debug1: ssh_exchange_identification:   <a href=//www.google.com/><span id=logo aria-label=Google></span></a>

debug1: ssh_exchange_identification:   <p><b>400.</b> <ins>That\342\200\231s an error.</ins>

debug1: ssh_exchange_identification:   <p>Your client has issued a malformed or illegal request.  <ins>That\342\200\231s all we know.</ins>

ssh_exchange_identification: Connection closed by remote host

 

Here is another example on how to test remote host whether a certain service such as DNS (bind) or telnetd is enabled and listening on remote local network  IP with ssh

[root@pciserver: /home ]# ssh 192.168.1.200 -p 53 -v -oConnectTimeout=5
OpenSSH_7.9p1 Debian-10+deb10u2, OpenSSL 1.1.1g  21 Apr 2020
debug1: Connecting to 192.168.1.200 [192.168.1.200] port 53.
debug1: connect to address 192.168.1.200 port 53: Connection timed out
ssh: connect to host 192.168.1.200 port 53: Connection timed out

[root@server: /home ]# ssh 192.168.1.200 -p 23 -v -oConnectTimeout=5
OpenSSH_7.9p1 Debian-10+deb10u2, OpenSSL 1.1.1g  21 Apr 2020
debug1: Connecting to 192.168.1.200 [192.168.1.200] port 23.
debug1: connect to address 192.168.1.200 port 23: Connection timed out
ssh: connect to host 192.168.1.200 port 23: Connection timed out


But what if Linux server you have tow work on is so paranoid that you even the ssh client is absent? Well you can use anything else that is capable of doing a connectivity to remote port such as wget or curl. Some web servers or application servers usually have wget or curl as it is integral part for some local shell scripts doing various operation needed for proper services functioning or simply to test locally a local or remote listener services, if that's the case we can use curl to connect and get output of a remote service simulating a normal telnet connection like this:

host:~# curl -vv 'telnet://remote-server-host5:22'
* About to connect() to remote-server-host5 port 22 (#0)
*   Trying 10.52.67.21… connected
* Connected to aflpvz625 (10.52.67.21) port 22 (#0)
SSH-2.0-OpenSSH_5.3

Now lets test whether we can connect remotely to a local net remote IP's Qmail mail server with curls telnet simulation mode:

host:~#  curl -vv 'telnet://192.168.0.200:25'
* Expire in 0 ms for 6 (transfer 0x56066e5ab900)
*   Trying 192.168.0.200…
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x56066e5ab900)
* Connected to 192.168.0.200 (192.168.0.200) port 25 (#0)
220 This is Mail Pc-Freak.NET ESMTP

Fine it works, lets now test whether a remote server who has MySQL listener service on standard MySQL port TCP 3306 is reachable with curl

host:~#  curl -vv 'telnet://192.168.0.200:3306'
* Expire in 0 ms for 6 (transfer 0x5601fafae900)
*   Trying 192.168.0.200…
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x5601fafae900)
* Connected to 192.168.0.200 (192.168.0.200) port 3306 (#0)
Warning: Binary output can mess up your terminal. Use "–output -" to tell
Warning: curl to output it to your terminal anyway, or consider "–output
Warning: <FILE>" to save to a file.
* Failed writing body (0 != 107)
* Closing connection 0
root@pcfreak:/var/www/images#  curl -vv 'telnet://192.168.0.200:3306'
* Expire in 0 ms for 6 (transfer 0x5598ad008900)
*   Trying 192.168.0.200…
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x5598ad008900)
* Connected to 192.168.0.200 (192.168.0.200) port 3306 (#0)
Warning: Binary output can mess up your terminal. Use "–output -" to tell
Warning: curl to output it to your terminal anyway, or consider "–output
Warning: <FILE>" to save to a file.
* Failed writing body (0 != 107)
* Closing connection 0

As you can see the remote connection is returning binary data which is unknown to a standard telnet terminal thus to get the output received we need to pass curl suggested arguments.

host:~#  curl -vv 'telnet://192.168.0.200:3306' –output –
* Expire in 0 ms for 6 (transfer 0x55b205c02900)
*   Trying 192.168.0.200…
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x55b205c02900)
* Connected to 192.168.0.200 (192.168.0.200) port 3306 (#0)
g


The curl trick used to troubleshoot remote port to remote host from a Windows OS host which does not have telnet installed by default but have curl instead.

Also When troubleshooting vSphere Replication, it is often necessary to troubleshoot port connectivity as common Windows utilities are not available.
As Curl is available in the VMware vCenter Server Appliance command line interface.

On servers where curl is not there but you have wget is installed you can use it also to test a remote port

 

# wget -vv -O /dev/null http://google.com:554 –timeout=5
–2020-12-30 16:54:22–  http://google.com:554/
Resolving google.com (google.com)… 172.217.169.206, 2a00:1450:4017:80b::200e
Connecting to google.com (google.com)|172.217.169.206|:554… failed: Connection timed out.
Connecting to google.com (google.com)|2a00:1450:4017:80b::200e|:554… failed: Cannot assign requested address.
Retrying.

–2020-12-30 16:54:28–  (try: 2)  http://google.com:554/
Connecting to google.com (google.com)|172.217.169.206|:554… ^C

As evident from output the port 554 is filtered in google which is pretty normal.

If curl or wget is not there either as a final alternative you can either install some perl, ruby, python or bash script etc. that can opens a remote socket to the remote IP.

Tags: , , , , , , , , , , , , , , , , , , , , ,
Posted in Curious Facts, Hacks, System Administration, Various | No Comments »

Use multiple certificates using one IP address (same IP address) on IIS Windows web server

Saturday, October 24th, 2020

If you had to administer some Windows webservers based on IIS and you're coming from the Linux realm, it would be really confusing on how you can use a single IP address to have binded multiple domain certificates.

For those who have done it on linux, they know Apache and other webservers in recent versions support the configuration Directive of a Wildcard instead of IP through the SNI extension capble to capture in the header of the incoming SSL connection the exact domain and match it correctly against the domain with the respective certificate.  Below is what I mean, lets say you have a website called yourdomain.com and you want this domain to be pointing to another location for example to yourdomain1.com

For example in Apache Webserver this is easily done by defining 2 separate virtualhost configuration files similar to below:

/etc/apache2/sites-available/yourdomain.com

<Virtualhost *>
…
Servername yourdomain.com
ServerAlias www.yourdomain.com
….

        SSLCertificateFile /etc/letsencrypt/live/yourdomain1.com/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/yourdomain1.com/privkey.pem
</VirtualHost>


 

/etc/apache2/sites-available/yourdomain1.com

<Virtualhost *>
…
Servername yourdomain1.com
ServerAlias yourdomain1.com
…

 

        SSLCertificateFile /etc/letsencrypt/live/yourdomain1.com/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/yourdomain1.com/privkey.pem
</VirtualHost>
 

Unfortunately for those who still run legacy Windows servers  with IIS version 7 / 7.5 your only option is to use separate IP addresses (or ports, but not really acceptable for public facing sites) and to bind each site with it's SSL certificate to that IP address.

IIS ver. 8+ supports the Server Name Indication extension of TLS which will allow you to bind multiple SSL sites to the same IP address/port based on the host name. It will be transparent and the binding will work the same as with non-HTTPS sites.

In Microsoft IIS Webserver to configure, it is not possible to simply edit some configurations but you have to do it the clicking way as usually happen in Windows. thus you will need to have generated the Domain Certificate requests and so on and then you can simply do as pointed in below screenshots.

howto-install-iis-8-webserver-ssl-sni-certificate-windows-screenshot
 

iis-config-domain-alias-windows-server-iis-8-webserver

iis-config-domain-alias-windows-server-iis-8-webserver-1

iis-config-domain-alias-windows-server-iis-8-webserver-2

iis-config-domain-alias-windows-server-iis-8-webserver-3

iis-config-domain-alias-windows-server-iis-8-webserver-4
 

Tags: , , , , , , , , ,
Posted in IIS, System Administration, Web and CMS, Windows | No Comments »

How to clear ARP cache on Linux / Windows for a single IP address / Flush All IPs ARP cache

Wednesday, December 11th, 2019

linux-how-to-delete-modify-arp-cache-entries-after-IP-is-migrated-from-one-server-or-VPN-host-to-another-resized

On times of Public Internet IP migration or Local IPs between Linux servers or especially in clustered Linux Application Services running on environments like Pacemaker / Corosync / Heartbeat with services such as Haproxy.
Once an IP gets migrated due to complex network and firewall settings often the Migrated IP from Linux Server 1 (A) to Linux Server 2 (B) keeps time until a request to reload the Internet server IP ARP cache with to point to the new IP location, causing a disruption of accessibility to the Newly configured IP address on the new locations. I will not get much into details here what are the ARP (Address Resolution protocol) and Network ARP records on a Network attached Computer and how they correspond uniquely to each IP address assigned on Ethernet or Aliased network Interfaces (eth0 eth0:1 eth0:2) . But in this article, I'll briefly explain once IP Version 4 address is migrated from one server Data Center location to another DC, how the unique corresponding ARP record kept in OS system memory should be flushed in the ARP corresponding Operating System so called ARP table (of which you should think as a logical block in memory keeping a Map of where IP addresses are located physically on a Network recognized by the corresponding Unique MAC Address.
 

1. List the current ARP cache entries do

Arp is part of net-tools on Debian GNU / Linux and is also available and installed by default on virtually any Linux distribution Fedora / CentOS / RHEL / Ubuntu / Arch Linux and even m$ Windows NT / XP / 2000 / 10 / whatever, the only difference is Linux tool has a bit of more functionality and has a bit more complex use.
Easiest use of arp on GNU / Linux OS-es is.
 

# arp -an 

sample-IP-address-list-with-the-assigned-ARP-cache-mac-addresses
The -a lists all records and -n flag is here to omit IP resolving as some IPs are really slow to resolve and output of command could get lagged.

2. Delete one IP entry from the cache


Assuming only one IP address was migrated, if you want to delete the IP entry from local ARP table on any interface:
 

# arp -d 192.168.0.8


It is useful to delete an ARP cached entry for IP address only on a certain interface, to do so:
 

# /usr/sbin/arp -i eth1 -d 10.0.0.1

 

3. Create ARP entry MAC address with a static one for tightened security


A useful Hack is to (assign) / bind specific Static MAC addresses to be static in the ARP cache, this is very useful to improve security and fight an ARP poisoning attacks.
Doing so is pretty easy, to do so:

Above will staticly make IP 192.168.0.8 to always appear in the ARP cache table to the MAC 00:50:ba:85:85:ca. So even if we have another system with the same MAC
trying to spoof our location and thus break our real record location for the Hostname in the network holding in reality the MAC 00:50:ba:85:85:ca, poisoning us
trying to make our host to recognize 192.168.0.8 to a different address this will not happen as the static ARP will be kept unchanged in ARP caching table.
 

 # arp -s 192.168.0.8 00:50:ba:85:85:ca

 

4. Flush all ARP records only for specific Ethernet Interface


After the IP on interface was migrated run:

 

# ip link set arp off dev eth0 ; ip link set arp on dev eth0

 

5. Remove a set of few IPs only migrated ARP cache entries

 

# for i in 192.168.0.1 10.0.0.1 172.168.0.3; do sudo arp -d $i; done


Once old ARP entries are removed the arp command would return as:

 

linux:~$ arp
? (192.168.0.8) at <incomplete>  on eth1
? (172.168.0.3) at <incomplete>  on eth2


The 192.168.0.8 / 172.168.0.3 entry now shows as incomplete, which means the ARP entry will be refreshed when it is needed again, this would also depend
on the used network switches / firewalls in the network settings so often could take up to 1 minute or so..
 

6. Flush all ARP table records on Linux

flush-all-arp-cache-addresses-on-linux-howto-with-ip-command

 

# ip -s -s neigh flush all

 

7. Delete ARP Cache on FreeBSD and other BSDs

# arp -d -a 

 

8.  Flush arp cache on Windows

Run command prompt as Administrator -> (cmd.exe)  and do:

C:\> ipconfig /all
netsh interface ip delete arpcache

 

9. Monitoring the arp table


On servers with multiple IP addresses, where you expect a number of IP addresses migrated to change it is useful to use watch + arp like so:
 

# watch -n 0.1 'arp -an'
…

The -n 0.1 will make the arp -an be rerun every 10 miliseconds and by the way is a useful trick to monitor stuff returned by commands that needs a higher refresh frequency.
 

Conclusion


In short in this article, was explained how to list your arp cache table.The arp command is also available both on Linux and Windows) and as integral part of OS networking it is useful to check thoroghfully to its man page (man arp).
Explained was how to create Static ARP table records to prevent ARP poisoning attacks on a server.
I went through how to delete only a single ARP records (in case if) only certain IPs on a host are changed and an ARP cache entry reload is needed, as well as how to flush the complete set of ARP records need to get refreshed, sometimes useful on networks with Buggy Network Switches or when completely changing the set of IP-addresses assigned on a server host.

Tags: , , , , ,
Posted in Everyday Life, Linux, Networking, Windows | No Comments »