Posts Tagged ‘mail’

Display Content of SSL certificate .pem file with openssl command

Thursday, October 11th, 2018

display-content-of-pem-der-and-scr-file-how-to-view-pem-file-linux

If you have generated a .pem formatted SSL certificate or you have multiple .pem SSL certificates and you're not sure which .pem file is generated for which domain / subdomain it is useful to Display content of SSL Certificate .PEM file with openssl command.

Viewing certificate's content is also very useful if you have hosted multiple websites hosted on a server and you want to check which of the SSLs assigned in the Virtualhosts has Expired (for example if you have domains that expire in short term period (365 days).


1. How to Display Content of SSL certificate .pem file?

 

root@pcfreak:~# openssl x509 -in cert.pem -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            04:d1:ad:55:91:f3:f9:ef:3e:53:ea:2c:3a:f4:5f:e6:ce:c1
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
        Validity
            Not Before: Oct 10 17:49:34 2018 GMT
            Not After : Jan  8 17:49:34 2019 GMT
        Subject: CN = mail.pc-freak.net

        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Modulus:
                    00:bb:b0:c9:1c:83:82:61:47:f9:c2:73:60:c0:48:
                    e6:0c:f2:a1:ff:db:ae:f1:84:17:14:5d:fc:a3:b2:
                    e4:00:3a:d1:85:42:90:da:41:a9:e9:a8:af:20:3d:
                    12:ef:8e:ca:61:a1:71:f2:cc:43:bf:40:0d:fa:08:
                    7d:d9:61:2b:ea:5d:30:e0:52:43:db:18:30:92:0c:
                    2c:ce:87:93:84:ea:91:61:b7:70:db:11:7c:b6:a4:
                    33:de:d8:3f:d6:61:47:42:f2:36:12:7f:3d:e3:f7:
                    5b:11:3e:1c:f0:af:96:cd:61:8a:1a:a0:f0:b5:23:
                    65:73:b6:b4:9c:19:a7:09:dd:43:96:37:ac:48:fc:
                    21:07:02:52:67:26:2c:81:24:f4:d7:10:e6:f4:12:
                    69:53:ef:91:2a:15:6a:21:06:22:ea:fe:31:38:82:
                    b4:5a:b5:9b:67:90:16:b8:31:e8:27:38:f2:41:b9:
                    19:02:8f:c7:6e:e1:2c:84:75:19:6d:bb:30:3b:d2:
                    02:f0:65:f1:76:82:15:9c:ce:31:3a:d4:7c:83:ca:
                    d1:f9:e1:b7:76:f6:78:93:47:d2:00:f9:63:aa:94:
                    41:d4:78:d0:ee:bc:e6:e9:14:14:e4:ae:54:31:88:
                    f8:58:8d:7b:3e:9f:87:5c:f2:04:e5:07:e0:4c:9a:
                    81:eb
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment
            X509v3 Extended Key Usage:
                TLS Web Server Authentication, TLS Web Client Authentication
            X509v3 Basic Constraints: critical
                CA:FALSE
            X509v3 Subject Key Identifier:
                DB:AB:81:E3:14:5F:6D:BE:B4:78:7B:5E:7D:FB:66:BF:56:37:C5:1D
            X509v3 Authority Key Identifier:
                keyid:A8:4A:6A:63:04:7D:DD:BA:E6:D1:39:B7:A6:45:65:EF:F3:A8:EC:A1

 

            Authority Information Access:
                OCSP – URI:http://ocsp.int-x3.letsencrypt.org
                CA Issuers – URI:http://cert.int-x3.letsencrypt.org/

            X509v3 Subject Alternative Name:
                DNS:mail.pc-freak.net
            X509v3 Certificate Policies:
                Policy: 2.23.140.1.2.1
                Policy: 1.3.6.1.4.1.44947.1.1.1
                  CPS: http://cps.letsencrypt.org
                  User Notice:
                    Explicit Text: This Certificate may only be relied upon by Relying Parties and only in accordance with the Certificate Policy found at https://letsencrypt.org/repository/

            CT Precertificate SCTs:
                Signed Certificate Timestamp:
                    Version   : v1 (0x0)
                    Log ID    : E2:69:4B:AE:26:E8:E9:40:09:E8:86:1B:B6:3B:83:D4:
                                3E:E7:FE:74:88:FB:A4:8F:28:93:01:9D:DD:F1:DB:FE
                    Timestamp : Oct 10 18:49:34.453 2018 GMT
                    Extensions: none
                    Signature : ecdsa-with-SHA256
                                30:46:02:21:00:D6:DE:47:AD:D2:32:BE:BE:DD:B3:EB:
                                EE:84:9E:02:8A:4F:33:E2:63:21:D5:F7:4D:47:82:92:
                                AB:B9:0A:49:62:02:21:00:E8:7D:17:81:32:E3:4F:CF:
                                2D:79:8C:97:46:E1:EF:5E:99:F4:8A:8B:B5:6D:23:5F:
                                05:84:E2:14:6A:56:8E:A0
                Signed Certificate Timestamp:
                    Version   : v1 (0x0)
                    Log ID    : 29:3C:51:96:54:C8:39:65:BA:AA:50:FC:58:07:D4:B7:
                                6F:BF:58:7A:29:72:DC:A4:C3:0C:F4:E5:45:47:F4:78
                    Timestamp : Oct 10 18:49:34.451 2018 GMT
                    Extensions: none
                    Signature : ecdsa-with-SHA256
                                30:44:02:20:6C:8E:E7:E2:70:AD:33:A6:5C:E0:89:84:
                                FB:0B:F6:E1:5C:05:06:0A:A8:DB:8B:1C:7A:D0:52:99:
                                5F:3F:A2:64:02:20:4B:CD:0B:E7:A0:27:04:31:19:18:
                                58:99:51:73:49:6B:77:25:A7:E7:5B:10:8C:BD:ED:54:
                                03:DD:40:E4:2D:31
    Signature Algorithm: sha256WithRSAEncryption
         9c:86:b3:34:64:af:ac:9d:c4:d3:a7:cc:fc:8a:32:18:75:95:
         95:47:9b:9c:3c:0e:3b:61:f9:88:61:38:1a:a6:92:69:3d:14:
         6a:53:13:14:65:e6:ca:fa:b9:8e:48:c9:d4:73:f6:e4:74:8a:
         1f:2b:f2:14:86:f1:18:55:26:1b:a0:97:89:15:0b:62:c6:2b:
         27:81:6f:60:af:55:68:b3:2c:5b:10:56:a2:7d:28:cb:8e:fc:
         f0:21:65:78:9b:3a:52:d3:9d:27:ff:d7:24:95:de:0f:d8:3d:
         a2:43:6e:fc:a5:2d:f2:ad:37:e9:ea:db:b5:75:b8:7c:ad:23:
         45:1d:bd:fe:4e:36:c7:f4:e2:3d:47:c9:06:fc:cb:75:ba:d4:
         0a:90:17:ea:e1:7f:49:e6:68:27:97:8a:70:c7:50:e9:19:4a:
         8a:21:18:26:79:a3:61:ff:1b:26:9e:fe:85:8f:20:ed:c6:4d:
         c1:0e:04:21:a8:05:d4:29:69:99:53:63:81:c7:d5:58:71:df:
         02:b5:94:c9:36:48:c9:35:80:ab:71:78:d9:12:f6:f5:10:25:
         3d:38:c5:40:75:25:b1:95:18:d8:1c:96:f1:c6:1a:d2:c4:99:
         f5:01:2e:f4:e1:4a:1f:10:42:0e:34:ed:92:8e:53:9f:c2:7b:
         11:51:78:6a
—–BEGIN CERTIFICATE—–
MIIGDTCCBPWgAwIBAgISBNGtVZHz+e8+U+osOvRf5s7BMA0GCSqGSIb3DQEBCwUA
MEoxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MSMwIQYDVQQD
ExpMZXQncyBFbmNyeXB0IEF1dGhvcml0eSBYMzAeFw0xODEwMTAxNzQ5MzRaFw0x
OTAxMDgxNzQ5MzRaMBwxGjAYBgNVBAMTEW1haWwucGMtZnJlYWsubmV0MIIBIjAN
BgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAu7DJHIOCYUf5wnNgwEjmDPKh/9uu
8YQXFF38o7LkADrRhUKQ2kGp6aivID0S747KYaFx8sxDv0AN+gh92WEr6l0w4FJD
2xgwkgwszoeThOqRYbdw2xF8tqQz3tg/1mFHQvI2En894/dbET4c8K+WzWGKGqDw
tSNlc7a0nBmnCd1DljesSPwhBwJSZyYsgST01xDm9BJpU++RKhVqIQYi6v4xOIK0
WrWbZ5AWuDHoJzjyQbkZAo/HbuEshHUZbbswO9IC8GXxdoIVnM4xOtR8g8rR+eG3
dvZ4k0fSAPljqpRB1HjQ7rzm6RQU5K5UMYj4WI17Pp+HXPIE5QfgTJqB6wIDAQAB
o4IDGTCCAxUwDgYDVR0PAQH/BAQDAgWgMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggr
BgEFBQcDAjAMBgNVHRMBAf8EAjAAMB0GA1UdDgQWBBTbq4HjFF9tvrR4e159+2a/
VjfFHTAfBgNVHSMEGDAWgBSoSmpjBH3duubRObemRWXv86jsoTBvBggrBgEFBQcB
AQRjMGEwLgYIKwYBBQUHMAGGImh0dHA6Ly9vY3NwLmludC14My5sZXRzZW5jcnlw
dC5vcmcwLwYIKwYBBQUHMAKGI2h0dHA6Ly9jZXJ0LmludC14My5sZXRzZW5jcnlw
dC5vcmcvMBwGA1UdEQQVMBOCEW1haWwucGMtZnJlYWsubmV0MIH+BgNVHSAEgfYw
gfMwCAYGZ4EMAQIBMIHmBgsrBgEEAYLfEwEBATCB1jAmBggrBgEFBQcCARYaaHR0
cDovL2Nwcy5sZXRzZW5jcnlwdC5vcmcwgasGCCsGAQUFBwICMIGeDIGbVGhpcyBD
ZXJ0aWZpY2F0ZSBtYXkgb25seSBiZSByZWxpZWQgdXBvbiBieSBSZWx5aW5nIFBh
cnRpZXMgYW5kIG9ubHkgaW4gYWNjb3JkYW5jZSB3aXRoIHRoZSBDZXJ0aWZpY2F0
ZSBQb2xpY3kgZm91bmQgYXQgaHR0cHM6Ly9sZXRzZW5jcnlwdC5vcmcvcmVwb3Np
dG9yeS8wggEEBgorBgEEAdZ5AgQCBIH1BIHyAPAAdwDiaUuuJujpQAnohhu2O4PU
Puf+dIj7pI8okwGd3fHb/gAAAWZfUA/1AAAEAwBIMEYCIQDW3ket0jK+vt2z6+6E
ngKKTzPiYyHV901HgpKruQpJYgIhAOh9F4Ey40/PLXmMl0bh716Z9IqLtW0jXwWE
4hRqVo6gAHUAKTxRllTIOWW6qlD8WAfUt2+/WHopctykwwz05UVH9HgAAAFmX1AP
8wAABAMARjBEAiBsjuficK0zplzgiYT7C/bhXAUGCqjbixx60FKZXz+iZAIgS80L
56AnBDEZGFiZUXNJa3clp+dbEIy97VQD3UDkLTEwDQYJKoZIhvcNAQELBQADggEB
AJyGszRkr6ydxNOnzPyKMhh1lZVHm5w8Djth+YhhOBqmkmk9FGpTExRl5sr6uY5I
ydRz9uR0ih8r8hSG8RhVJhugl4kVC2LGKyeBb2CvVWizLFsQVqJ9KMuO/PAhZXib
OlLTnSf/1ySV3g/YPaJDbvylLfKtN+nq27V1uHytI0Udvf5ONsf04j1HyQb8y3W6
1AqQF+rhf0nmaCeXinDHUOkZSoohGCZ5o2H/Gyae/oWPIO3GTcEOBCGoBdQpaZlT
Y4HH1Vhx3wK1lMk2SMk1gKtxeNkS9vUQJT04xUB1JbGVGNgclvHGGtLEmfUBLvTh
Sh8QQg407ZKOU5/CexFReGo=
—–END CERTIFICATE—–

 

Same way a .der files content / encryption algorithm and domain name could be grasped.
 

root@pcfreak:~# openssl x509 -in cert.der -inform der -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            ad:c2:96:6f:4b:db:31:5c
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN = example.com
        Validity
            Not Before: Jun 22 04:00:37 2015 GMT
            Not After : Jul 22 04:00:37 2015 GMT

        Subject: CN = example.com
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (512 bit)

                Modulus:
                    00:ac:75:73:b4:51:ed:1f:dd:ae:70:52:43:fc:df:
                    c7:5b:d0:2c:75:1b:14:b8:75:01:04:10:e5:1f:03:
                    65:45:dd:df:a7:9f:34:ae:fd:be:e9:05:84:df:47:
                    16:81:d9:89:4b:ce:8e:6d:1c:fa:95:44:e8:af:84:
                    74:4f:ed:c2:e5
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Subject Key Identifier:
                26:CF:C8:ED:4B:D7:94:B2:E4:25:03:58:24:8F:04:C0:74:D5:97:8A
            X509v3 Authority Key Identifier:
                keyid:26:CF:C8:ED:4B:D7:94:B2:E4:25:03:58:24:8F:04:C0:74:D5:97:8A

 

            X509v3 Basic Constraints:
                CA:TRUE
    Signature Algorithm: sha256WithRSAEncryption
         0c:8b:ff:12:80:9e:4c:90:bc:26:b0:96:20:ab:76:0c:64:71:
         d2:15:48:a5:33:f6:47:e4:03:df:76:5e:0f:cd:e1:1b:5e:d1:
         4d:c2:1f:8d:b8:63:2f:c9:7d:6e:5c:3b:cb:cd:a3:d0:d8:27:
         74:66:a3:76:06:a5:fb:81:3a:b6
—–BEGIN CERTIFICATE—–
MIIBdTCCAR+gAwIBAgIJAK3Clm9L2zFcMA0GCSqGSIb3DQEBCwUAMBYxFDASBgNV
BAMMC2V4YW1wbGUuY29tMB4XDTE1MDYyMjA0MDAzN1oXDTE1MDcyMjA0MDAzN1ow
FjEUMBIGA1UEAwwLZXhhbXBsZS5jb20wXDANBgkqhkiG9w0BAQEFAANLADBIAkEA
rHVztFHtH92ucFJD/N/HW9AsdRsUuHUBBBDlHwNlRd3fp580rv2+6QWE30cWgdmJ
S86ObRz6lUTor4R0T+3C5QIDAQABo1AwTjAdBgNVHQ4EFgQUJs/I7UvXlLLkJQNY
JI8EwHTVl4owHwYDVR0jBBgwFoAUJs/I7UvXlLLkJQNYJI8EwHTVl4owDAYDVR0T
BAUwAwEB/zANBgkqhkiG9w0BAQsFAANBAAyL/xKAnkyQvCawliCrdgxkcdIVSKUz
9kfkA992Xg/N4Rte0U3CH424Yy/JfW5cO8vNo9DYJ3Rmo3YGpfuBOrY=
—–END CERTIFICATE—–

 

2. How to display content and info about .CSR (Certificate Signing request)

 

root@pcfreak:~# openssl req -in cert.csr -noout -text
 

Certificate Request:
    Data:
        Version: 1 (0x0)
        Subject: C = BG, ST = BG, L = Dobrich, O = Pc Freak, CN = mail.pc-freak.net, emailAddress = hipo@pc-freak.net
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Modulus:
                    00:b1:83:a7:07:62:19:d4:60:95:58:49:de:b3:48:
                    a8:39:31:fa:5a:bd:2b:d6:73:94:50:36:72:74:18:
                    58:b6:27:d3:0b:26:75:15:a8:ba:1b:77:a7:c9:54:
                    96:1f:c7:8d:da:bd:c9:17:91:53:85:9e:0a:f4:71:
                    3c:fb:d6:e4:75:41:c1:95:32:e1:22:fc:7b:1f:36:
                    23:12:00:ca:37:27:d5:f9:9b:29:48:4a:51:95:d1:
                    40:d0:ea:94:51:98:98:6b:d3:d9:79:1d:a1:65:bb:
                    a9:d1:ab:c9:46:6e:03:ee:24:45:e5:f2:73:e5:f4:
                    82:4a:08:57:b1:06:52:c3:cc:42:9a:02:5b:7a:7c:
                    bd:34:d5:5f:d7:ba:ef:27:d5:3d:97:16:69:06:c7:
                    c1:06:5e:d9:07:16:3f:a3:61:50:9d:dd:ea:95:32:
                    f1:ee:93:82:48:df:20:8b:ae:d2:95:89:05:e4:3d:
                    0c:d7:e1:cf:07:ae:55:84:11:06:92:be:34:b4:a2:
                    a1:ce:07:06:bf:21:bc:80:e2:03:d2:85:b4:64:02:
                    8d:cd:d2:86:1c:49:41:52:43:a8:12:f8:ef:2c:f4:
                    be:a0:dc:ac:ea:27:3a:f9:ab:ab:27:da:28:63:1d:
                    10:5a:4f:b8:51:42:40:ae:be:c0:2d:e9:a3:5a:5a:
                    23:7f
                Exponent: 65537 (0x10001)
        Attributes:
            a0:00
    Signature Algorithm: sha256WithRSAEncryption
         47:f0:54:cd:5e:46:6f:2c:cc:48:7e:85:f0:a8:96:10:ca:a3:
         15:98:77:d3:02:95:8c:67:84:e3:55:d2:0c:e8:d5:a7:ba:82:
         95:fb:ce:73:4c:bc:8f:da:85:97:0c:a8:59:32:b3:a4:af:0a:
         80:4c:78:7f:62:cd:1b:00:01:e8:51:27:9c:eb:75:29:80:e9:
         99:24:fc:86:e2:09:28:be:47:5a:1d:bf:b1:b4:c4:29:4e:6e:
         f3:70:b4:58:f8:d9:a6:63:03:8b:a1:ef:ee:6d:1a:35:33:1e:
         b2:32:25:c1:33:37:3d:46:82:37:9b:0d:4c:40:20:ae:ff:e0:
         cc:51:a2:6b:dd:74:26:d6:93:26:89:c7:76:29:13:cf:6e:5a:
         0f:7c:1b:f5:80:be:3b:6a:a3:c0:10:cd:07:1e:a2:31:8b:49:
         94:d7:63:cf:93:8d:80:03:75:4a:76:b4:cd:14:fe:96:62:61:
         6b:96:8f:c0:a5:ef:67:c7:5e:c0:a5:4b:4f:95:57:b6:43:03:
         8b:6d:10:5f:ab:f2:95:54:ba:85:8e:8b:c1:99:ea:fd:3f:5e:
         23:01:d4:27:f3:e9:20:37:c4:05:47:30:67:94:53:f0:87:27:
         48:73:57:55:f2:70:04:b1:e9:29:eb:2e:2c:9a:cc:55:f4:cc:
         a4:71:c2:5a


That's all folks 🙂

 

How to force logrorate process logs / Make logrotate changes take effect immediately

Sunday, April 10th, 2016

how-to-force-logrorate-to-process-logs-make-logrorate-changes-take-effect-immediately-log-rotate-300x299

Dealing with logrorate as admins we need to change or add new log-rorate configurations (on most Linux distributions configs are living uder
/etc/logrotate.d/
 

logrotate uses crontab to work. It's scheduled work, not as daemon, so usually no need to reload its configuration.
When the crontab executes logrotate, it will use your new config file automatically.

Most of the logrotate setups I've seen on various distros runs out of the /etc/cron.daily

$ ls -l /etc/cron.daily/logrotate 
-rwxr-xr-x 1 root root 180 May 18  2014 /etc/cron.daily/logrotate

Here is content of cron job scheduled script:

$ cat /etc/cron.daily/logrorate

#!/bin/sh /usr/sbin/logrotate /etc/logrotate.conf EXITVALUE=$? if [ $EXITVALUE != 0 ]; then /usr/bin/logger -t logrotate "ALERT exited abnormally with [$EXITVALUE]" fi exit 0

Configurations change to lograte configs takes effect on next crontab run,
but what If you need to test your config you can also execute logrotate
on your own with below ommand:

 

logrotate -vf /etc/logrotate.conf 

If you encounter some issues with just modified or newly added logrorate script to check out the status of last logrorate executing bunch of log-rotate scripts run on Debian  / Ubuntu etc. deb based Linux:

cat /var/lib/logrotate/status

Or on RHEL, Fedora, CentOS Linux


cat /var/lib/logrotate.status

logrotate state -- version 2

 

"/var/log/syslog" 2016-4-9
"/var/log/dpkg.log" 2016-4-1
"/var/log/unattended-upgrades/unattended-upgrades.log" 2012-9-20
"/var/log/unattended-upgrades/unattended-upgrades-shutdown.log" 2013-5-17
"/var/log/apache2/mailadmin.pc-freak.net-access.log" 2012-9-19
"/var/log/snort/portscan.log" 2012-9-12
"/var/log/apt/term.log" 2016-4-1
"/var/log/squid/access.log" 2015-3-21
"/var/log/mysql/mysql-slow.log" 2016-4-9
"/var/log/debug" 2016-4-3
"/var/log/mysql.log" 2016-4-9
"/var/log/squid/store.log" 2015-3-21
"/var/log/apache2/mailadmin.pc-freak.net-error.log" 2012-9-19
"/var/log/daemon.log" 2016-4-3
"/var/log/munin/munin-update.log" 2016-4-9
"/var/log/unattended-upgrades/unattended-upgrades*.log" 2013-5-16
"/var/log/razor-agent.log" 2015-2-19
"/var/log/btmp" 2016-4-1
"/var/log/squid/*.log" 2014-11-24
"/var/log/munin/munin-graph.log" 2016-4-9
"/var/log/mysql/mysql.log" 2012-9-12
"/var/log/munin/munin-html.log" 2016-4-9
"/var/log/clamav/freshclam.log" 2016-4-3
"/var/log/munin/munin-node.log" 2016-1-23
"/var/log/mail.info" 2016-4-3
"/var/log/apache2/other_vhosts_access.log" 2016-4-3
"/var/log/exim4/rejectlog" 2012-9-12
"/var/log/squid/cache.log" 2015-3-21
"/var/log/messages" 2016-4-3
"/var/log/stunnel4/stunnel.log" 2012-9-19
"/var/log/apache2/php_error.log" 2012-10-21
"/var/log/ConsoleKit/history" 2016-4-1
"/var/log/rsnapshot.log" 2013-4-15
"/var/log/iptraf/*.log" 2012-9-12
"/var/log/snort/alert" 2012-10-17
"/var/log/privoxy/logfile" 2016-4-3
"/var/log/auth.log" 2016-4-3
"/var/log/postgresql/postgresql-8.4-main.log" 2012-10-21
"/var/log/apt/history.log" 2016-4-1
"/var/log/pm-powersave.log" 2012-11-1
"/var/log/proftpd/proftpd.log" 2016-4-3
"/var/log/proftpd/xferlog" 2016-4-1
"/var/log/zabbix-agent/zabbix_agentd.log" 2016-3-25
"/var/log/alternatives.log" 2016-4-7
"/var/log/mail.log" 2016-4-3
"/var/log/kern.log" 2016-4-3
"/var/log/privoxy/errorfile" 2013-5-28
"/var/log/aptitude" 2015-5-6
"/var/log/apache2/access.log" 2016-4-3
"/var/log/wtmp" 2016-4-1
"/var/log/pm-suspend.log" 2012-9-20
"/var/log/snort/portscan2.log" 2012-9-12
"/var/log/mail.warn" 2016-4-3
"/var/log/bacula/log" 2013-5-1
"/var/log/lpr.log" 2012-12-12
"/var/log/mail.err" 2016-4-3
"/var/log/tor/log" 2016-4-9
"/var/log/fail2ban.log" 2016-4-3
"/var/log/exim4/paniclog" 2012-9-12
"/var/log/tinyproxy/tinyproxy.log" 2015-3-25
"/var/log/munin/munin-limits.log" 2016-4-9
"/var/log/proftpd/controls.log" 2012-9-19
"/var/log/proftpd/xferreport" 2012-9-19
"/var/spool/qscan/qmail-queue.log" 2013-5-15
"/var/log/user.log" 2016-4-3
"/var/log/apache2/error.log" 2016-4-3
"/var/log/exim4/mainlog" 2012-10-16
"/var/log/privoxy/jarfile" 2013-5-28
"/var/log/cron.log" 2016-4-3
"/var/log/clamav/clamav.log" 2016-4-3

 

The timestamp date next to each of the rotated service log is when the respective log was last rorated

It is also a handy thing to rorate only a certain service log, lets say clamav-server, mysql-server, apache2 and nginx
 


logrorate /etc/logrorate.d/clamav-server
logrorate /etc/logrorate.d/mysql-server
logrotate /etc/logrotate.d/nginx

Removing exim and installing qmail / Generate and install pseudo mta dummy package on Debian / Ubuntu etc. .deb based Linux

Thursday, March 10th, 2016

debian-dummy-mta-package-install-howto-tux-mail-nice-mascot
If you happen to be installing Qmail Mail server on a Debian or Ubuntu (.deb) based Linux, you will notice by default there will be some kind of MTA (Mail Transport Agent) already installed mail-transfer-agent package will be installed and because of Debian .deb package depedency to have an MTA always installed on the system you will be unable to remove Exim MTA without installing some other MTA (Postix / Qmail) etc.

This will be a problem for those like me who prefer to compile and install Qmail from source, thus to get around this it is necessery to create a dummy package that will trick the deb packaging depencies that actually mta-local MTA package is present on the server.

The way to go here is to use equivs (Circumvent debian package dependencies):
 

debian:~# apt-cache show equivs|grep -i desc -A 10

Description: Circumvent Debian package dependencies
 This package provides a tool to create trivial Debian packages.
 Typically these packages contain only dependency information, but they
 can also include normal installed files like other packages do.
 .
 One use for this is to create a metapackage: a package whose sole
 purpose is to declare dependencies and conflicts on other packages so
 that these will be automatically installed, upgraded, or removed.
 .
 Another use is to circumvent dependency checking: by letting dpkg
 think a particular package name and version is installed when it

Btw creating a .deb dummy package will be necessery in many other cases when you have to install from some third party debian repositories or some old and alrady unmaintaned deb-src packages for the sake of making some archaic software to resurrect somewhere, so sooner or later even if you're not into Mail servers you will certainly need equivs.

Then install equivs and go on proceeding creating the dummy mail-transport-agent package
 

debian:~# cd /tmp debian:~# cp -rpf /usr/share/doc/equivs/examples/mail-transport-agent.ctl . debian:~# equivs-build mail-transport-agent.ctl


Above command will build and package /tmp/mta-local_1.0_all.deb dummy package.
So continue and install it with dpkg as you use to install debian packages
 

 

debian:~# dpkg -i /tmp/mta-local_1.0_all.deb


From then on you can continue your standard LWQ – Life with Qmail or any other source based qmail installation with:

 

 

./config-fast mail.yourmaildomain.net


So that's it now .deb packaging system consistency will be complete so standard security package updates with apt-get and aptitude updates or dpkg -i third party custom software insatlls will not be breaking up any more.

Hope that helped someone 🙂

 

 

 

 

Archive Outlook mail in Outlook 2010 to free space in your mailbox

Thursday, May 15th, 2014

outlook-archive-old-mail-to-prevent-out-of-space-problems-outlook-logo
If you're working in a middle or big sized IT company or corporation like IBM or HP, you're already sucked into the Outlook "mail whirlwind of corporate world" and daily flooded with tons of corporate spam emails with fuzzy business random terms like taken from Corporate Bullshit Generator

Many corporations, because probably of historic reasons still provide employees with small sized mailboxes half a gigabyte, a gigabyte or even in those with bigger user Mailboxes like in Hewlett Packard, this is usually no more than 2 Gigabytes.

This creates a lot of issues in the long term because usually mail communication in Inbox, Sent Items, Drafts Conversation History, Junk Email and Outbox grows up quickly and for a year or a year and a half, available Mail space fills up and you stop receiving email communication from customers. This is usually not too big problem if your Mailbox gets filled when you're in the Office (in office hours). However it is quite unpleasent and makes very bad impression to customers when you're in a few weeks Summar Holiday with no access to your mailbox and your Mailbox free space  depletes, then you don't get any mail from the customer and all the time the customer starts receiving emails disrupting your personal or company image with bouncing messages saying the "INBOX" is full.

To prevent this worst case scenario it is always a good idea to archive old mail communication (Items) to free up space in Outlook 2010 mailbox.
Old Outlook Archived mail is (Saved) exported in .PST outlook data file format. Later exported Mail Content and Contacts could be easily (attached) from those .pst file to Outlook Express, leaving you possibility to still have access to your old archived mail keeping the content on your hard drive instead on the Outlook Exchange Mailserver (freeing up space from your Inbox).

Here is how to archive your Outlook mail Calendar and contacts:

Archive-outlook-mail-in-microsoft-outlook-2010-free-space-in-your-mailbox

1. Click on the "File" tab on the top horizontal bar.Select "Cleanup Tools" from the options.

2. Click "Cleanup Tools" from the options.

3. Click on the "Archive this folder and all subfolders" option.

4. Select what to archive (e.g. Inbox, Drafts, Sent Items, Calendar whatever …)

5. Choose archive items older than (this is quite self-explanatory)

6. Select the location of your archive file (make sure you palce the .PST file into directory you will not forget later)

That's all now you have old mails freed up from Outlook Exchange server. Now make sure you create regular backups ot old-archived-mail.pst file you just created, it is a very good idea to upload this folder to encrypted file system on USB stick or use something like TrueCrypt to encrypt the file and store it to external hard drive, if you already don't have a complete backup corporate solution backuping up all your Laptop content.

Later Attaching or detaching exported .PST file in Outlook is done from:

File -> Open -> Open Outlook Data File

outlook-open-backupped-pst-datafile-archive-importing-to-outlook-2010


Once .PST file is opened and attached in Left Inbox pane you will have the Archived old mail folder appear.

 

outlook-archived-mail-pannel-screenshot-windows-7
You can change Archived name (like I did to some meaningful name) like I've change it to Archives-2013 by right clicking on it (Data File properties -> Advanced)

Finding spam sending php scripts on multiple sites servers – Tracing and stopping spammer PHP scripts

Monday, April 14th, 2014

stop_php_mail-spam-find-spammer-and-stop-php-spammer-websites
Spam has become a severe issue for administrators, not only for mail server admins but also for webshosting adms. Even the most secure spam protected mail server can get affected by spam due to fact it is configured to relay mail from other servers acting as web hosting sites.

Webhosting companies almost always suffer seriously from spam issues and often their mail servers gets blocked (enter spam blacklists), because of their irresponsible clients uploading lets say old vulnerable Joomla, WordPress without Akismet or proper spam handling plugin,a CMS which is not frequently supported / updated or custom client insecure php code.

What I mean is Shared server A is often configured to sent mail via (mail) server B. And often some of the many websites / scripts hosted on server A gets hacked and a spam form is uploaded and tons of spam start being shipped via mail server B.

Of course on mail server level it is possible to configure delay between mail sent and adopt a couple of policies to reduce spam, but the spam protection issue can't be completely solved thus admin of such server is forced to periodically keep an eye on what mail is sent from hosting server to mail server.
 


If you happen to be one of those Linux (Unix) webhosting admins who find few thousand of spammer emails into mail server logs or your eMail server queue and you can't seem to find what is causing it, cause there are multiple websites shared hosting using mainly PHP + SQL and you can't identify what php script is spamming by reviewing  Apache log / PHP files. What you can do is get use of:

PHP mail.log directive

Precious tool in tracking spam issues is a PHP Mail.log parameter, mail log paramater is available since PHP version >= 5.3.0 and above.
PHP Mail.log parameter records all calls to the PHP mail() function including exact PHP headers, line numbers and path to script initiating mail sent.

Here is how it is used:
 

1. Create empty PHP Mail.log file

touch /var/log/phpmail.log

File has to be writtable to same user with which Apache is running in case of Apache with SuPHP running file has to be writtable by all users.

On Debian, Ubunut Linux:

chown www:data:www-data /var/log/phpmail.log

On CentOS, RHEL, SuSE phpmail.log has to be owned by httpd:

chown httpd:httpd /var/log/phpmail.log

On some other distros it might be chown nobody:nobody etc. depending on the user with which Apache server is running.

 

2. Add to php.ini configuration following lines

mail.add_x_header = On
mail.log = /var/log/phpmail.log

PHP directive instructs PHP to log complete outbund Mail header sent by mail() function, containing the UID of the web server or PHP process and the name of the script that sent the email;
 

(X-PHP-Originating-Script: 33:mailer.php)


i.e. it will make php start logging to phpmail.log stuff like:
 

 

mail() on [/var/www/pomoriemonasteryorg/components/com_xmap/2ktdz2.php:1]: To: info@globalremarketing.com.au — Headers: From: "Priority Mail" <status_93@pomoriemon
astery.org> X-Mailer: MailMagic2.0 Reply-To: "Priority Mail" <status_93@pomoriemonastery.org> Mime-Version: 1.0 Content-Type: multipart/alternative;boundary="——
—-13972215105347E886BADB5"
mail() on [/var/www/pomoriemonasteryorg/components/com_xmap/2ktdz2.php:1]: To: demil7167@yahoo.com — Headers: From: "One Day Shipping" <status_44@pomoriemonastery.
org> X-Mailer: CSMTPConnectionv1.3 Reply-To: "One Day Shipping" <status_44@pomoriemonastery.org> Mime-Version: 1.0 Content-Type: multipart/alternative;boundary="—
——-13972215105347E886BD344"
mail() on [/var/www/pomoriemonasteryorg/components/com_xmap/2ktdz2.php:1]: To: domainmanager@nadenranshepovser.biz — Headers: From: "Logistics Services" <customer.
id86@pomoriemonastery.org> X-Mailer: TheBat!(v3.99.27)UNREG Reply-To: "Logistics Services" <customer.id86@pomoriemonastery.org> Mime-Version: 1.0 Content-Type: mult
ipart/alternative;boundary="———-13972215105347E886BF43E"
mail() on [/var/www/pomoriemonasteryorg/components/com_xmap/2ktdz2.php:1]: To: bluesapphire89@yahoo.com — Headers: From: "Priority Mail" <status_73@pomoriemonaster
y.org> X-Mailer: FastMailer/Webmail(versionSM/1.2.6) Reply-To: "Priority Mail" <status_73@pomoriemonastery.org> Mime-Version: 1.0 Content-Type: multipart/alternativ
e;boundary="———-13972215105347E886C13F2"

 

On Debian / Ubuntu Linux to enable this logging, exec:

echo 'mail.add_x_header = On' >> /etc/php5/apache2/php.ini
echo 'mail.log = /var/log/phpmail.log' >> /etc/php5/apache2/php.ini


I find it useful to symlink /etc/php5/apache2/php.ini to /etc/php.ini its much easier to remember php location plus it is a standard location for many RPM based distros.

ln -sf /etc/php5/apache2/php.ini /etc/php.ini

Or another "Debian recommended way" to enable mail.add_x_header logging on Debian is via:

echo 'mail.add_x_header = On' >> /etc/php5/conf.d/mail.ini
echo 'mail.log = /var/log/phpmail.log' >> /etc/php5/conf.d/mail.ini

On Redhats (RHEL, CentOS, SuSE) Linux issue:

echo 'mail.add_x_header = On' >> /etc/php.ini
echo 'mail.log = /var/log/phpmail.log' >> /etc/php.ini

3. Restart Apache

On Debian / Ubuntu based linuces:

/etc/init.d/apache2 restart

P.S. Normally to restart Apache without interrupting client connections graceful option can be used, i.e. instead of restarting do:

/etc/init.d/apache2 graceful

On RPM baed CentOS, Fedora etc.:

/sbin/service httpd restart

or

apachectl graceful
 

4. Reading the log

To review in real time exact PHP scripts sending tons of spam tail it:

tail -f /var/log/phpmail.log

 

mail() on [/var/www/remote-admin/wp-includes/class-phpmailer.php:489]: To: theosfp813@hotmail.com — Headers: Date: Mon, 14 Apr 2014 03:27:23 +0000 Return-Path: wordpress@remotesystemadministration.com From: WordPress Message-ID: X-Priority: 3 X-Mailer: PHPMailer (phpmailer.sourceforge.net) [version 2.0.4] MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="UTF-8"
mail() on [/var/www/pomoriemonasteryorg/media/rsinstall_4de38d919da01/admin/js/tiny_mce/plugins/inlinepopups/skins/.3a1a1c.php:1]: To: 2070ccrabb@kiakom.net — Headers: From: "Manager Elijah Castillo" <elijah_castillo32@pomoriemonastery.org> X-Mailer: Mozilla/5.0 (Windows; U; Windows NT 5.0; es-ES; rv:1.9.1.7) Gecko/20100111 Thunderbird/3.0.1 Reply-To: "Manager Elijah Castillo" <elijah_castillo32@pomoriemonastery.org> Mime-Version: 1.0 Content-Type: multipart/alternative;boundary="———-1397463670534B9A76017CC"
mail() on [/var/www/pomoriemonasteryorg/media/rsinstall_4de38d919da01/admin/js/tiny_mce/plugins/inlinepopups/skins/.3a1a1c.php:1]: To: 20wmwebinfo@schools.bedfordshire.gov.uk — Headers: From: "Manager Justin Murphy" <justin_murphy16@pomoriemonastery.org> X-Mailer: Opera Mail/10.62 (Win32) Reply-To: "Manager Justin Murphy" <justin_murphy16@pomoriemonastery.org> Mime-Version: 1.0 Content-Type: multipart/alternative;boundary="———-1397463670534B9A7603ED6"
mail() on [/var/www/pomoriemonasteryorg/media/rsinstall_4de38d919da01/admin/js/tiny_mce/plugins/inlinepopups/skins/.3a1a1c.php:1]: To: tynyrilak@yahoo.com — Headers: From: "Manager Elijah Castillo" <elijah_castillo83@pomoriemonastery.org> X-Mailer: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; pl; rv:1.9.1.9) Gecko/20100317 Thunderbird/3.0.4 Reply-To: "Manager Elijah Castillo" <elijah_castillo83@pomoriemonastery.org> Mime-Version: 1.0 Content-Type: multipart/alternative;boundary="———-1397463670534B9A7606308"
mail() on [/var/www/pomoriemonasteryorg/media/rsinstall_4de38d919da01/admin/js/tiny_mce/plugins/inlinepopups/skins/.3a1a1c.php:1]: To: 2112macdo1@armymail.mod.uk — Headers: From: "Manager Justin Murphy" <justin_murphy41@pomoriemonastery.org> X-Mailer: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; pl; rv:1.9.1.9) Gecko/20100317 Thunderbird/3.0.4 Reply-To: "Manager Justin Murphy" <justin_murphy41@pomoriemonastery.org> Mime-Version: 1.0 Content-Type: multipart/alternative;boundary="———-1397463670534B9A76086D1"

 

As you can see there is a junky spam mails sent via some spammer script uploaded under name .3a1a1c.php, so to stop the dirty bastard, deleted the script:

rm -f /var/www/pomoriemonasteryorg/media/rsinstall_4de38d919da01/admin/js/tiny_mce/plugins/inlinepopups/skins/.3a1a1c.php

It is generally useful to also check (search) for all hidden .php files inside directoring storing multiple virtualhost websites, as often a weirdly named hidden .php is sure indicator of either a PHP Shell script kiddie tool or a spammer form.

Here is how to Find all Hidden Perl / PHP scripts inside /var/www:

find . -iname '.*.php'
./blog/wp-content/plugins/fckeditor-for-wordpress-plugin/ckeditor/plugins/selection/.0b1910.php
./blog/wp-content/plugins/fckeditor-for-wordpress-plugin/filemanager/browser/default/.497a0c.php
./blog/wp-content/plugins/__MACOSX/feedburner_feedsmith_plugin_2.3/._FeedBurner_FeedSmith_Plugin.php

find . -iname '.*.pl*'

….

Reviewing complete list of all hidden files is also often useful to determine shitty cracker stuff

 find . -iname ".*"

Debugging via  /var/log/phpmail.log enablement is useful but is more recommended on development and staging (QA) environments. Having it enable on productive server with high amounts of mail sent via PHP scripts or just on dedicated shared site server could cause both performance issues, hard disk could quickly get and most importantly could be a severe security hole as information from PHP scripts could be potentially exposed to external parties.

Outlook Express hotkeys (short keys) list – Use your MS Outlook efficiently

Thursday, March 13th, 2014

I'm foced to use Microsoft Outlook Express daily for my daily work in Hewlett Packard. Thus I decided to learn the hotkeys to make my Mail use more efficient. Here are list of Outlook Express hotkeys, hopefully to be useful to others too:
 

Print the selected message –  Ctrl+P
Send and receive mail –  Ctrl+M
Delete a mail message - DEL or Ctrl+D
Open or post a new message –  Ctrl+N
Open the Address Book - Ctrl+SHIFT+B
Reply to the message author –  Ctrl+R
Reply to all –  Ctrl+Shitf+R, Ctrl+G (news only)
Refresh news message and headers –  F5
Forward a message - Ctrl+F
Find Text - F3
Find a message - CTRL+Shift+F
Close Message - ESC
Check Names - Ctrl+K, Alt+S
Go to your Inbox - Ctrl+I
Go to the next message in the list - Ctrl+> or Ctrl+SHIFT+>
Go to the previous message in the list - Ctrl+< or Ctrl+SHIFT+<
View properties of a selected message - ALT+ENTER
View the full source of the message - Ctrl+F3
Go to the next unread mail message - Ctrl+U
Go to the next unread news conversation - Ctrl+Shift+U
Go to a folder –  Ctrl+Y
Open a selected message – Ctrl+O or ENTER
Mark a message as read - Ctrl+ENTER or Ctrl+Q
Move between the Folders list (if on), message list, preview pane, and Contacts list (if on) - TAB
Mark all news messages as read - Ctrl+SHIFT+A
Check spelling – F7
Insert signature - Ctrl+SHIFT+S
Send (post) a message –  Ctrl+ENTER or ALT+S
Select all messages –  Ctrl+A

Enjoy 🙂

 

Quitting my job as IT Manager and moving to Further Horizons in Hewlett Packard

Friday, September 13th, 2013

International University College Logo IUC

I haven't blogged for a while for a plenty of reasons, I'm going through a change period in my life and as any change it is not easy.
This post will be not informative and will not teach any of my dear readers, anything on Computers its pretty personal but still for my friends it might cause interest.
Here is my personal life story over the last few months …

For a while I worked in a International University College situated in my hometown Dobrich. I was hired on position of IT Manager, and actually was doing a bit of E-Marketing to try to boost traffic to College's website – www.vumk.eu and mostly helping the old school hacker ad college system administrator over the last 10 yrs – Ertan to fix a bunch of Linux Mail / SQL and Webservers and some Windows machines. In college I learned from Ertan how to install and backups of restaurants software called BARBEQUE as well as how to fix problems with billing terminals situated in College Restaurant (3rd floor of building). Other of my work time I had to  fix infested Windows computers with viruses re-install Windowses and fix various printing and network problems of College's teachers, accountants, cash desk, marketing and rest of college  employees.

Talking about Ertan I should express my sincere tremendous Thanks (Thanks Ertan) to it for recommending me for this job position. Right before I started work in the college I was jobless for a while starting to get desperate that its impossible find work. Current IUC sysadmin – Mr. Ertan Geldiev is a remarkable man and one of the people that made great impression in my mind. Something I found interesting I can learn from Ertan was to get from his cheerful "admin" attitude. As a true hacker Ertan had this hacker attitude of playfulness I myself has for a while lost over the years. So seeing someone like this near my life make me a good favor and had a positive influence on myself.

I have learned a lot from Ertan during the 3 months and 3 days in International University College. Just for a bit of historic information earlier IUC was known as International College – Albena, also among Dobrich citizens known as The Dutch College – as earlier IUC had good relations with Dutch Universities and was issuing double degrees both Bulgarian and Dutch. Nowadays I'm not sure if still Double-degrees partnership between IUC and Dutch Universities exist, what I know for sure is college is issuing European Double Degrees in partnership with Cardiff Metropolitan University. I myself have earlier studied in the college and already know the place well thus will use this post to say a few words on my impressions on it …

International University College - one of top prestigious colleges in Eastern Europe

The college is a great place to be as you have chance to meet plenty of people both lecturers (professors), participate in the various events organized by College's as well as get involved in the many European Projects which are being handled by a European Projects department special department situated on the back of the College Building. Other positive about College is it is small and located on a peaceful town of Dobrich. This gives the bright people a lot of space for personal development, anyways on the other hand it can make you also a bad as Dobrich as a small city is a bit boring. The studies in College are good for students who want personal freedom as there is not too strict requirements for professors on how to teach.

Though college had help me grow, especially in my knowledge in Windows 7 and 8 (Ertan had a really good Windows background), I couldn't have the chance to develop myself too further in the long term. So my job offer to work in HP as Web and Middleware Implementation Engineer opened much broader opportunities for my long term IT career. Other reason I quit the College IT job was simply because I needed more money I had the vision to make a family with a girl from Belarus – Svetlana and in order to take care for her I need to earn good money. My official salary in the college was the funny for the position – 640 lv (though after a few months I was promised to have a raise and earn 400 EUR :)) . Such low sallary was for the reason I had the idea to continue studying in College and complete my Bachelor Business degree and we had agreed with the College CEO Mr. Todor Radev to extract part of my salary monthly and with that to pay my 1 semester tuition fee (2200 EUR) – necessary for my graduation assignment. Though completing the Bachelor is important phase to close in my life for a long time, I found for the moment more valuable to work for HP and earn normal living salary with which to possibly finance myself and create family with woman of my life (hopefully) in the short term.

In this post I want express my sincere thanks to all people in International College (Elena Urchenko, Krasimir – for helping me in my job duties), Pavel and Silvia for being a colleague for a while I worked partly in the Marketing Department.

Talking about Marketing Department what I did there is some Twitter Marketing (building some twitter followers) and wrote a tiny document with recommendation on how to optimize College website – vumk.eu (future version) – for better SEO ranking. This included complete analysis from user outlook to Indexing bots and site current code. 

Mr Docent Phd Todor Radev

I have to do a big underline on how great person the College President and UNI Rector – Docent Todor Radev is. I have already bitter experience studying for a while in a government universities when younger and I know from experience usually Rectors and Universities management of state universities is pure "Hell". Thanks to Mr. Todor Radev for he did me a big favor letting me quit  job just a week later (instead of 1 month as it is officially set by Bulgarian Dismissal Law and explicitly stated in my Work Contract. Also as a person my experience from Docent Radev is wonderful too. He is extremely intelligent, brilliant gentleman and  most importantly open minded and always open for innovation.
 

As a close up I would like to say Big Thanks to everyone which I worked with or met in International University College! Thanks guys for all your support and help, thanks for being work mates and friends for the time.

10 must know and extremely useful Linux commands that every sys admin should know

Tuesday, July 30th, 2013

10 must know extremely useful gnu linux command line tools tips and tricks
There are plenty of precious command line stuff every admin should be aware on Linux. In this article I just decided to place some I use often and are interesting to know. Below commands are nothing special and probably many of experienced sys admins already know them. However I'm pretty sure novice admins and start-up Linux enthusiasts will find it useful. I know there much more to be said on the topic. So anyone is mostly welcome to share his used cmds.
 
1. Delete all files in directory except files with certain file extension

It is good trick to delete all files in directory except certain file formats, to do so:

root@linux:~# rm !(*.c|*.py|*.txt|*.mp3)

2. Write command output to multiple files (tee)

The normal way to write to file is by using redirect (to overwrite file) ">" or (to append to file) ">>";. However when you need to write output to multiple files there is a command called tee, i.e.:

root@linux:~# ps axuwwf | tee file1 file2 file3

3. Search for text in plain text file printing number of lines after match

Whether you need to print all number of lines after match of "search_text" use:

root@linux:~# grep -A 5 -i "search_text" text_file.txt

4. Show all files where text string is matched with GREP (Search for text recursively)

Searching for text match is extremely helpful for system administration. I use  grep recursive (capability) almost on daily basis:

root@websrv:/etc/dovecot# grep -rli text *
conf.d/10-auth.conf
conf.d/10-mail.conf
dovecot.conf

-l (instructs to only print file names matching string), -r (stands for recursive search), and -i flag (instructs grep to print all matches  inogoring case-sensitivity ( look for text nomatter if with capital or small letters)

5. Finding files and running command on each file type matched

In Linux with find command it is possible to search for files and run command on each file matched.
Lets say you we want to look in current directory for all files .swp (temporary) files produced so often by VIM and wipe them out:

root@linux:~# find . -iname '*.swp*' -exec rm -f {} \;

6. Convert DOS end of file (EOF) to UNIX with sed

If it happens you not have dos2unix command installed on Linux shell and you need to translate DOS end of file (\r\n – return carriage, new line) to UNIX's (\r – return carriage)), do it with sed:

root@linux:~# sed 's/.$//' filename

7. Remove file duplicate lines with awk:

cat test.txt
test
test
test duplicate
The brown fox jump over ...
Richard Stallman rox

root@linux:~# awk '!($0 in array) { array[$0]; print }' test.txt
test
test duplicate
The brown fox jump over ...
Richard Stallman rox

To remove duplicate text from all files in directory same can be easily scripped with bash for loop:

root@linux:~# for i in *; do
awk '!($0 in array) { array[$0]; print }' $i;
done

8. Print only selected columns from text file

To print text only in 1st and 7th column in plain text file with awk:

root@linux:~# awk '{print $1,$6;}' filename.txt ...

To print only all existing users on Linux with their respective set shell type:

root@linux:~# cat /etc/passwd|sed -e 's#:# #g'|awk '{print $1,$6;}'

9. Open file with VIM text editor starting from line

I use only vim for console text processing, and I often had to edit and fix file which fail to compile on certain line number. Thus use vim to open file for writing from necessary line num. To open file and set cursor to line 35 root@linux:~# vim +35 /home/hipo/current.c

10. Run last command with "!!" bash shorcut

Lets say last command you run is uname -a:

root@websrv:/home/student# uname -a
Linux websrv 3.2.0-4-686-pae #1 SMP Debian 3.2.46-1 i686 GNU/Linux

To re-run it simply type "!!":

root@websrv:/home/student# !!
uname -a
Linux websrv 3.2.0-4-686-pae #1 SMP Debian 3.2.46-1 i686 GNU/Linux

root@websrv:/home/student#

 

FreeBSD Happy Birthday – FreeBSD OS turns 20 years old

Monday, June 24th, 2013

Happy Birrthday Freebsd / FreeBSD becomes 20 years old - Classic logo bsd avatar beastie mascot

On 19th of June FreeBSD had its 20 birthday! Its great joy for us the FreeBSD geeks 🙂
This is a bit outdated but as I'm a long years FreeBSD user I found myself obliged to mention the good news.

FreeBSD project was officially introduced for first time under the new name by David Greenman, Jordan Hubbard and Rod Grimes as fork of BSD (Berkley Software Distrbution 4.3).

Even more and less known is that big chunks of FreeBSD (kernel and userland) code are running on the shiny Mac OS X.

Below is mail which made up the name FreeBSD 🙂
 

To: interim@bsd.coe.montana.edu (Interim 0.1.5)
Subject: Re: “386BSD” trademark (fwd)
From: David Greenman <davidg@implode.rain.com>
Date: Sat, 19 Jun 93 17:26:02 -0700

> Okay folks.. taking new name suggestions.. we have:
>
> BSDFree86 – Rod, who is going with Jordans improved NON BSDI name..
> Free86BSD – Jordan, Rod likes this one two…
> – (F86BSD for short)
>
> vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
>                v                                                                     v
>                v          This is the hat to drop yours in!       v
>                v                                                                     v
>                vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv

How about just simply “FreeBSD”? No confusion, no fuss, seems like a good compromise to me. 🙂

-DG

FreeBSD Berkley Software Distrubution OS new logo happy birthday FreeBSD

Now 20 years after, FreeBSD itself had numerous forks very much in tradition of Free Software – most notable ones are of course BSD for desktop DragonFly BSD and PC-BSD.
As part of FreeBSD evolution nowadays FreeBSD kernel is ported for Debian Linux, i.e.:

debian:~# apt-cache show kfreebsd-source-8.1|grep -i description -A 2

Description: source code for kernel of FreeBSD 8.1 with Debian patches

 This package provides the source code for kernel of FreeBSD 8.1, base of

 a GNU/kFreeBSD system.

Description: source code for kernel of FreeBSD 8.1 with Debian patches

 This package provides the source code for kernel of FreeBSD 8.1, base of

 a GNU/kFreeBSD system.

 

 and can be run onas substitute for Linux kernel on most GNU / Linux distributions 
 

FreeBSD has unquestionably played key role over the last 20 yrs for development of Hacker Culture of Freedom and Free Software.

 Congrats and cheers to all FreeBSD users! 🙂  

Linux: Fixing Qmail server qmail-smtpd port 25 slow (lagged) connect problem

Thursday, May 16th, 2013

qmail logo fixing qmail mail SMTP port 25 connect delays

After updating my Debian Squeeze to latest stable packages from repository with standard:
# apt-get update && apt-get upgrade

I routinely checked, if afterwards all is fine with Qmail?, just to find out connect to port 25 was hell delayed about 40-50 seconds before qmail responds with standard assigned Mail Greeting.
I Googled long time to see if I can find a post or forum thread discussing, exact issue, but though I found similar discussions I didn't found anything that exactly match problem. Thus I decided to follow the good old experimental try / fail method to figure out what causes it.

elow is pastes from telnet, illustrating delays in Qmail SMTP greeting respond:

# telnet localhost 25
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.

# telnet localhost 25
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.

# telnet localhost 25
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.

I spend about 2 hours, checking Qmail for the standard so common errors, usually causing it to not work properly following my previous article testing qmail installation problems

After going, through all of possible causes the only clue for problems, were some slowness with spamassassin. This brought me the idea that something is done wrong with spamassassin .I tried disabling, Spamassassin Razon and Pyzor restarting spamd through (in my case done not via the standard start/stop debian script) but through daemontools with svc and qmailctl i.e.:

# svc -d /service/spamd
# svc -u /service/spamd
# svc -a /service/spamd

qmailctl restart
* Stopping qmail-smtpdssl.
* Stopping qmail-smtpd.
* Sending qmail-send SIGTERM and restarting.
* Restarting qmail-smtpd.
* Restarting qmail-smtpdssl.
* Restarting qmail-pop3d.
This doesn't help, so I continued trying to figure out, what is wrong .One assumption for slow  qmail-smtpd responce was of course slow DNS resolve issues. I checked /etc/resolv.conf to find out server is configured to use local  configured DJBDNS server as first line DNS resolver. I used djbdns for it is simple and easy to configure, however it is a bit obsolete so it was possible bottleneck. After commenting line to use localhost 127.0.0.1
and settings as primary DNS Google Public DNS 8.8.8.8, problem persisted so problems with hosts resolving was obviously not the problem.

I pondered for about 30 minutes, checking again all logs and checking machine processes. Just to remember before I experienced similar issues caused by unresolving RBL (blacklist IP) hosts. I checked configured SPF records in
(process list) and noticed following 4 hosts;

# ps auxwwf

7190 ?        S      0:00 tcpserver -vR -l /var/qmail/control/me -c 30 -u 89 -g 89 -x /etc/tcp.smtp.cdb 0 25 rblsmtpd -t0 -r zen.spamhaus.org -r dnsbl.njabl.org -r dnsbl.sorbs.net -r bl.spamcop.net qmail-smtpd /var/qmail/control/me /home/vpopmail/bin/vchkpw /bin/true
 

I checked one by one hosts and find out 1st two hosts in line are no longer resolving (blacklist is no longer accessible) as before:

 

zen.spamhaus.org, dnsbl.njabl.org

DNSBL (DNS blocklist) is configured on this host via /service/qmail-smtpd/run, hence to remove two unresolvable hosts forcing the weird qmail-smtpd connect delay I had to modify in it:

RBL_BAD="zen.spamhaus.org dnsbl.njabl.org dnsbl.sorbs.net bl.spamcop.net"

to

RBL_BAD="dnsbl.sorbs.net bl.spamcop.net"

After a close examinations in mail server config /var/qmail/control/spfrules, found one other Unresolvable SPF Blacklist host configured ;
# cat /var/qmail/control/spfrules
include:spf.trusted-forwarder.org

To move that one I null-ed file:

# cat /dev/null > /var/qmail/control/spfrules

Finally to take affect all changes, launched Qmail start:

# qmailctl restart
Restarting qmail:
* Stopping qmail-smtpdssl.
* Stopping qmail-smtpd.
* Sending qmail-send SIGTERM and restarting.
* Restarting qmail-smtpd.
* Restarting qmail-smtpdssl.
* Restarting qmail-pop3d.

To check all was fine afterwards, again used telnet:

# telnet localhost 25
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
220 This is Mail Pc-Freak.NET ESMTP

Mail greeting now appears in about 2-3 seconds time.