Archive for the ‘Web and CMS’ Category

How to calculate connections from IP address with shell script and log to Zabbix graphic

Thursday, March 11th, 2021

We had to test the number of connections incoming IP sorted by its TCP / IP connection state.

For example:

TIME_WAIT, ESTABLISHED, LISTEN etc.


The reason behind is sometimes the IP address '192.168.0.1' does create more than 200 connections, a Cisco firewall gets triggered and the connection for that IP is filtered out. To be able to know in advance that this problem is upcoming. a Small userparameter script is set on the Linux servers, that does print out all connections from IP by its STATES sorted out.

 

The script is calc_total_ip_match_zabbix.sh is below:

#!/bin/bash
#  check ESTIMATED / FIN_WAIT etc. netstat output for IPs and calculate total
# UserParameter=count.connections,(/usr/local/bin/calc_total_ip_match_zabbix.sh)
CHECK_IP='192.168.0.1';
f=0; 

 

for i in $(netstat -nat | grep "$CHECK_IP" | awk '{print $6}' | sort | uniq -c | sort -n); do

echo -n "$i ";
f=$((f+i));
done;
echo
echo "Total: $f"

 

root@pcfreak:/bashscripts# ./calc_total_ip_match_zabbix.sh 
1 TIME_WAIT 2 ESTABLISHED 3 LISTEN 

Total: 6

 

root@pcfreak:/bashscripts# ./calc_total_ip_match_zabbix.sh 
2 ESTABLISHED 3 LISTEN 
Total: 5


images/zabbix-webgui-connection-check1

To make process with Zabbix it is necessery to have an Item created and a Depedent Item.

 

webguiconnection-check1

webguiconnection-check1
 

webgui-connection-check2-item

images/webguiconnection-check1

Finally create a trigger to trigger alarm if you have more than or eqaul to 100 Total overall connections.


images/zabbix-webgui-connection-check-trigger

The Zabbix userparameter script should be as this:

[root@host: ~]# cat /etc/zabbix/zabbix_agentd.d/userparameter_webgui_conn.conf
UserParameter=count.connections,(/usr/local/bin/webgui_conn_track.sh)

 

Some collleagues suggested more efficient shell script solution for suming the overall number of connections, below is less time consuming version of script, that can be used for the calculation.
 

#!/bin/bash -x
# show FIN_WAIT2 / ESTIMATED etc. and calcuate total
count=$(netstat -n | grep "192.168.0.1" | awk ' { print $6 } ' | sort -n | uniq -c | sort -nr)
total=$((${count// /+}))
echo "$count"
echo "Total:" "$total"

      2 ESTABLISHED
      1 TIME_WAIT
Total: 3

 


Below is the graph built with Zabbix showing all the fluctuations from connections from monitored IP. ebgui-check_ip_graph

 

How to calculate connections from IP address with shell script and log to Zabbix graphic

Thursday, March 11th, 2021

We had to test the number of connections incoming IP sorted by its TCP / IP connection state.

For example:

TIME_WAIT, ESTABLISHED, LISTEN etc.


The reason behind is sometimes the IP address '192.168.0.1' does create more than 200 connections, a Cisco firewall gets triggered and the connection for that IP is filtered out. To be able to know in advance that this problem is upcoming. a Small userparameter script is set on the Linux servers, that does print out all connections from IP by its STATES sorted out.

 

The script is calc_total_ip_match_zabbix.sh is below:

#!/bin/bash
#  check ESTIMATED / FIN_WAIT etc. netstat output for IPs and calculate total
# UserParameter=count.connections,(/usr/local/bin/calc_total_ip_match_zabbix.sh)
CHECK_IP='192.168.0.1';
f=0; 

 

for i in $(netstat -nat | grep "$CHECK_IP" | awk '{print $6}' | sort | uniq -c | sort -n); do

echo -n "$i ";
f=$((f+i));
done;
echo
echo "Total: $f"

 

root@pcfreak:/bashscripts# ./calc_total_ip_match_zabbix.sh 
1 TIME_WAIT 2 ESTABLISHED 3 LISTEN 

Total: 6

 

root@pcfreak:/bashscripts# ./calc_total_ip_match_zabbix.sh 
2 ESTABLISHED 3 LISTEN 
Total: 5


To make process with Zabbix it is necessery to have an Item created and a Depedent Item.

images/zabbix-webgui-connection-check1

 

 

 

 

webguiconnection-check1

webguiconnection-check1
 

webgui-connection-check2-item

images/webguiconnection-check1

Finally create a trigger to trigger alarm if you have more than or eqaul to 100 Total overall connections.


images/zabbix-webgui-connection-check-trigger

The Zabbix userparameter script should be as this:
cat /etc/zabbix/zabbix_agentd.d/userparameter_webgui_conn.conf
UserParameter=count.connections,(/usr/local/bin/webgui_conn_track.sh)
 

Some collleagues suggested more efficient shell script solution for suming the overall number of connections, below is less time consuming version of script, that can be used for the calculation.
 

#!/bin/bash -x
# show FIN_WAIT2 / ESTIMATED etc. and calcuate total
count=$(netstat -n | grep "192.168.0.1" | awk ' { print $6 } ' | sort -n | uniq -c | sort -nr)
total=$((${count// /+}))
echo "$count"
echo "Total:" "$total"

 

      2 ESTABLISHED
      1 TIME_WAIT
Total: 3

 


Below is the graph built with Zabbix showing all the fluctuations from connections from monitored IP.
ebgui-check_ip_graph

Create simple proxy http server with netcat ( nc ) based tiny shell script

Tuesday, January 26th, 2021

use-Netcat_proxy-picture

The need of proxy server is inevitable nowadays especially if you have servers located in a paranoid security environments. Where virtually all is being passed through some kind of a proxy server. In my work we have recently started a  CentOS Linux release 7.9.2009 on HP Proliant DL360e Gen8 (host named rhel-testing).

HP DL360e are quite old nowadays but since we have spare servers and we can refurnish them to use as a local testing internal server Hypervisor it is okay for us. The machine is attached to a Rack that is connected to a Secured Deimilitarized Zone LAN (DMZ Network) which is so much filtered that even simple access to the local company homebrew RPM repository is not accessible from the machine.
Thus to set and remove software from the machine we needed a way to make yum repositories be available, and it seems the only way was to use a proxy server (situated on another accessible server which we use as a jump host to access the testing machine).

Since opening additional firewall request was a time consuming non-sense and the machine is just for testing purposes, we had to come with a solution where we can somehow access a Local repository RPM storage server http://rpm-package-server-repo.com/ for which we have a separate /etc/yum.repos.d/custom-rpms.repo definition file created.

This is why we needed a simplistic way to run a proxy but as we did not have the easy way to install privoxy / squid / haproxy or apache webserver configured as a proxy (to install one of those of relatively giant piece of software need to copy many rpm packages and manually satisfy dependencies), we looked for a simplistic way to run a proxy server on jump-host machine host A.

A note to make here is jump-host that was about to serve as a proxy  had already HTTP access towards the RPM repositories http://rpm-package-server-repo.com and could normally fetch packages with curl or wget via it …

For to create a simple proxy server out of nothing, I've googled a bit thinking that it should be possible either with BASH's TCP/IP capabilities or some other small C written tool compiled as a static binary, just to find out that netcat swiss army knife as a proxy server bash script is capable of doing the trick.

Jump host machine which was about to be used as a proxy server for http traffic did not have enabled access to tcp/port 8888 (port's firewall policies were prohibiting access to it).Since 8888 was the port targetted to run the proxy to allow TCP/IP port 8888 accessibility from the testing RHEL machine towards jump host, we had to issue first on jump host:

[root@jump-host: ~ ]# firewall-cmd –permanent –zone=public –add-port=8888/tcp

To run the script once placed under /root/tcp-proxy.sh on jump-host we had to run a never ending loop in a GNU screen session to make sure it runs forever:

Original tcp-proxy.sh script used taken from above article is:
 

#!/bin/sh -e

 

if [ $# != 3 ]
then
    echo "usage: $0 <src-port> <dst-host> <dst-port>"
    exit 0
fi

TMP=`mktemp -d`
BACK=$TMP/pipe.back
SENT=$TMP/pipe.sent
RCVD=$TMP/pipe.rcvd
trap 'rm -rf "$TMP"' EXIT
mkfifo -m 0600 "$BACK" "$SENT" "$RCVD"
sed 's/^/ => /' <"$SENT" &
sed 's/^/<=  /' <"$RCVD" &
nc -l -p "$1" <"$BACK" | tee "$SENT" | nc "$2" "$3" | tee "$RCVD" >"$BACK"

 

Above tcp-proxy.sh script you can download here.

I've tested the script one time and it worked, the script syntax is:

 [root@jump-host: ~ ]#  sh tcp-proxy.sh
usage: tcp-proxy.sh <src-port> <dst-host> <dst-port>


To make it work for one time connection I've run it as so:

 

 [root@jump-host: ~ ]# sh tcp-proxy.sh 8888 rpm-package-server-repo.com 80

 

 

To make the script work all the time I had to use one small one liner infinite bash loop which goes like this:

[root@jump-host: ~ ]#  while [ 1 ]; do sh tcp-proxy.sh 8888 rpm-package-server-repo.com 80; done​

On rhel-testing we had to configure for yum and all applications to use a proxy temporary via
 

[root@rhel-tresting: ~ ]# export http_proxy=jump-host_machine_accessibleIP:8888


And then use the normal yum check-update && yum update to apply to rhel-testing machine latest RPM package security updates.

The nice stuff about the tcp-proxy.sh with netcat in a inifite loop is you will see the binary copy of traffic flowing on the script which will make you feel like in those notorious Hackers movies ! 🙂

The stupid stuff is that sometimes some connections and RPM database updates or RPMs could be cancelled due to some kind of network issues.

To make the connection issues that are occuring to the improvised proxy server go away we finally used a slightly modified version from the original netcat script, which read like this.
 

#!/bin/sh -e

 

if [ $# != 3 ]
then
    echo "usage: $0 <src-port> <dst-host> <dst-port>"
        exit 0
        fi

        TMP=`mktemp -d`
        BACK=$TMP/pipe.back
        SENT=$TMP/pipe.sent
        RCVD=$TMP/pipe.rcvd
        trap 'rm -rf "$TMP"' EXIT
        mkfifo -m 0600 "$BACK" "$SENT" "$RCVD"
        sed 's/^/ => /' <"$SENT" &
        sed 's/^/<=  /' <"$RCVD" &
        nc –proxy-type http -l -p "$1" <"$BACK" | tee "$SENT" | nc "$2" "$3" | tee "$RCVD" >"$BACK"


Modified version tcp-proxy1.sh with –proxy-type http argument passed to netcat script u can download here.

With –proxy-type http yum check-update works normal just like with any normal fully functional http_proxy configured.

Next step wasto make the configuration permanent you can either add to /root/.bashrc or /etc/bashrc (if you need the setting to be system wide for every user that logged in to Linux system).

[root@rhel-tresting: ~ ]#  echo "http_proxy=http://jump-host_machine_accessibleIP:8888/" > /etc/environment


If you need to set the new built netcat TCP proxy only for yum package update tool include proxy only in /etc/yum.conf:

[root@rhel-tresting: ~ ]# vi /etc/yum.conf
proxy=http_proxy=http://jump-host_machine_accessibleIP:8888/


That's all now you have a proxy out of nothing with just a simple netcat enjoy.

Short SSL generate new and self-signed certificates PEM, view and convert to and from PKCS12 to java key store cookbook commands cheat sheet

Tuesday, January 12th, 2021

OpenSSL-logo

Below is a short compilation of common used openssl commands (a kind of cookbook) helpful for sysadmins who has to commonly deal with OpenSSL certificates.

Lets say you have to generate new certificate / key and a PEM files, prepare self-signed certificates, show CSR / PEM or KEY ssl file contents, get information about certificate such as expiry date a type of encryption algorithm or sign certificate with self-signed authority convert PEM to PKCS12, convert from PKCS12 file format to .PEM, convert java X509 to java key store SSL encryptionor convert java key store format certificate to PKCS12, then below will be of use to you.

1. Generate Private RSA Key with 2048 bits

# openssl genrsa -out $ (hostname -f) .key 2048

2. Create CSR file

# openssl req -new -key $ (hostname -f) .key -out $ (hostname -f) .csr

3. Create a Self Certified Certificate:

# openssl x509 -req -days 30 -in $ (hostname -f) .csr -signkey $ (hostname -f) .key -out $ (hostname -f) .crt
Enter password:

# openssl rsa -in key.pem -out newkey.pem


4. Show CSR file content

# openssl req -in newcsr.csr -noout -text


5. Get Certificate version / serial number / signature algorithm / RSA key lenght / modulus / exponent etc.

# openssl x509 -in newcert.pem -noout -text


6. Server certificate as CA self signeded

# openssl ca -in newcert.csr -notext -out newcert.pem


7. Generate a certificate signing request based on an existing certificate

# openssl x509 -x509toreq -in certificate.crt -out CSR.csr -signkey privateKey.key


8. Convert .pem / .key / .crt file format to pkcs12 format
 

# openssl pkcs12 -export -in newcert.pem -inkey newkey.key -certfile ca.crt -out newcert.p12


9. Convert pkcs12 pfx to common .pem

# openssl pkcs12 -in mycert.pfx -out mycert.pem


10. The Formats available

# openssl x509 -inform the -in certificate.cer -out certificate.crt


11. Convert a pkcs # 7 certificate into PEM format

# openssl pkcs7 -in cert.p7c -inform DER -outform PEM -out certificate.p7b
# openssl pkcs7 -print_certs -in certificate.p7b -out certificate.pem


12. Convert X509 to java keystore file

# java -cp not-yet-commons-ssl-0.3.11.jar org.apache.commons.ssl.KeyStoreBuilder pass_for_new_keystore key.key certificate.crt

13. Convert java keystore file to pkcs12

# keytool -importkeystore -srckeystore keystore.jks -destkeystore intermediate.p12 -deststoretype PKCS12

SEO: Best day and time to write new articles and tweet to get more blog reads – Social Network Timing

Tuesday, June 17th, 2014

what-is-best-time-to-write-articles-to-increase-your-blog-traffic

I'm trying to regularly blog – as this gives me a roadmap what I'm into and how I spent my time. When have free time,  I blog almost daily except on weekends (as in weekends I'm trying to stay away from computers). So if you want to attract more readers to your blog the interesting question arises
 

What time is best to hit publish on your posts?

writing-in-the-mogning-on-the-internet-timing-morning-is-best-for-your-posts
Now there are different angles from where you can extract conclusions on best timing to blog post.One major thing to consider always when posting is that highest percentage of users read blogs in the morning with their morning coffee. Here are some more facts on when web content is more red:

  • 70% of users say they read blogs in the morning
  • More men read blogs at night than woman
  • Mondays are the highest traffic days for avarage blogs
  • 11 a.m. is normally the highest traffic hour for blogs
  • Usually most comments are put on Saturdays
  • Blogs with more than one post a day has higher chance of inbound links and usually get more unique visitors

As my blog is more technical oriented most of my visitors are men and therefore posting my blogs at night doesn't interfere much with my readers.
However, I've noticed that for me personally posting in time interval from 13:00 to 17:00 influence positively the amount of unique visitors the blog gets.

According to research done by Social Fresh – Thursday is the best day to publish an article if you want to get more Social SharesBest-Day-to-Blog-to-get-more-shares-in-social-networks

As a rule of thumb Thursday wins 10% more shares than all other days. In fact, 31% of the top 100 social share days in 2011 fell on Thursday.
My logical explanation on this phenomenon is that people tend to be more and more bored from their work and try to entertain more and more as the week progresses.

To get more attention on what I'm writting I use a bit of social networking but I prefer using only a micro blogging social networking.  I use Twitter to share what I'm into. When I write a new article on my blog I tweet its title with a link to my article, because this drives people attention to what I have to say.

In overall I am skeptical about social siting like Facebook and MySpace because it has negative impact on how people use their time and especially negative on youngsters Other reason why I don't like Friends Networks is because sharing what you have to say on sites like FB, Google+ or "The Russian Facebook" –  Vkontekte VK.com are not respecting privacy of your data.

 

You write free fresh content for their website for free and you get nothing!

 

Moreover by daily posting latest buzz you read / watched on Facebook etc. or simply saying what's happening with you, where you're situated now etc., you slowly get addicted to posting – yes for good or bad people tend to be maniacal).

By placing all of your pesronal or impersonal stuff online, you're making these sites better index their sites into Google / Yahoo / Yandex search engines and therefore making them profitable and high ranked websites on the internet and giving out your personal time for Facebook profit? + you loose control over your data (your data is not physically on your side but situated on some remote server, somewhere on the internet).
 

Best avarage time to post on Tweet Facebook, Google+ and Linkedin

best-time-and-day-to-write-new-articles-schedule-content-at-the-right-time-on-social-media-to-get-high-trafficrank

So What is Best Day timing to Post, Pin or Tweet?

Below is an infographic I fond on this blog (visual data is originalcompiled by SurePayRoll) and showing visualized results from some extensive research on the topic.

best-time-to-post-and-tweet-blog-articles-social-media-infographic


Here is most important facts this infographic reveals:


The avarage best time to post tweet and pin your new articles is about 15:00 h
 

  • Best timing to post on Twitter is on Mondays to Thursdays from 13:00 to 15:00 h
  • Best timing to post on facebook is between 13:00 and 16:00 h
  • For Linkedin it is best to place your publish between Tuesdays to Thursdays


Peak times on Facebook, Twitter and Linkedin

  • Peak times for use of Facebook is on Wednesdays about 15:00 h
  • Peak times for use of Twitter is from Monday to Thursdays from 9:00  to 15:00 h
  • Linkedin Peak time is from 17:00 to 18:00 h
  • Including images to your articles increases traffic, tweets with images increase visits, favorites and leads


Worst time (when users will probably not view your content) on FB, Twitter and Linkedin

  • Weekends before 08:00  and after 20:00 h
  • Everyday after 20:00 and Fridays after 15:00 noon
  • Mondays and Fridays from 22:00 to 06:00 morning

Facts about Google+
 

  • Google+ is the fastest growing demographic social network for people aged 45 to 54
  • Best time to share your posts on Google+ is from 09:00 to 10:00 in the morning
  • Including images to your articles increases traffic, tweets with images increase visits, favorites and leads
     

Images generate more traffic and engagement

  • Including images to your articles increases traffic, tweets with images increase visits, favorites and leads


I'm aware as every research above info on best time to tweet and post is just a generalization and according to field of information posted suggested time could be different from optiomal time for individual writer, however as a general direction, info is very useful and it gives you some idea.
Twitter engagement for brands is 17% higher on weekends according to Dan Zarrella’s research. Tweets posted on Friday, Saturday and Sunday had higher CTR (Click Through Rate) than those posted in the rest of the week.

tweet-on-the-weekends-is-better-for-high-click-through-rate

Other best day to tweet other than weekends is mid-week time Wednesday.
Whether your site or blog is using retweet to generate more traffic to website best time to retweet is said to be around 5 pm. CTR is higher

Install certbot on Debian, Ubuntu, CentOS, Fedora Linux 10 / Generate and use Apache / Nginx SSL Letsencrypt certificates

Monday, December 21st, 2020

letsencrypt certbot install on any linux distribution with apache or nginx webserver howto</a><p> Let's Encrypt is a free, automated, and open certificate authority brought to you by the nonprofit <a data-cke-saved-href=
Internet Security Research Group (ISRG). ISRG group gave initiative with the goal to "encrypt the internet", i.e. offer free alternative to the overpriced domani registrer sold certificates with the goal to make more people offer SSL / TSL Free secured connection line on their websites. 
ISRG group supported Letsencrypt non-profit certificate authority actrively by Internet industry standard giants such as Mozilla, Cisco, EFF (Electronic Frontier Foundation),  Facebook, Google Chrome, Amazon AWS, OVH Cloud, Redhat, VMWare, Github and many many of the leading companies in IT.

Letsencrpyt is aimed at automating the process designed to overcome manual creation, validation, signing, installation, and renewal of certificates for secure websites. I.e. you don't have to manually write on console complicated openssl command lines with passing on Certificate CSR /  KEY / PEM files etc and generate Self-Signed Untrusted Authority Certificates (noted in my previous article How to generate Self-Signed SSL Certificates with openssl or use similar process to pay money generate secret key and submit the key to third party authority through a their website webadmin  interface in order to Generate SSL brought by Godaddy or Other Certificate Authority.

But of course as you can guess there are downsides as you submit your private key automatically via letsencrypt set of SSL certificate automation domain scripts to a third party Certificate Authority which is at Letsencrypt.org. A security intrusion in their private key store servers might mean a catastrophy for your data as malicious stealer might be able to decrypt your data with some additional effort and see in plain text what is talking to your Apache / Nginx or Mail Server nevertheless the cert. Hence for a high standards such as PCI environments Letsencrypt as well as for the paranoid security freak admins,  who don't trust the mainstream letsencrypt is definitely not a choice. Anyways for most small and midsized businesses who doesn't hold too much of a top secret data and want a moderate level of security Letsencrypt is a great opportunity to try. But enough talk, lets get down to business.

How to install and use certbot on Debian GNU / Linux 10 Buster?
Certbot is not available from the Debian software repositories by default, but it’s possible to configure the buster-backports repository in your /etc/apt/sources.list file to allow you to install a backport of the Certbot software with APT tool.
 

1. Install certbot on Debian / Ubuntu Linux

 

root@webserver:/etc/apt# tail -n 1 /etc/apt/sources.list
deb http://ftp.debian.org/debian buster-backports main


If not there append the repositories to file:

 

  • Install certbot-nginx certbot-apache deb packages

root@webserver:/ # echo 'deb http://ftp.debian.org/debian buster-backports main' >> /etc/apt/sources.list

 

  • Install certbot-nginx certbot-apache deb packages

root@webserver:/ # apt update
root@webserver:/ # apt install certbot python-certbot-nginx python3-certbot-apache python-certbot-nginx-doc


This will install the /usr/bin/certbot python executable script which is used to register / renew / revoke / delete your domains certificates.
 

2. Install letsencrypt certbot client on CentOS / RHEL / Fedora and other Linux Distributions

 


For RPM based distributions and other Linux distributions you will have to install snap package (if not already installed) and use snap command :

 

 

[root@centos ~ :] # yum install snapd
systemctl enable –now snapd.socket

To enable classic snap support, enter the following to create a symbolic link between

[root@centos ~ :] # ln -s /var/lib/snapd/snap /snap

snap command lets you install, configure, refresh and remove snaps.  Snaps are packages that work across many different Linux distributions, enabling secure delivery and operation of the latest apps and utilities.

[root@centos ~ :] # snap install core; sudo snap refresh core

Logout from console or Xsession to make the snap update its $PATH definitions.

Then use snap universal distro certbot classic package

 [root@centos ~ :] # snap install –classic certbot
[root@centos ~ :] # ln -s /snap/bin/certbot /usr/bin/certbot
 

 

If you're having an XOrg server access on the RHEL / CentOS via Xming or other type of Xemulator you might check out also the snap-store as it contains a multitude of packages installable which are not usually available in RPM distros.

 [root@centos ~ :] # snap install snap-store


how-to-install-snap-applications-on-centos-rhel-linux-snap-store

snap-store is a powerful and via it you can install many non easily installable stuff on Linux such as eclipse famous development IDE, notepad++ , Discord, the so favourite for the Quality Assurance guy Protocol tester Postman etc.

  • Installing certbot to any distribution via acme.sh script

Another often preferred solution to Universally deploy  and upgrade an existing LetsEncrypt program to any Linux distribution (e.g. RHEL / CentOS / Fedora etc.) is the acme.sh script. To install acme you have to clone the repository and run the script with –install

P.S. If you don't have git installed yet do

root@webserver:/ # apt-get install –yes git


and then the usual git clone to fetch it at your side

# cd /root
# git clone https://github.com/acmesh-official/acme.sh
Cloning into 'acme.sh'…
remote: Enumerating objects: 71, done.
remote: Counting objects: 100% (71/71), done.
remote: Compressing objects: 100% (53/53), done.
remote: Total 12475 (delta 39), reused 38 (delta 18), pack-reused 12404
Receiving objects: 100% (12475/12475), 4.79 MiB | 6.66 MiB/s, done.
Resolving deltas: 100% (7444/7444), done.

# sh acme.sh –install


To later upgrade acme.sh to latest you can do

# sh acme.sh –upgrade


In order to renew a concrete existing letsencrypt certificiate

# sh acme.sh –renew domainname.com


To renew all certificates using acme.sh script

# ./acme.sh –renew-all

 

3. Generate Apache or NGINX Free SSL / TLS Certificate with certbot tool

Now lets generate a certificate for a domain running on Apache Webserver with a Website WebRoot directory /home/phpdev/public/www

 

root@webserver:/ # certbot –apache –webroot -w /home/phpdev/public/www/ -d your-domain-name.com -d your-domain-name.com

root@webserver:/ # certbot certonly –webroot -w /home/phpdev/public/www/ -d your-domain-name.com -d other-domain-name.com


As you see all the domains for which you will need to generate are passed on with -d option.

Once certificates are properly generated you can test it in a browser and once you're sure they work as expected usually you can sleep safe for the next 3 months ( 90 days) which is the default for TSL / SSL Letsencrypt certificates the reason behind of course is security.

 

4. Enable freshly generated letsencrypt SSL certificate in Nginx VirtualHost config

Go to your nginx VirtualHost configuration (i.e. /etc/nginx/sites-enabled/phpmyadmin.adzone.pro ) and inside chunk of config add after location { … } – 443 TCP Port SSL listener (as in shown in bolded configuration)
 

server {

….
   location ~ \.php$ {
      include /etc/nginx/fastcgi_params;
##      fastcgi_pass 127.0.0.1:9000;
      fastcgi_pass unix:/run/php/php7.3-fpm.sock;
      fastcgi_index index.php;
      fastcgi_param SCRIPT_FILENAME /usr/share/phpmyadmin$fastcgi_script_name;
   }
 

 

 

    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/phpmyadmin.adzone.pro/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/phpmyadmin.adzone.pro/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

 

5. Enable new generated letsencrypt SSL certificate in Apache VirtualHost


In /etc/apache2/{sites-available,sites-enabled}/your-domain.com-ssl.conf you should have as a minimum a configuration setup like below:
 

 

NameVirtualHost *:443 <VirtualHost 123.123.123.12:443>
    ServerAdmin hipo@domain.com
    ServerName www.pc-freak.net
    ServerAlias www.your-domain.com wwww.your-domain.com your-domain.com
 
    HostnameLookups off
    DocumentRoot /var/www
    DirectoryIndex index.html index.htm index.php index.html.var

 

 

CheckSpelling on
SSLEngine on

    <Directory />
        Options FollowSymLinks
        AllowOverride All
        ##Order allow,deny
        ##allow from all
        Require all granted
    </Directory>
    <Directory /var/www>
        Options Indexes FollowSymLinks MultiViews
        AllowOverride All
##      Order allow,deny
##      allow from all
Require all granted
    </Directory>

Include /etc/letsencrypt/options-ssl-apache.conf
SSLCertificateFile /etc/letsencrypt/live/your-domain.com/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/your-domain.com/privkey.pem
</VirtualHost>

 

6. Simulate a certificate regenerate with –dry-run

Soon before the 90 days period expiry approaches, it is a good idea to test how all installed Nginx webserver certficiates will be renewed and whether any issues are expected this can be done with the –dry-run option.

root@webserver:/ # certbot renew –dry-run

 

– – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – –
** DRY RUN: simulating 'certbot renew' close to cert expiry
**          (The test certificates below have not been saved.)

Congratulations, all renewals succeeded. The following certs have been renewed:
  /etc/letsencrypt/live/adzone.pro/fullchain.pem (success)
  /etc/letsencrypt/live/cdn.natsr.pro/fullchain.pem (success)
  /etc/letsencrypt/live/mail.adzone.pro/fullchain.pem (success)
  /etc/letsencrypt/live/natsr.pro-0001/fullchain.pem (success)
  /etc/letsencrypt/live/natsr.pro/fullchain.pem (success)
  /etc/letsencrypt/live/phpmyadmin.adzone.pro/fullchain.pem (success)
  /etc/letsencrypt/live/www.adzone.pro/fullchain.pem (success)
  /etc/letsencrypt/live/www.natsr.pro/fullchain.pem (success)
** DRY RUN: simulating 'certbot renew' close to cert expiry
**          (The test certificates above have not been saved.)
– – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – –

 

7. Renew a certificate from a multiple installed certificate list

In some time when you need to renew letsencrypt domain certificates you can list them and choose manually which one you want to renew.

root@webserver:/ # certbot –force-renewal
Saving debug log to /var/log/letsencrypt/letsencrypt.log

How would you like to authenticate and install certificates?
– – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – –
1: Apache Web Server plugin (apache)
2: Nginx Web Server plugin (nginx)
– – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – –
Select the appropriate number [1-2] then [enter] (press 'c' to cancel): 2
Plugins selected: Authenticator nginx, Installer nginx

Which names would you like to activate HTTPS for?
– – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – –
1: adzone.pro
2: mail.adzone.pro
3: phpmyadmin.adzone.pro
4: www.adzone.pro
5: natsr.pro
6: cdn.natsr.pro
7: www.natsr.pro
– – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – –
Select the appropriate numbers separated by commas and/or spaces, or leave input
blank to select all options shown (Enter 'c' to cancel): 3
Renewing an existing certificate
Deploying Certificate to VirtualHost /etc/nginx/sites-enabled/phpmyadmin.adzone.pro

Please choose whether or not to redirect HTTP traffic to HTTPS, removing HTTP access.
– – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – –
1: No redirect – Make no further changes to the webserver configuration.
2: Redirect – Make all requests redirect to secure HTTPS access. Choose this for
new sites, or if you're confident your site works on HTTPS. You can undo this
change by editing your web server's configuration.
– – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – –
Select the appropriate number [1-2] then [enter] (press 'c' to cancel): 2
Redirecting all traffic on port 80 to ssl in /etc/nginx/sites-enabled/phpmyadmin.adzone.pro

– – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – –
Your existing certificate has been successfully renewed, and the new certificate
has been installed.

The new certificate covers the following domains: https://phpmyadmin.adzone.pro

You should test your configuration at:
https://www.ssllabs.com/ssltest/analyze.html?d=phpmyadmin.adzone.pro
– – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – –

IMPORTANT NOTES:
 – Congratulations! Your certificate and chain have been saved at:
   /etc/letsencrypt/live/phpmyadmin.adzone.pro/fullchain.pem

   Your key file has been saved at:
   /etc/letsencrypt/live/phpmyadmin.adzone.pro/privkey.pem
   Your cert will expire on 2021-03-21. To obtain a new or tweaked
   version of this certificate in the future, simply run certbot again
   with the "certonly" option. To non-interactively renew *all* of
   your certificates, run "certbot renew"
 – If you like Certbot, please consider supporting our work by:

   Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
   Donating to EFF:                    https://eff.org/donate-le

 

8. Renew all present SSL certificates

root@webserver:/ # certbot renew

Processing /etc/letsencrypt/renewal/www.natsr.pro.conf
– – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – –
Cert not yet due for renewal

 

– – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – –

The following certs are not due for renewal yet:
  /etc/letsencrypt/live/adzone.pro/fullchain.pem expires on 2021-03-01 (skipped)
  /etc/letsencrypt/live/cdn.natsr.pro/fullchain.pem expires on 2021-02-28 (skipped)
  /etc/letsencrypt/live/mail.adzone.pro/fullchain.pem expires on 2021-02-28 (skipped)
  /etc/letsencrypt/live/natsr.pro-0001/fullchain.pem expires on 2021-03-01 (skipped)
  /etc/letsencrypt/live/natsr.pro/fullchain.pem expires on 2021-02-25 (skipped)
  /etc/letsencrypt/live/phpmyadmin.adzone.pro/fullchain.pem expires on 2021-03-21 (skipped)
  /etc/letsencrypt/live/www.adzone.pro/fullchain.pem expires on 2021-02-28 (skipped)
  /etc/letsencrypt/live/www.natsr.pro/fullchain.pem expires on 2021-03-01 (skipped)
No renewals were attempted.
– – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – –

 

 

9. Renew all existing server certificates from a cron job


The certbot package will install a script under /etc/cron.d/certbot to be run that will attempt every 12 hours however from my experience
often this script is not going to work, the script looks similar to below:

# Upgrade all existing SSL certbot machine certificates

 

0 */12 * * * root test -x /usr/bin/certbot -a \! -d /run/systemd/system && perl -e 'sleep int(rand(43200))' && certbot -q renew

Another approach to renew all installed certificates if you want to have a specific options and keep log of what happened is using a tiny shell script like this:

 

10. Auto renew installed SSL / TSL Certbot certificates with a bash loop over all present certificates

#!/bin/sh
# update SSL certificates
# prints from 1 to 104 (according to each certbot generated certificate and triggers rewew and logs what happened to log file
# an ugly hack for certbot certificate renew
for i in $(seq 1 104); do echo "Updating $i SSL Cert" | tee -a /root/certificate-update.log; yes "$i" | certbot –force-renewal | tee -a /root/certificate-update.log 2>&1; sleep 5; done

Note: The seq 1 104 is the range depends on the count of installed SSL certificates you have installed on the machine, that can be seen and set the proper value according to your case when you run one time certbot –force-renewal.
 

How to Configure Nginx as a Reverse Proxy Load Balancer on Debian, CentOS, RHEL Linux

Monday, December 14th, 2020

set-up-nginx-reverse-proxy-howto-linux-logo

What is reverse Proxy?

Reverse Proxy (RP) is a Proxy server which routes all incoming traffic to secondary Webserver situated behind the Reverse Proxy site.

Then all incoming replies from secondary webserver (which is not visible) from the internet gets routed back to Reverse Proxy service. The result is it seems like all incoming and outgoing HTTP requests are served from Reverse Proxy host where in reality, reverse proxy host just does traffic redirection. Problem with reverse proxies is it is one more point of failure the good side of it can protect and route only certain traffic to your webserver, preventing the behind reverse proxy located server from crackers malicious HTTP requests.

Treverse proxy, which accepts all traffic and forwards it to a specific resource, like a server or container.  Earlier I've blogged on how to create Apache reverse Proxy to Tomcat.
Having a reverse proxy with apache is a classical scenarios however as NGINX is taking lead slowly and overthrowing apache's use due to its easiness to configure, its high reliability and less consumption of resources.


One of most common use of Reverse Proxy is to use it as a software Load Balancer for traffic towards another webserver or directly to a backend server. Using RP as a to mitigate DDoS attacks from hacked computers Bot nets (coming from a network range) is very common Anti-DDoS protection measure.
With the bloom of VM and Contrainerizations technology such as docker. And the trend to switch services towards micro-services, often reverse proxy is used to seamessly redirect incoming requests traff to multiple separate OS docker running containers etc.


Some of the other security pros of using a Reverse proxy that could be pointed are:

  • Restrict access to locations that may be obvious targets for brute-force attacks, reducing the effectiveness of DDOS attacks by limiting the number of connections and the download rate per IP address. 
  • Cache pre-rendered versions of popular pages to speed up page load times.
  • Interfere with other unwanted traffic when needed.

 


what-is-reverse-proxy-explained-proxying-tubes

 

1. Install nginx webserver


Assuming you have a Debian at hand as an OS which will be used for Nginx RP host, lets install nginx.
 

[hipo@proxy ~]$ sudo su – root

[root@proxy ~]#  apt update

[root@proxy ~]# apt install -y nginx


Fedora / CentOS / Redhat Linux could use yum or dnf to install NGINX
 

[root@proxy ~]# dnf install -y nginx
[root@proxy ~]# yum install -y nginx

 

2. Launch nginx for a first time and test


Start nginx one time to test default configuration is reachable
 

systemctl enable –now nginx


To test nginx works out of the box right after install, open a browser and go to http://localhost if you have X or use text based browser such as lynx or some web console fetcher as curl to verify that the web server is running as expected.

nginx-test-default-page-centos-linux-screenshot
 

3. Create Reverse proxy configuration file

Remove default Nginx package provided configuration

As of 2020 by default nginx does load configuration from file /etc/nginx/sites-enabled/default on DEB based Linuxes and in /etc/nginx/nginx.conf on RPM based ones, therefore to setup our desired configuration and disable default domain config to be loaded we have to unlink on Debian

[root@proxy ~]# unlink /etc/nginx/sites-enabled/default

or move out the original nginx.conf on Fedora / CentOS / RHEL:
 

[root@proxy ~]# mv /etc/nginx/nginx.conf /etc/nginx/nginx.conf-distro

 

Lets take a scenario where you have a local IP address that's NAT-ted ot DMZ-ed and and you want to run nginx to server as a reverse proxy to accelerate traffic and forward all traffic to another webserver such as LigHTTPD / Apache or towards java serving Application server Jboss / Tomcat etc that listens on the same host on lets say on port 8000 accessible via app server from /application/.

To do so prepare new /etc/nginx/nginx.conf to look like so
 

[root@proxy ~]# mv /etc/nginx/nginx.conf /etc/nginx.conf.bak
[root@proxy ~]# vim /etc/nginx/nginx.conf

user nginx;
worker_processes auto;
worker_rlimit_nofile 10240;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

 

events {
#       worker_connections 768;
        worker_connections 4096;
        multi_accept on;
        # multi_accept on;
}

http {

        ##
        # Basic Settings
        ##

        sendfile on;
        tcp_nopush on;
        tcp_nodelay on;
        #keepalive_timeout 65;
        keepalive_requests 1024;
        client_header_timeout 30;
        client_body_timeout 30;
        keepalive_timeout 30;
        types_hash_max_size 2048;
        # server_tokens off;

        # server_names_hash_bucket_size 64;
        # server_name_in_redirect off;

        include /etc/nginx/mime.types;
        default_type application/octet-stream;

        ##
        # SSL Settings
        ##

        ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
        ssl_prefer_server_ciphers on;

        ##
        # Logging Settings
        ##

        access_log /var/log/nginx/domain.com/access.log;
        error_log /var/log/nginx/domain.com/error.log;

        ##
        # Gzip Settings
        ##

        gzip on;

        # gzip_vary on;
        # gzip_proxied any;
        # gzip_comp_level 6;
        # gzip_buffers 16 8k;
        # gzip_http_version 1.1;
        # gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

        ##
        # Virtual Host Configs
        ##

        include /etc/nginx/conf.d/*.conf;
        include /etc/nginx/sites-enabled/*;

 include /etc/nginx/default.d/*.conf;

upstream myapp {
    server 127.0.0.1:8000 weight=3;
    server 127.0.0.1:8001;
    server 127.0.0.1:8002;
    server 127.0.0.1:8003;
# Uncomment and use Load balancing with external FQDNs if needed
#  server srv1.example.com;
#   server srv2.example.com;
#   server srv3.example.co

}

#mail {
#       # See sample authentication script at:
#       # http://wiki.nginx.org/ImapAuthenticateWithApachePhpScript
#
#       # auth_http localhost/auth.php;
#       # pop3_capabilities "TOP" "USER";
#       # imap_capabilities "IMAP4rev1" "UIDPLUS";
#
#       server {
#               listen     localhost:110;
#               protocol   pop3;
#               proxy      on;
#       }
#
#       server {
#               listen     localhost:143;
#               protocol   imap;
#               proxy      on;
#       }
#}

 

In the example above, there are 3 instances of the same application running on 3 IPs on different ports, just for the sake to illustrate Fully Qualified Domain Names (FQDNs) Load balancing is also supported you can see the 3 commented application instances srv1-srv3.
 When the load balancing method is not specifically configured, it defaults to round-robin. All requests are proxied to the server group myapp1, and nginx applies HTTP load balancing to distribute the requests.Reverse proxy implementation in nginx includes load balancing for HTTP, HTTPS, FastCGI, uwsgi, SCGI, memcached, and gRPC.
To configure load balancing for HTTPS instead of HTTP, just use “https” as the protocol.


To download above nginx.conf configured for High traffic servers and supports Nginx virtualhosts click here.

Now lets prepare for the reverse proxy nginx configuration a separate file under /etc/nginx/default.d/ all files stored there with .conf extension are to be red by nginx.conf as instructed by /etc/nginx/nginx.conf :

We'll need prepare a sample nginx

[root@proxy ~]# vim /etc/nginx/sites-available/reverse-proxy.conf

server {

        listen 80;
        listen [::]:80;


 server_name domain.com www.domain.com;
#index       index.php;
# fallback for index.php usually this one is never used
root        /var/www/domain.com/public    ;
#location / {
#try_files $uri $uri/ /index.php?$query_string;
#}

        location / {
                    proxy_pass http://127.0.0.1:8080;
  }

 

location /application {
proxy_pass http://domain.com/application/ ;

proxy_http_version                 1.1;
proxy_cache_bypass                 $http_upgrade;

# Proxy headers
proxy_set_header Upgrade           $http_upgrade;
proxy_set_header Connection        "upgrade";
proxy_set_header Host              $host;
proxy_set_header X-Real-IP         $remote_addr;
proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host  $host;
proxy_set_header X-Forwarded-Port  $server_port;

# Proxy timeouts
proxy_connect_timeout              60s;
proxy_send_timeout                 60s;
proxy_read_timeout                 60s;

        access_log /var/log/nginx/reverse-access.log;
        error_log /var/log/nginx/reverse-error.log;

}

##listen 443 ssl;
##    ssl_certificate /etc/letsencrypt/live/domain.com/fullchain.pem;
##    ssl_certificate_key /etc/letsencrypt/live/domain.com/privkey.pem;
##    include /etc/letsencrypt/options-ssl-nginx.conf;
##    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;

}

Get above reverse-proxy.conf from here

As you see config makes all incoming traffic towards root ( / ) NGINX directory for domain http://domain.com on port 80 on Nginx Webserver to be passed on http://127.0.0.1:8000/application.

      location / {
                    proxy_pass http://127.0.0.1:8000;
  }


Another set of configuration has configuration domain.com/application to reverse proxy to Webserver on Port 8080 /application.

 

location /application {
proxy_pass http://domain.com/application/ ;

proxy_http_version                 1.1;
proxy_cache_bypass                 $http_upgrade;

# Proxy headers
proxy_set_header Upgrade           $http_upgrade;
proxy_set_header Connection        "upgrade";
proxy_set_header Host              $host;
proxy_set_header X-Real-IP         $remote_addr;
proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host  $host;
proxy_set_header X-Forwarded-Port  $server_port;

# Proxy timeouts
proxy_connect_timeout              60s;
proxy_send_timeout                 60s;
proxy_read_timeout                 60s;

        access_log /var/log/nginx/reverse-access.log;
        error_log /var/log/nginx/reverse-error.log;

}

– Enable new configuration to be active in NGINX

 

[root@proxy ~]# ln -s /etc/nginx/sites-available/reverse-proxy.conf /etc/nginx/sites-enabled/reverse-proxy.conf

 

4. Test reverse proxy nginx config for syntax errors

 

[root@proxy ~]# nginx -t

 

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

Test connectivity to listen external IP address

 

5. Enable nginx SSL letsencrypt certificates support

 

[root@proxy ~]# apt-get update
[root@proxy ~]# apt-get install software-properties-common

[root@proxy ~]# apt-get update
[root@proxy ~]# apt-get install python-certbot-nginx

 

6. Generate NGINX Letsencrypt certificates

 

[root@proxy ~]# certbot –nginx

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator nginx, Installer nginx

Which names would you like to activate HTTPS for?
– – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – –
1: your.domain.com
– – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – –
Select the appropriate numbers separated by commas and/or spaces, or leave input
blank to select all options shown (Enter 'c' to cancel): 1
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for your.domain.com
Waiting for verification…
Cleaning up challenges
Deploying Certificate to VirtualHost /etc/nginx/sites-enabled/reverse-proxy.conf

Please choose whether or not to redirect HTTP traffic to HTTPS, removing HTTP access.
– – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – –
1: No redirect – Make no further changes to the webserver configuration.
2: Redirect – Make all requests redirect to secure HTTPS access. Choose this for
new sites, or if you're confident your site works on HTTPS. You can undo this
change by editing your web server's configuration.
– – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – –
Select the appropriate number [1-2] then [enter] (press 'c' to cancel): 2
Redirecting all traffic on port 80 to ssl in /etc/nginx/sites-enabled/reverse-proxy.conf

– – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – –
Congratulations! You have successfully enabled https://your.domain.com

You should test your configuration at:
https://www.ssllabs.com/ssltest/analyze.html?d=your.domain.com
– – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – –

 

7. Set NGINX Reverse Proxy to auto-start on Linux server boot

On most modern Linux distros use systemctl for legacy machines depending on the Linux distribution use the addequate runlevel /etc/rc3.d/ symlink link on Debian based distros on Fedoras / CentOS / RHEL and other RPM based ones use chkconfig RedHat command.

 

[root@proxy ~]# systemctl start nginx
[root@proxy ~]# systemctl enable nginx

 

8. Fixing weird connection permission denied errors


If you get a weird permission denied errors right after you have configured the ProxyPass on Nginx and you're wondering what is causing it you have to know by default on CentOS 7 and RHEL you'll get this errors due to automatically enabled OS selinux security hardening.

If this is the case after you setup Nginx + HTTPD or whatever application server you will get errors in  /var/log/nginx.log like:

2020/12/14 07:46:01 [crit] 7626#0: *1 connect() to 127.0.0.1:8080 failed (13: Permission denied) while connecting to upstream, client: 127.0.0.1, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8080/", host: "localhost"
2020/12/14 07:46:01 [crit] 7626#0: *1 connect() to 127.0.0.1:8080 failed (13: Permission denied) while connecting to upstream, client: 127.0.0.1, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8080/", host: "localhost"
2020/12/14 07:46:01 [crit] 7626#0: *1 connect() to 127.0.0.1:8080 failed (13: Permission denied) while connecting to upstream, client: 127.0.0.1, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8080/", host: "localhost"
2020/12/14 07:46:02 [crit] 7626#0: *1 connect() to 127.0.0.1:8080 failed (13: Permission denied) while connecting to upstream, client: 127.0.0.1, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8080/", host: "localhost"


The solution to proxy_pass weird permission denied errors is to turn off selinux

[root@proxy ~]# setsebool -P httpd_can_network_connect 1

To permanently allow nginx and httpd

[root@proxy ~]# cat /var/log/audit/audit.log | grep nginx | grep denied | audit2allow -M mynginx
[root@proxy ~]# semodule -i mynginx.pp

 

[root@proxy ~]# cat /var/log/audit/audit.log | grep httpd | grep denied | audit2allow -M myhttpd
[root@proxy ~]# semodule -i myhttpd.pp


Then to let know nginx and httpd (or whatever else app you run) be aware of new settings restart both

[root@proxy ~]# systemctl restart nginx
[root@proxy ~]# systemctl restart httpd

How to install and use memcached on Debian GNU / Linux to share php sessions between DNS round robined Apache webservers

Monday, November 9th, 2020

apache-load-balancing-keep-persistent-php-sessions-memcached-logo

Recently I had to come up with a solution to make A bunch of websites hosted on a machine to be high available. For the task haproxy is one of logical options to use. However as I didn't wanted to set new IP addresses and play around to build a cluster. I decided the much more simplistic approach to use 2 separate Machines each running Up-to-date same version of Apache Webserver as front end and using a shared data running on Master-to-Master MySQL replication database as a backend. For the load balancing itself I've used a simple 2 multiple DNS 'A' Active records, configured via the Bind DNS name server an Round Robin DNS load balancing for each of the domains, to make them point to the the 2 Internet IP addresses (XXX.XXX.XXX.4 and YYY.YYY.YYY.5) each configured on the 2 Linux servers eth0.

So far so good, this setup worked but immediately, I've run another issue as I found out the WordPress and Joomla based websites's PHP sessions are lost, as the connectivity by the remote client browser reaches one time on XXX…4 and one time on YYY…4 configured listerner on TCP port 80 and TCP p. 443. In other words if request comes up to Front end Apache worker webserver 1 with opened channel data is sent back to Client Browser and the next request is sent due to the other IP resolved by the DNS server to come to Apache worker webserver 2 of course webserver 2 has no idea about this previous session data and it gets confused and returns soemething like a 404 or 500 or any other error … not exciting really huh …

I've thought about work around and as I didn't wanted to involve thirty party stuff as Privoxy / Squid  / Varnish / Polipo etc. just as that would add extra complexity as if I choose to use haproxy from the beginning, after short investigation came to a reason to use memcached as a central PHP sessions storage.

php-memcached-apache-workers-webbrowser-keep-sessions-diagram
 

Why I choose memcached ?


Well it is relatively easy to configure, it doesn't come with mambo-jambo unreadable over-complicated configuration and the time to configure everything is really little as well as the configuration is much straight forward, plus I don't need to occupy more IP addresses and I don't need to do any changes to the already running 2 WebServers on 2 separate Linux hosts configured to be reachable from the Internet.
Of course using memcached is not a rock solid and not the best solution out there, as there is risk that if a memcached dies out for some reason all sessions stored in are lost as they're stored only in volatile memory, as well as there is a drawback that if a communication was done via one of the 2 webservers and one of them goes down sessions that were known by one of Apache's workers disappears.

So let me proceed and explain you the steps to take to configure memcached as a central session storage system.
 

1. Install memcached and php-memcached packages


To enable support for memcached besides installing memcached daemon, you need to have the php-memcached which will provide the memcached.so used by Apache loaded php script interpretter module.

On a Debian / Ubuntu and other deb based GNU / Linux it should be:

webserver1:~# apt-get install memcached php-memcached

TO use php-memcached I assume Apache and its support for PHP is already installed with lets say:
 

webserver1:~# apt-get install php libapache2-mod-php php-mcrypt


On CentOS / RHEL / Fedora Linux it is a little bit more complicated as you'll need to install php-pear and compile the module with pecl

 

[root@centos ~]# yum install php-pear

[root@centos ~]# yum install php-pecl-memcache


Compile memcache

[root@centos ~]# pecl install memcache

 

2. Test if memcached is properly loaded in PHP


Once installed lets check if memcached service is running and memcached support is loaded as module into PHP core.

 

webserver1:~# ps -efa  | egrep memcached
nobody   14443     1  0 Oct23 ?        00:04:34 /usr/bin/memcached -v -m 64 -p 11211 -u nobody -l 127.0.0.1 -l 192.168.0.1

root@webserver1:/# php -m | egrep memcache
memcached


To get a bit more verbose information on memcache version and few of memcached variable settings:

root@webserver1:/# php -i |grep -i memcache
/etc/php/7.4/cli/conf.d/25-memcached.ini
memcached
memcached support => enabled
libmemcached version => 1.0.18
memcached.compression_factor => 1.3 => 1.3
memcached.compression_threshold => 2000 => 2000
memcached.compression_type => fastlz => fastlz
memcached.default_binary_protocol => Off => Off
memcached.default_connect_timeout => 0 => 0
memcached.default_consistent_hash => Off => Off
memcached.serializer => php => php
memcached.sess_binary_protocol => On => On
memcached.sess_connect_timeout => 0 => 0
memcached.sess_consistent_hash => On => On
memcached.sess_consistent_hash_type => ketama => ketama
memcached.sess_lock_expire => 0 => 0
memcached.sess_lock_max_wait => not set => not set
memcached.sess_lock_retries => 5 => 5
memcached.sess_lock_wait => not set => not set
memcached.sess_lock_wait_max => 150 => 150
memcached.sess_lock_wait_min => 150 => 150
memcached.sess_locking => On => On
memcached.sess_number_of_replicas => 0 => 0
memcached.sess_persistent => Off => Off
memcached.sess_prefix => memc.sess.key. => memc.sess.key.
memcached.sess_randomize_replica_read => Off => Off
memcached.sess_remove_failed_servers => Off => Off
memcached.sess_sasl_password => no value => no value
memcached.sess_sasl_username => no value => no value
memcached.sess_server_failure_limit => 0 => 0
memcached.store_retry_count => 2 => 2
Registered save handlers => files user memcached


Make sure /etc/default/memcached (on Debian is enabled) on CentOS / RHELs this should be /etc/sysconfig/memcached

webserver1:~# cat default/memcached 
# Set this to no to disable memcached.
ENABLE_MEMCACHED=yes

As assured on server1 memcached + php is ready to be used, next login to Linux server 2 and repeat the same steps install memcached and the module and check it is showing as loaded.

Next place under some of your webservers hosted websites under check_memcached.php below PHP code
 

<?php
if (class_exists('Memcache')) {
    $server = 'localhost';
    if (!empty($_REQUEST[‘server’])) {
        $server = $_REQUEST[‘server’];
    }
    $memcache = new Memcache;
    $isMemcacheAvailable = @$memcache->connect($server);

    if ($isMemcacheAvailable) {
        $aData = $memcache->get('data');
        echo '<pre>';
        if ($aData) {
            echo '<h2>Data from Cache:</h2>';
            print_r($aData);
        } else {
            $aData = array(
                'me' => 'you',
                'us' => 'them',
            );
            echo '<h2>Fresh Data:</h2>';
            print_r($aData);
            $memcache->set('data', $aData, 0, 300);
        }
        $aData = $memcache->get('data');
        if ($aData) {
            echo '<h3>Memcache seem to be working fine!</h3>';
        } else {
            echo '<h3>Memcache DOES NOT seem to be working!</h3>';
        }
        echo '</pre>';
    }
}

if (!$isMemcacheAvailable) {
    echo 'Memcache not available';
}

?>


Launch in a browser https://your-dns-round-robined-domain.com/check_memcached.php, the browser output should be as on below screenshot:

check_memcached-php-script-website-screenshot

3. Configure memcached daemons on both nodes

All we need to set up is the listen IPv4 addresses

On Host Webserver1
You should have in /etc/memcached.conf

-l 127.0.0.1
-l 192.168.0.1

webserver1:~# grep -Ei '\-l' /etc/memcached.conf 
-l 127.0.0.1
-l 192.168.0.1


On Host Webserver2

-l 127.0.0.1
-l 192.168.0.200

 

webserver2:~# grep -Ei '\-l' /etc/memcached.conf
-l 127.0.0.1
-l 192.168.0.200

 

4. Configure memcached in php.ini

Edit config /etc/php.ini (on CentOS / RHEL) or on Debians / Ubuntus etc. modify /etc/php/*/apache2/php.ini (where depending on the PHP version you're using your php location could be different lets say /etc/php/5.6/apache2/php.ini):

If you wonder where is the php.ini config in your case you can usually get it from the php cli:

webserver1:~# php -i | grep "php.ini"
Configuration File (php.ini) Path => /etc/php/7.4/cli
Loaded Configuration File => /etc/php/7.4/cli/php.ini

 

! Note: That on on PHP-FPM installations (where FastCGI Process Manager) is handling PHP requests,path would be rather something like:
 

/etc/php5/fpm/php.ini

in php.ini you need to change as minimum below 2 variables
 

session.save_handler =
session.save_path =


By default session.save_path would be set to lets say session.save_path = "

/var/lib/php7/sessions"


To make php use a 2 central configured memcached servers on webserver1 and webserver2 or even more memcached configured machines set it to look as so:

session.save_path="192.168.0.200:11211, 192.168.0.1:11211"


Also modify set

session.save_handler = memcache


Overall changed php.ini configuration on Linux machine 1 ( webserver1 ) and Linux machine 2 ( webserver2 ) should be:

session.save_handler = memcache
session.save_path="192.168.0.200:11211, 192.168.0.1:11211"

 

Below is approximately how it should look on both :

webserver1: ~# grep -Ei 'session.save_handler|session.save_path' /etc/php.ini
;; session.save_handler = files
session.save_handler = memcache
;     session.save_path = "N;/path"
;     session.save_path = "N;MODE;/path"
;session.save_path = "/var/lib/php7/sessions"
session.save_path="192.168.0.200:11211, 192.168.0.1:11211"
;       (see session.save_path above), then garbage collection does *not*
 

 

webserver2: ~# grep -Ei 'session.save_handler|session.save_path' /etc/php.ini
;; session.save_handler = files
session.save_handler = memcache
;     session.save_path = "N;/path"
;     session.save_path = "N;MODE;/path"
;session.save_path = "/var/lib/php7/sessions"
session.save_path="192.168.0.200:11211, 192.168.0.1:11211"
;       (see session.save_path above), then garbage collection does *not*


As you can see I have configured memcached on webserver1 to listen on internal local LAN IP 192.168.0.200 and on Local LAN eth iface 192.168.0.1 on TCP port 11211 (this is the default memcached connections listen port), for security or obscurity reasons you might choose another empty one. Make sure to also set the proper firewalling to that port, the best is to enable connections only between 192.168.0.200 and 192.168.0.1 on each of machine 1 and machine 2.

loadbalancing2-php-sessions-scheme-explained
 

5. Enable Memcached for session redundancy


Next step is to configure memcached to allow failover (e.g. use both memcached on 2 linux hosts) and configure session redundancy.
Configure /etc/php/7.3/mods-available/memcache.ini or /etc/php5/mods-available/memcache.ini or respectively to the right location depending on the PHP installed and used webservers version.
 

webserver1 :~#  vim /etc/php/7.3/mods-available/memcache.ini

; configuration for php memcached module
; priority=20
; settings to write sessions to both servers and have fail over
memcache.hash_strategy=consistent
memcache.allow_failover=1
memcache.session_redundancy=3
extension=memcached.so

 

webserver2 :~# vim /etc/php/7.3/mods-available/memcache.ini

; configuration for php memcached module
; priority=20
; settings to write sessions to both servers and have fail over
memcache.hash_strategy=consistent
memcache.allow_failover=1
memcache.session_redundancy=3
extension=memcached.so

 

memcache.session_redundancy directive must be equal to the number of memcached servers + 1 for the session information to be replicated to all the servers. This is due to a bug in PHP.
I have only 2 memcached configured that's why I set it to 3.
 

6. Restart Apache Webservers

Restart on both machines webserver1 and webserver2 Apache to make php load memcached.so
 

webserver1:~# systemctl restart httpd

webserver2:~# systemctl restart httpd

 

7. Restart memcached on machine 1 and 2

 

webserver1 :~# systemctl restart memcached

webserver2 :~# systemctl restart memcached

 

8. Test php sessions are working as expected with a php script

Copy to both website locations to accessible URL a file test_sessions.php:
 

<?php  
session_start();

if(isset($_SESSION[‘georgi’]))
{
echo "Sessions is ".$_SESSION[‘georgi’]."!\n";
}
else
{
echo "Session ID: ".session_id()."\n";
echo "Session Name: ".session_name()."\n";
echo "Setting 'georgi' to 'cool'\n";
$_SESSION[‘georgi’]='cool';
}
?>

 

Now run the test to see PHP sessions are kept persistently:
 

hipo@jeremiah:~/Desktop $ curl -vL -s https://www.pc-freak.net/session.php 2>&1 | grep 'Set-Cookie:'
< Set-Cookie: PHPSESSID=micir464cplbdfpo36n3qi9hd3; expires=Tue, 10-Nov-2020 12:14:32 GMT; Max-Age=86400; path=/

hipo@jeremiah:~/Desktop $ curl -L –cookie "PHPSESSID=micir464cplbdfpo36n3qi9hd3" http://83.228.93.76/session.php http://213.91.190.233/session.php
Session is cool!
Session is cool!

 

Copy to the locations that is resolving to both DNS servers some sample php script such as sessions_test.php  with below content:

<?php
    header('Content-Type: text/plain');
    session_start();
    if(!isset($_SESSION[‘visit’]))
    {
        echo "This is the first time you're visiting this server\n";
        $_SESSION[‘visit’] = 0;
    }
    else
            echo "Your number of visits: ".$_SESSION[‘visit’] . "\n";

    $_SESSION[‘visit’]++;

    echo "Server IP: ".$_SERVER[‘SERVER_ADDR’] . "\n";
    echo "Client IP: ".$_SERVER[‘REMOTE_ADDR’] . "\n";
    print_r($_COOKIE);
?>

Test in a Web Opera / Firefox / Chrome browser.

You should get an output in the browser similar to:
 

Your number of visits: 15
Server IP: 83.228.93.76
Client IP: 91.92.15.51
Array
(
    [_ga] => GA1.2.651288003.1538922937
    [__utma] => 238407297.651288003.1538922937.1601730730.1601759984.45
    [__utmz] => 238407297.1571087583.28.4.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not provided)
    [shellInABox] => 467306938:1110101010
    [fpestid] => EzkIzv_9OWmR9PxhUM8HEKoV3fbOri1iAiHesU7T4Pso4Mbi7Gtt9L1vlChtkli5GVDKtg
    [__gads] => ID=8a1e445d88889784-22302f2c01b9005b:T=1603219663:RT=1603219663:S=ALNI_MZ6L4IIaIBcwaeCk_KNwmL3df3Z2g
    [PHPSESSID] => mgpk1ivhvfc2d0daq08e0p0ec5
)

If you want to test php sessions are working with text browser or from another external script for automation use something as below PHP code:
 

<?php
// save as "session_test.php" inside your webspace  
ini_set('display_errors', 'On');
error_reporting(6143);

session_start();

$sessionSavePath = ini_get('session.save_path');

echo '<br><div style="background:#def;padding:6px">'
   , 'If a session could be started successfully <b>you should'
   , ' not see any Warning(s)</b>, otherwise check the path/folder'
   , ' mentioned in the warning(s) for proper access rights.<hr>';
echo "WebServer IP:" . $_SERVER[‘SERVER_ADDR’] . "\n<br />";
if (empty($sessionSavePath)) {
    echo 'A "<b>session.save_path</b>" is currently',
         ' <b>not</b> set.<br>Normally "<b>';
    if (isset($_ENV[‘TMP’])) {
        echo  $_ENV[‘TMP’], ‘” ($_ENV[“TMP”]) ';
    } else {
        echo '/tmp</b>" or "<b>C:\tmp</b>" (or whatever',
             ' the OS default "TMP" folder is set to)';
    }    
    echo ' is used in this case.';
} else {
    echo 'The current "session.save_path" is "<b>',
         $sessionSavePath, '</b>".';
}

echo '<br>Session file name: "<b>sess_', session_id()
   , '</b>".</div><br>';
?>

You can download the test_php_sessions.php script here.

To test with lynx:

hipo@jeremiah:~/Desktop $ lynx -source 'https://www.pc-freak.net/test_php_sessions.php'
<br><div style="background:#def;padding:6px">If a session could be started successfully <b>you should not see any Warning(s)</b>, otherwise check the path/folder mentioned in the warning(s) for proper access rights.<hr>WebServer IP:83.228.93.76
<br />The current "session.save_path" is "<b>tcp://192.168.0.200:11211, tcp://192.168.0.1:11211</b>".<br>Session file name: "<b>sess_5h18f809b88isf8vileudgrl40</b>".</div><br>

Start Stop Restart Microsoft IIS Webserver from command line and GUI

Thursday, April 17th, 2014

start-stop-restart-microsoft-iis-howto-iis-server-logo
For a decomissioning project just recently I had the task to stop Microsoft IIS  on Windows Server system.
If you have been into security for a while you know well how many vulnerabilities Microsoft (Internet Information Server) Webserver used to be. Nowadays things with IIS are better but anyways it is better not to use it if possible …

Nomatter what the rason if you need to make IIS stop serving web pages here is how to do it via command line:

At Windows Command Prompt, type:

net stop WAS

If the command returns error message to stop it type:

net stop W3SVC

stop-microsoft-IIS-webservice
Just in case you have to start it again run:

net start W3SVC

start-restart-IIS-webserver-screenshot

For those who prefer to do it from GUI interface, launch services.msc command from Windows Run:

> services.msc

services-msc-stop-microsoft-iis-webserver

In list of services lookup for
IIS Admin Service and HTTP SSL
a) (Click over it with right mouse button -> Properties)
b) Set Startup type to Manual
c) Click Stop Button

You're done now IIS is stopped to make sure it is stopped you can run from cmd.exe:

telnet localhost 80

when not working you should get 'Could not open connection to the host. on port 80: Connection failed' like shown up in screenshot.

How to check Microsoft IIS webserver version

Monday, July 21st, 2014

If you have to tune some weirdly behaviour Microsoft IIS (Internet Information Services) webserver, the first thing to do is to collect information about the system you're dealing with – get version of installed Windows and check what kind of IIS version is running on the Windows server?

To get the version of installed Windows on the system you just logged in, the quickest way I use is:
 

Start -> My Computer (right mouse button) Properties

check-windows-server-version-screenshot-windows-2003-r2

Run regedit from cmd.exe and go and check value of registry value:

 

HKEY_LOCAL_MACHINE\SOFTWARE\MicrosoftInetStp\VersionString


check-iis-webserver-version-with-windows-registry-screenshot

As you can see in screenshot in this particular case it is IIS version 6.0.

An alternative way to check the IIS version in some cases (if IIS version return is not disabled) is to telnet to webserver:

telnet your-webserver 80
 


Once connected Send:

HEAD / HTTP/1.0


Also on some Windows versions it is possible to check IIS webserver version from Internet Information Services Management Cosnole:

To check IIS version from IIS Manager:

Start (button) -> Control Panel -> Administrative Tools -> "Internet Information Services" IIS Manager

From IIS Manager go to:

Help -> About Microsoft Management Console


Here is a list with most common IIS version output you will get depending on the version of Windows server:

 

Windows NT 3.51 1.0
Windows NT 4 2.0-4.0
Windows Server 2000 5.0
Windows XP Professional 5.1
Windows Server 2003 6.0
Windows Vista 7.0
Windows Server 2008 7.0
Windows Server 2008 R2 7.5
Windows 7 7.5
Windows Server 2012 8.0
Windows 8 8.0
Windows Server 2012 R2 8.5
Windows 8.1 8.5

If you have only an upload FTP access to a Folder served by IIS Webserver – i.e. no access to the Win server running IIS, you can also grasp the IIS version with following .ASP code:
 

<%
response.write(Request.ServerVariables("SERVER_SOFTWARE"))
%>


Save the file as anyfile.asp somewhere in IIS docroot and invoke it in browser.