Posts Tagged ‘simple’

How to run multiple processes in parallel with xargs

Wednesday, May 29th, 2024

In our company there is a legacy application which developers run in multiple consoles launching its in intedependent components together in different consoles by simply running each component, the question comes then how this can be scripted so no waste of people is done to manually run the different componets from different parallel consoles. To achive the run in parallel of the multiple programs in parallel and background it with xargs and eval (integrated bash command) in Linux from a single script you can use a simple one liner like in the example.

#run in parallel
xargs -P <n> allows you to run <n> commands in parallel.
time xargs -P 3 -I {} sh -c 'eval "$1"' – {} <<'EOF'
program1; sleep 1; echo 1
program2 ; sleep 2; echo 2
program3; sleep 3; echo 3
echo 4
EOF

You can attune the delay up to the exact requirements or completely remove it and the multi run script is ready, enjoy.

Tags: consoles, EOF, How to, linux?, multiple, running, script, simple, sleep, waste
Posted in Bash Scripting, Linux | No Comments »

How to count number of ESTABLISHED state TCP connections to a Windows server

Wednesday, March 13th, 2024

Even if you have the background of a Linux system administrator, sooner or later you will have have to deal with some Windows hosts, thus i'll blog in this article shortly on how the established TCP if it happens you will have to administarte a Windows hosts or help a windows sysadmin noobie 🙂

In Linux it is pretty easy to check the number of established conenctions, because of the wonderful command wc (word count). with a simple command like:

$ netstat -etna |wc -l

Then you will get the number of active TCP connections to the machine and based on that you can get an idea on how busy the server is.

But what if you have to deal with lets say a Microsoft Windows 2012 /2019 / 2020 or 2022 Server, assuming you logged in as Administrator and you see the machine is quite loaded and runs multiple Native Windows Administrator common services such as IIS / Active directory Failover Clustering, Proxy server etc.
How can you identify the established number of connections via a simple command in cmd.exe?

1.Count ESTABLISHED TCP connections from Windows Command Line

Here is the answer, simply use netstat native windows command and combine it with find, like that and use the /i (ignores the case of characters when searching the string) /c (count lines containing the string) options

C:\Windows\system32>netstat -p TCP -n| find /i "ESTABLISHED" /c
1268

Voila, here are number of established connections, only 1268 that is relatively low.
However if you manage Windows servers, and you get some kind of hang ups as part of the monitoring, it is a good idea to setup a script based on this simple command for at least Windows Task Scheduler (the equivallent of Linux's crond service) to log for Peaks in Established connections to see whether Server crashes are not related to High Rise in established connections.
Even better if company uses Zabbix / Nagios, OpenNMS or other old legacy monitoring stuff like Joschyd even as of today 2024 used in some big of the TOP IT companies such as SAP (they were still using it about 4 years ago for their SAP HANA Cloud), you can set the script to run and do a Monitoring template or Alerting rules to draw you graphs and Trigger Alerts if your connections hits a peak, then you at least might know your Windows server is under a "Hackers" Denial of Service attack or there is something happening on the network, like Cisco Network Infrastructure Switch flappings or whatever.

Perhaps an example script you can use if you decide to implement the little nestat established connection checks Monitoring in Zabbix is the one i've writen about in the previous article "Calculate established connection from IP address with shell script and log to zabbix graphic".

2. Few Useful netstat options for the Windows system admin

C:\Windows\System32> netstat -bona

Cmd.exe will lists executable files, local and external IP addresses and ports, and the state in list form. You immediately see which programs have created connections or are listening so that you can find offenders quickly.

b – displays the executable involved in creating the connection.
o – displays the owning process ID.
n – displays address and port numbers.
a – displays all connections and listening ports.

As you can see in the screenshot, by using netstat -bona you get which process has binded to which local address and the Process ID PID of it, that is pretty useful in debugging stuff.

3. Use a Third Party GUI tool to debug more interactively connection issues

If you need to keep an eye in interactive mode, sometimes if there are issues CurrPorts tool can be of a great help

currports-windows-network-connections-diagnosis-cports

CurrPorts Tool own Description

CurrPorts is network monitoring software that displays the list of all currently opened TCP/IP and UDP ports on your local computer. For each port in the list, information about the process that opened the port is also displayed, including the process name, full path of the process, version information of the process (product name, file description, and so on), the time that the process was created, and the user that created it.
In addition, CurrPorts allows you to close unwanted TCP connections, kill the process that opened the ports, and save the TCP/UDP ports information to HTML file , XML file, or to tab-delimited text file.
CurrPorts also automatically mark with pink color suspicious TCP/UDP ports owned by unidentified applications (Applications without version information and icons).

Sum it up

What we learned is how to calculate number of established TCP connections from command line, useful for scripting, how you can use netstat to display the process ID and Process name that relates to a used Local / Remote TCP connections, and how eventually you can use this to connect it to some monitoring tool to periodically report High Peaks with TCP established connections (usually an indicator of servere system issues).

Tags: blog, cmd, connections, count tcp connections, created, CurrPorts, find, hang ups, How to, idea, information, linux?, list, machine, netstat, number, ports, process, script, simple, string, tcp, use, Windows, windows server
Posted in Curious Facts, Monitoring, System Administration, System Optimization, Various, Windows | No Comments »

Create Haproxy Loadbalancer Access Control Lists and forward incoming frontend traffics based on simple logic

Friday, February 16th, 2024

Haproxy Load Balancers could do pretty much to load balance traffic between application servers. The most straight forward way to use is to balance traffic for incoming Frontends towards a Backend configuration with predefined Application machines and ports to send the traffic, where one can be the leading one and others be set as backup or we can alternatively send the traffic towards a number of machines incoming to a Frontend port bind IP listener and number of backend machine.

Besides this the more interesting capabilities of Haproxy comes with using Access Control Lists (ACLs) to forward Incoming Frontend (FT) traffic towards specific backends and ports based on logic, power ACLs gives to Haproxy to do a sophisticated load balancing are enormous.
In this post I'll give you a very simple example on how you can save some time, if you have already a present Frontend listening to a Range of TCP Ports and it happens you want to redirect some of the traffic towards a spefic predefined Backend.

This is not the best way to it as Access Control Lists will put some extra efforts on the server CPU, but as today machines are quite powerful, it doesn't really matter. By using a simple ACLs as given in below example, one can save much of a time of writting multiple frontends for a complete sequential port range, if lets say only two of the ports in the port range and distinguish and redirect traffic incoming to Haproxy frontend listener in the port range of 61000-61230 towards a certain Ports that are supposed to go to a Common Backends to a separate ones, lets say ports 61115 and 61215.

Here is a short description on the overall screnarios. We have an haproxy with 3 VIP (Virtual Private IPs) with a Single Frontend with 3 binded IPs and 3 Backends, there is a configured ACL rule to redirect traffic for certain ports, the overall Load Balancing config is like so:

Frontend (ft):

ft_PROD:
listen IPs:

192.168.0.77
192.168.0.83
192.168.0.78

On TCP port range: 61000-61299

Backends (bk):

bk_PROD_ROUNDROBIN
bk_APP1
bk_APP2

Config Access Control Liststo seperate incoming haproxy traffic for CUSTOM_APP1 and CUSTOM_APP2

By default send all incoming FT traffic to: bk_PROD_ROUNDROBIN

With exception for frontend configured ports on:
APP1 port 61115
APP2 port 61215

If custom APP1 send to bk:
RULE1
If custom APP2 send to bk:
RULE2

Config on frontends traffic send operation:

bk_PROD_ROUNDROBIN (roundrobin) traffic send to App machines all in parallel
traffic routing mode (roundrobin)
Appl1
Appl2
Appl3
Appl4

bk_APP1 and bk_APP2

traffic routing mode: (balance source)
Appl1 default serving host

If configured check port 61888, 61887 is down, traffic will be resend to configured pre-configured backup hosts:

Appl2
Appl3
Appl4

/etc/haproxy/haproxy.cfg that does what is described with ACL LB capabilities looks like so:

#———————————————————————
# Global settings
#———————————————————————
global
log 127.0.0.1 local2

chroot /var/lib/haproxy
pidfile /var/run/haproxy.pid
maxconn 4000
user haproxy
group haproxy
daemon

# turn on stats unix socket
stats socket /var/lib/haproxy/stats

#———————————————————————
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#———————————————————————
defaults
mode tcp
log global
option tcplog
#option dontlognull
#option http-server-close
#option forwardfor except 127.0.0.0/8
option redispatch
retries 7
#timeout http-request 10s
timeout queue 10m
timeout connect 30s
timeout client 20m
timeout server 10m
#timeout http-keep-alive 10s
timeout check 30s
maxconn 3000

#———————————————————————
# Synchronize server entries in sticky tables
#———————————————————————

peers hapeers
peer haproxy1-fqdn.com 192.168.0.58:8388
peer haproxy2-fqdn.com 192.168.0.79:8388

#———————————————————————
# HAProxy Monitoring Config
#———————————————————————
listen stats 192.168.0.77:8080 #Haproxy Monitoring run on port 8080
mode http
option httplog
option http-server-close
stats enable
stats show-legends
stats refresh 5s
stats uri /stats #URL for HAProxy monitoring
stats realm Haproxy\ Statistics
stats auth hauser:secretpass4321 #User and Password for login to the monitoring dashboard
stats admin if TRUE
#default_backend bk_Prod1 #This is optionally for monitoring backend
#———————————————————————
# HAProxy Monitoring Config
#———————————————————————
#listen stats 192.168.0.83:8080 #Haproxy Monitoring run on port 8080
# mode http
# option httplog
# option http-server-close
# stats enable
# stats show-legends
# stats refresh 5s
# stats uri /stats #URL for HAProxy monitoring
# stats realm Haproxy\ Statistics
# stats auth hauser:secretpass321 #User and Password for login to the monitoring dashboard
# stats admin if TRUE
# #default_backend bk_Prod1 #This is optionally for monitoring backend

#———————————————————————
# HAProxy Monitoring Config
#———————————————————————
# listen stats 192.168.0.78:8080 #Haproxy Monitoring run on port 8080
# mode http
# option httplog
# option http-server-close
# stats enable
# stats show-legends
# stats refresh 5s
# stats uri /stats #URL for HAProxy monitoring
# stats realm Haproxy\ Statistics
# stats auth hauser:secretpass123 #User and Password for login to the monitoring dashboard
# stats admin if TRUE
# #default_backend bk_DKV_PROD_WLPFO #This is optionally for monitoring backend

#———————————————————————
# frontend which proxys to the backends
#———————————————————————
frontend ft_PROD
mode tcp
bind 192.168.0.77:61000-61299
bind 192.168.0.83:51000-51300
bind 192.168.0.78:51000-62300
option tcplog
# (4) Peer Sync: a sticky session is a session maintained by persistence
stick-table type ip size 1m peers hapeers expire 60m
# Commented for change CHG0292890
# stick on src
log-format %ci:%cp\ [%t]\ %ft\ %b/%s\ %Tw/%Tc/%Tt\ %B\ %ts\ %ac/%fc/%bc/%sc/%rc\ %sq/%bq
acl RULE1 dst_port 61115
acl RULE2 dst_port 61215
use_backend APP1 if app1
use_backend APP2 if app2
default_backend bk_PROD_ROUNDROBIN

#———————————————————————
# round robin balancing between the various backends
#———————————————————————
backend bk_PROD_ROUNDROBIN
mode tcp
# (0) Load Balancing Method.
balance roundrobin
# (4) Peer Sync: a sticky session is a session maintained by persistence
stick-table type ip size 1m peers hapeers expire 60m
# (5) Server List
# (5.1) Backend
server appl1 10.33.0.50 check port 31232
server appl2 10.33.0.51 check port 31232
server appl2 10.45.0.78 check port 31232
server appl3 10.45.0.79 check port 31232

#———————————————————————
# source balancing for the GUI
#———————————————————————
backend bk_APP2
mode tcp
# (0) Load Balancing Method.
balance source
# (4) Peer Sync: a sticky session is a session maintained by persistence
stick-table type ip size 1m peers hapeers expire 60m
stick on src
# (5) Server List
# (5.1) Backend
server appl1 10.33.0.50 check port 55232
server appl2 10.32.0.51 check port 55232 backup
server appl3 10.45.0.78 check port 55232 backup
server appl4 10.45.0.79 check port 55232 backup

#———————————————————————
# source balancing for the OLW
#———————————————————————
backend bk_APP1
mode tcp
# (0) Load Balancing Method.
balance source
# (4) Peer Sync: a sticky session is a session maintained by persistence
stick-table type ip size 1m peers hapeers expire 60m
stick on src
# (5) Server List
# (5.1) Backend
server appl1 10.33.0.50 check port 53119
server appl2 10.32.0.51 check port 53119 backup
server appl3 10.45.0.78 check port 53119 backup
server appl4 10.45.0.79 check port 53119 backup

You can also check and download the haproxy.cfg here.
Enjjoy !

Tags: backup, based, Create Haproxy Loadbalancer Access Control Lists, frontends, logic, option, peers, ports, range, Server List, session, short description, simple, size, tcp, traffic, User
Posted in Everyday Life, Haproxy, Linux, Web and CMS | No Comments »

How to monitor Postfix Mail server work correct with simple one liner Zabbix user parameter script / Simple way to capture and report SMTP machine issues Zabbix template

Thursday, June 22nd, 2023

In this article, I'm going to show you how to setup a very simple monitoring if a local running SMTP (Postfix / Qmail / Exim) is responding correctly on basic commands. The check would helpfully keep you in track to know whether your configured Linux server local MTA (Mail Transport Agent) is responding on requests on TCP / IP protocol Port 25, as well as a check for process existence of master (that is the main postfix) proccess, as well as the usual postfix spawned sub-processes qmgr (the postfix queue manager), tsl mgr (TLS session cache and PRNG manager), pickup (Postfix local mail pickup) – or email receiving process.

Normally a properly configured postfix installation on a Linux whatever you like distribution would look something like below:

# ps -ef|grep -Ei 'master|postfix'|grep -v grep
root 1959 1 0 Jun21 ? 00:00:00 /usr/libexec/postfix/master -w
postfix 1961 1959 0 Jun21 ? 00:00:00 qmgr -l -t unix -u
postfix 4542 1959 0 Jun21 ? 00:00:00 tlsmgr -l -t unix -u
postfix 2910288 1959 0 11:28 ? 00:00:00 pickup -l -t unix -u

At times, during mail server restarts the amount of processes that are sub spawned by postfix, may very and if you a do a postfix restart

# systemctl restart postfix

The amout of spawned processes running as postfix username might decrease, and only qmgr might be available for second thus in the consequential shown Template the zabbix processes check to make sure the Postfix is properly operational on the Linux machine is made to check for the absolute minumum of

1. master (postfix process) that runs with uid root
2. and one (postfix) username binded proccess

If the amount of processes on the host is less than this minimum number and the netcat is unable to simulate a "half-mail" sent, the configured Postfix alarm Action (media and Email) will take place, and you will get immediately notified, that the monitored Mail server has issue!

The idea is to use a small one liner connection with netcat and half simulate a normal SMTP transaction just like you would normally do:

root@pcfrxen:/root # telnet localhost 25
Trying 127.0.0.1…
Connected to localhost.
Escape character is '^]'.
220 This is Mail2 Pc-Freak.NET ESMTP
HELO localhost
250 This is Mail2 Pc-Freak.NET
MAIL FROM:<hipopo@pc-freak.net>
250 ok
RCPT TO:<hip0d@remote-smtp-server.com>

and then disconnect the connection.

1. Create new zabbix userparameter_smtp_check.conf file

The simple userparameter one liner script to do the task looks like this:

# vi /etc/zabbix/zabbix_agent.d/userparameter_smtp_check.conf

UserParameter=smtp.check,(if [[ $(echo -e “HELO localhost\n MAIL FROM: root@$HOSTNAME\n RCPT TO: report-email@your-desired-mail-server.com\n QUIT\n” | /usr/bin/nc localhost 25 -w 5 2>&1 | grep -Ei ‘220\s.*\sESMTP\sPostfix|250\s\.*|250\s\.*\sOk|250\s\.*\sOk|221\.*\s\w’|wc -l) == ‘5’ ]]; then echo "SMTP OK 1"; else echo "SMTP NOK 0"; fi)

Set the proper permissions so either file is owned by zabbix:zabbix or it is been able to be read from all system users.

# chmod a+r /etc/zabbix/zabbix_agent.d/userparameter_smtp_check.conf

2. Create a new Template for the Mail server monitoring

Just like any other template name it with what fits you as you see, I've call it PROD SMTP Monitoring, as the template is prepared to specifically monitor In Production Linux machines, and a separate template is used to monitor the Quality Assurance (QAs) as well as PreProd (Pre Productions).

3. Create the followng Items and Depedent Item to process zabbix-agent received data from the Userparam script

Above is the list of basic Items and Dependent Item you will need to configure inside the SMTP Check zabbix Template.

The Items should have the following content and configurations:

*Name: postfix_main_proc.service
Type: Zabbix agent(active)
*Key: proc.num[master,root]
Type of Information: Numeric (unassigned)
*Update interval: 30s
Custom Intervals: Flexible
*History storage period: 90d
*Trend storage period: 365d
Show Value: as is
Applications: Postfix Checks
Populated host inventory field: -None-
Description: The item counts master daemon process that runs Postfix daemons on demand

Where the arguments pased to proc.num[] function are:
master is the process that is being looked up for and root is the username with which the the postfix master daemon is running. If you need to adapt it for qmail or exim that shouldn't be a big deal you only have to in advance check the exact processes that are normally running on the machine
and configure a similar process check for it.

*Name: postfix_sub_procs.service_cnt
Type: Zabbix agent(active)
*Key: proc.num[,postfix]
Type of information: Numeric (unassigned)
Update Interval: 30s
*History Storage period: Storage Period 90d
*Trend storage period: Storage Period 365d
Description: The item counts master daemon processes that runs postfix daemons on demand.

Here the idea with this Item is to check the number of processes that are running with user / groupid that is postfix. Again for other SMPT different from postfix, just set it to whatever user / group
you would like zabbix to look up for in Linux the process list. As you can see here the check for existing postfix mta process is done every 30 seconds (for more critical environments you can put it to less).

For simple zabbix use this Dependent Item is not necessery required. But as we would like to process more closely the output of the userparameter smtp script, you have to set it up.
If you want to write graphical representation by sending data to Grafana.

*Name: postfix availability check
Key: postfix_boolean_check[boolean]
Master Item: PROD SMTP Monitoring: postfix availability check
Type of Information: Numeric unassigned
*History storage period: Storage period 90d
*Trend storage period: 365d
Applications: Postfix Checks
Description: It returns boolean value of SMTP check
1 – True (SMTP is OK)
0 – False (SMTP does not responds)
Enabled: Tick

*Name: postfix availability check
*Key: smtp.check
Custom intervals: Flexible
*Update interval: 30 m
History sotrage period: Storage Period 90d
Applications: Postfix Checks
Populates host inventory field: -None-
Description: This check is testing if the SMTP relay is reachable, without actual sending an email
Enabled: Tick

4. Configure following Zabbix Triggers

Note: The severity levels you should have previosly set in Zabbix up to your desired ones.

Name: postfix master root process is not running
*Problem Expression: {PROD SMTP Monitoring:proc.num[master,root].last()}<1
OK event generation: Recovery expression
*Recovery Expression: {PROD SMTP Monitoring:proc.num[master,root].last()}>=1
Allow manual close: Tick
Description: The item counts master daemon process that runs Postfix daemon on demand.
Enabed: Tick

I would like to have an AUTO RESOLVE for any detected mail issues, if an issue gets resolved. That is useful especially if you don't have the time to put the Zabbix monitoring in Maintainance Mode during Operating system planned updates / system reboots or unexpected system reboots due to electricity power loss to the server colocated – Data Center / Rack .

*Name: postfix master sub processes are not running
*Problem Expression: {P09 PROD SMTP Monitoring:proc.num[,postfix].last()}<1
PROBLEM event generation mode: Single
OK event closes: All problems
*Recovery Expression: {P09 PROD SMTP Monitoring:proc.num[,postfix].last()}>=1
Problem event generation mode: Single
OK event closes: All problems
Allow manual close: Tick
Enabled: Tick

Name: SMTP connectivity check
Severity: WARNING
*Expression: {PROD SMTP Monitoring:postfix_boolen_check[boolean].last()}=0
OK event generation: Expression
PROBLEM even generation mode: SIngle
OK event closes: All problems
Allow manual close: Tick
Enabled: Tick

5. Configure respective Zabbix Action

As the service is tagged with 'pci service' tag we define the respective conditions and according to your preferences, add as many conditions as you need for the Zabbix Action to take place.

NOTE! :
Assuming that communication chain beween Zabbix Server -> Zabbix Proxy (if zabbix proxy is used) -> Zabbix Agent works correctly you should start receiving that from the userparameter script in Zabbix with the configured smtp.check userparam key every 30 minutes.
Note that this simple nc check will keep a trail records inside your /var/log/maillog for each netcat connection, so keep in mind that in /var/log/maillog on each host which has configured the SMTP Check zabbix template, you will have some records similar to:

# tail -n 50 /var/log/maillog
2023-06-22T09:32:18.164128+02:00 lpgblu01f postfix/smtpd[2690485]: improper command pipelining after HELO from localhost[127.0.0.1]: MAIL FROM: root@your-machine-fqdn-address.com\n RCPT TO: your-supposable-receive-addr@whatever-mail-address.com\n QUIT\n
2023-06-22T09:32:18.208888+02:00 lpgblu01f postfix/smtpd[2690485]: 32EB02005B: client=localhost[127.0.0.1]
2023-06-22T09:32:18.209142+02:00 lpgblu01f postfix/smtpd[2690485]: disconnect from localhost[127.0.0.1] helo=1 mail=1 rcpt=1 quit=1 commands=4
2023-06-22T10:02:18.889440+02:00 lpgblu01f postfix/smtpd[2747269]: connect from localhost[127.0.0.1]
2023-06-22T10:02:18.889553+02:00 lpgblu01f postfix/smtpd[2747269]: improper command pipelining after HELO from localhost[127.0.0.1]: MAIL FROM: root@your-machine-fqdn-address.com\n RCPT TO: your-supposable-receive-addr@whatever-mail-address.com\n QUIT\n
2023-06-22T10:02:18.933933+02:00 lpgblu01f postfix/smtpd[2747269]: E3ED42005B: client=localhost[127.0.0.1]
2023-06-22T10:02:18.934227+02:00 lpgblu01f postfix/smtpd[2747269]: disconnect from localhost[127.0.0.1] helo=1 mail=1 rcpt=1 quit=1 commands=4
2023-06-22T10:32:26.143282+02:00 lpgblu01f postfix/smtpd[2804195]: connect from localhost[127.0.0.1]
2023-06-22T10:32:26.143439+02:00 lpgblu01f postfix/smtpd[2804195]: improper command pipelining after HELO from localhost[127.0.0.1]: MAIL FROM: root@your-machine-fqdn-address.com\n RCPT TO: your-supposable-receive-addr@whatever-mail-address.com\n QUIT\n
2023-06-22T10:32:26.186681+02:00 lpgblu01f postfix/smtpd[2804195]: 2D7F72005B: client=localhost[127.0.0.1]
2023-06-22T10:32:26.186958+02:00 lpgblu01f postfix/smtpd[2804195]: disconnect from localhost[127.0.0.1] helo=1 mail=1 rcpt=1 quit=1 commands=4
2023-06-22T11:02:26.924039+02:00 lpgblu01f postfix/smtpd[2860398]: connect from localhost[127.0.0.1]
2023-06-22T11:02:26.924160+02:00 lpgblu01f postfix/smtpd[2860398]: improper command pipelining after HELO from localhost[127.0.0.1]: MAIL FROM: root@your-machine-fqdn-address.com\n RCPT TO: your-supposable-receive-addr@whatever-mail-address.com\n QUIT\n
2023-06-22T11:02:26.963014+02:00 lpgblu01f postfix/smtpd[2860398]: EB08C2005B: client=localhost[127.0.0.1]
2023-06-22T11:02:26.963257+02:00 lpgblu01f postfix/smtpd[2860398]: disconnect from localhost[127.0.0.1] helo=1 mail=1 rcpt=1 quit=1 commands=4
2023-06-22T11:32:29.145553+02:00 lpgblu01f postfix/smtpd[2916905]: connect from localhost[127.0.0.1]
2023-06-22T11:32:29.145664+02:00 lpgblu01f postfix/smtpd[2916905]: improper command pipelining after HELO from localhost[127.0.0.1]: MAIL FROM: root@your-machine-fqdn-address.com\n RCPT TO: your-supposable-receive-addr@whatever-mail-address.com\n QUIT\n
2023-06-22T11:32:29.184539+02:00 lpgblu01f postfix/smtpd[2916905]: 2CF7D2005B: client=localhost[127.0.0.1]
2023-06-22T11:32:29.184729+02:00 lpgblu01f postfix/smtpd[2916905]: disconnect from localhost[127.0.0.1] helo=1 mail=1 rcpt=1 quit=1 commands=4

That's all folks use the :
Configuration -> Host (menu)

and assign the new SMTP check template to as many of the Linux hosts where you have setup the Userparameter script and Enjoy the new mail server monitoring at hand.

Tags: close, Connected, Enabled Tick, field, How to, Information Numeric, look, mail server, netcat, parameter, Postfix Mail, problem, script, setup, simple, Warning, work
Posted in Linux, Monitoring, System Administration, Zabbix | 3 Comments »

Saint Markianos and Martyrios a church reader and sub-deacon holy martyrs for Christ – The feast of Sub-deacons

Sunday, October 25th, 2020

saint-Markian-and-Saint-Martirios-cleargymen-church-martyrs-3rd-century
Saint Markianos (Saint Markian) and Martyrios are little known saints in the Western realm and there is too little of information in English about this two early martyrs who lived circa year 340. What is special about them is that besides being a strong confessors of the True Eastern Orthodox faith, they served in the Church as simple 'reader' and 'sub-deacon'. This two designations were very much respected in the early Church as sub-deacons were usually the ones who have served in the Church inseparable as a Church service helpers to the patriarchs or some high clergy as Metropolitans and Bishops. We have many saints in the Church that are from a simple warriors as Saint Georg and Saint Dimitrios the Wonderworker (The MyrhBringer) to monks, bishops, patriarchs and pretty much all kind of people from the society from the begger to the richest and most famous kings and queens. However it is rare to meet in the ( Act of the Martyrs – latin: Acta Martyrum), to find canonized saints that were in the lowest step in Church hierarchy as a simple 'psalm' and holy writtings reader or a sub-deacon. A Sub-deacon for those who don't know is a pearon that is a like a servant helper to the priest or bishop) that has been responsible for helping with the Church service and resolution of material and administrative needs of the christian community.
Usually in the Eastern Orthodox Church, the church reader or sub-deacons were and asre still called hipodeacon or "ipodiakon" in Greek / Slavonic church language), they didn't have the right at that early ages of christianity to publicly teach on faith matters or do apologetics (defendings of faith), however this 2 saintly man Markianos and Martyrios seem to have been a burning with the power of the spirit of God in their heart and the situation they were put in when the Church was under persecution and the patriarch Paul of Constantinople I (was patriarch from 340 ~ 350 AD). Saint Paul removed from his Church headship sent to Exile in Armenia and some time after drawned. He is commemorated in the Church on 6th of November. Hence considering situation St. Markian and Martyrius had to either defend and die for the faith or be scared and run away far in the caves or distant places of the empire such as villages on the outskits far away from the center city Rome …

The Heresy of Arius has been the most modern and the new modified faith claiming Christianity gathering followers in a viral way, and due to that the Arians have been in position where most of the public authorities in the Roman empire has been on their side against the Orthodox Christians.

Marcian_and_Martyrius_the_notaries_of_Constantinople-circa-355AD.

Due to that in the church communities in near and distant lands of empire, the Arians were fiercely persecuting the Orthodox, and for a time even Emperor Saint Constantine The Great were deceived by their hypocrisy. It was terrible times for true confessors of faith. But not only Arians were persecuting Christians, as paganism were still deeply rooted in many of the lands and the Edict of Mediolan who gave equal rights to the religion in AD 313 was not strictly followed and senators of Roman regions with Paganist beliefs, were also harshly raising persucutions against their enemies the Christians who according to them are destroying the ancient culture and beautfy of paganism, not venerating the old pagan gods and against the wicked debauchery customs who were followed by pagans in 3rd / 4th century.

beheading-of-saint-Martyrios

Practically everyone who have admitted publicly Jesus Christ as a Creator of the World and a Son of God one hipostasys of the Holy Trinity God The Father, The Son and the Holy Spirit, were captured put to prison and quickly executed, if they don't turn out from their christian beliefs.

Arians has taken a lead even more with the set on the throne of Emperor Constantius II the son of Constantine I-st, as he has also fallen in the Arianism* heresy and who has taken in the court as a close advisory Eusebius and Philip who due to their half-pagan half-arian half superstitious understanding of the world have led a fierce war against Christianity and did a lot of evils to Christ Church.

* Arianism – believes that Jesus Christ is the Son of God, who was begotten by God the Father, and is distinct from the Father (therefore subordinate to him), but the Son is also God the Son but not co-eternal with God the Father. Arian theology was first attributed to Arius (c. AD 256–336), a Christian presbyter in Alexandria of Egypt.
saint-Markian-and-Saint-Martirios-cleargymen-church-martyrs-3rd-century.jpg
Until dethronment of Patriarch Paul I, St. Markianos and St. Martyrios have been a notaries of St. Paul (a typist to the patriarch and a kind of personal secretaries of the Patriarch) besides serving as Church reader and sub-deacon. They were famous for their time with their warm preaching of the Words of God – the Gospel of the Christ following the example of the apostles. Due to the raising heresies they also take an active part in writting many documents against the heretical "arians" and so called "macedonians" who teached anti-christian teachings who were newly invented and unknown to the ancient church teachings. They've had a special gift from God to be able to speak in a way to defend the faith so noone with his knowledge or high-education couldn't stand overcome them in disputes on church matters and many times they have disputed with Arian heretics exposing their fallacy (delusions) putting them to shame.

After the exile of Patriarch Paul heresy-archs arians turned their poisonous hatred against the patriarch two pupils Markianos and Martyrios. Craftly acting they acted slyly with a craftul lie and promised them a lot of gold a good place in the emperor's court, to raise them in the church hierarcy (in the part of the church which was already confessing arian heresy) and give them a lot of privileges from the king with the condition to accept, support and confess arianism.

But God's servents despised everything from this world, rejected the offered golden gifts, preferred eternal Heavenly honors than short and vain worldly and even laughed at them.

As Arians saw nothing can't convince them to their malice teaching, heretics condemned them to death, which was desired by the confessors (which remembered well the exile and the manly martyrdom of their teacher St. Patriarch Paul) and with all their being desired to be with Christ in the Eternal prepared palaces, where life will be without end in never ending bliss as promised by Christ in the Holy Scriptures. They preferred Christ more than the temporary life enjoyments.

When brought to the place of the execution of their false made accusement and sentence for being blasphemers of Christ, two saints asked for a small time
to pray. Brough up their eyes to the heaven and prayed with the words:

" – Oh Lord, who have unseenly created our hearts, who arrange all our deeds – "He formed the hearts of them all; he understands everything they do." (Psalm 33:15), receive with peace the souls of your servents, because we're mortified for your name – "Yet for Your sake we are killed all day long; We are accounted as sheep for the slaughter." (Psalm 44:22). We're joyful that you give us such a death, we depart from this life because of your name. Let us to participate in the eternal life in You, the source and giver of life."

Praying with this words, they bowed their holy heads and under sword and was killed by beheading by the unfortunate arians because of their confession of the divinity of Christ as true uncreated Son of God who existed before all ages before the creation of the world as we Christians believe to this date.

Some of the Christians took their holy relics and buried them outside the Melandissia Gate of the Constantinople. Later Saint John of Chrysostom built a church in their name over the place of their miracle-working relics. There the sick for many ages received divine healings of different incurable diseases by the prayers of the holy martyrs of God, Praised in Trinity in all ages.

By the prayers of your Holy Martyrs St. Markianos and Martyrios Lord Jesus Christ have mercy on us !

Tags: against, all kind of people, Christ, Christianity, church, due, eastern orthodox church, faith, Greek Slavonic, language, lead, martyrs, Patriarch Paul, simple, terrible times, two
Posted in Christianity | No Comments »

Build and install Docker Squid Open Proxy image on Kubernetes cluster

Tuesday, October 30th, 2018

As Docker containerization is starting to become a standard for installing a brand new servers especially servers who live in Self-Made Clusters based on Orchestration technologites like Kubernetes (k8s) (check out – http://kubernetes.io),

Recently, I've had the task to set-up a Squid Cache (Open Proxy) server on a custom Port number to make it harder for internet open proxy scanners to identify and ship it to a customer.

What is Squid Open Proxy?

An open proxy is a proxy server that is accessible by any Internet user, in other words anyone could access the proxy without any authentication.

Squid is a caching proxy for the Web supporting HTTP, HTTPS, FTP and other protocols.
It reduces bandwidth and improves response times by caching and reusing frequently-requested web pages.
Squid has extensive access controls and makes a great server accelerator.
It runs on most available operating systems, including Windows and is licensed under the GNU GPL.

What is Docker?

For those who hear about Docker for a first time, Docker is an open-source software platform to create, deploy and manage virtualized application containers on a common OS such as GNU / Linux or Windows, it has a surrounding ecosystem of tools. Besides its open source version there is also a commercial version of the product by Docker Inc. the original company that developed docker and is today in active help of the project.

Docker components – picture source docker.com

What is Kubernetes?

Kubernetes, in short, is an open source system for managing clusters of containers. To do this, it provides tools for deploying applications, scaling those application as needed, managing changes to existing containerized applications, and helps you optimize the use of the underlying hardware beneath your containers.
Kubernetes is designed to be extensible and fault-tolerant by allowing application components to restart and move across systems as needed.

Kubernetes is itself not a Platform as a Service (PaaS) tool, but it serves as more of a basic framework, allowing users to choose the types of application frameworks, languages, monitoring and logging tools, and other tools of their choice. In this way, Kubernetes can be used as the basis for a complete PaaS to run on top of; this is the architecture chosen by the OpenShift Origin open source project in its latest release.

Kubernetes architecture (shortly explained) – picture source Wikipedia

The Kubernetes project is written in the Google developed Go programming language, and you can browse its source code on GitHub.

Hence, In this article I'll give a brief introuduction on what is Docker and show you, how to easily:

a. Build Docker Image with Ubuntu, Update the system and Install Squid inside the container
using a sample Dockerfile build file

b. Run Docker Image to test deployed Ubuntu Linux and Squid on top of it works fine

c. Push Docker Image to DockerHub (Docker Images Central Official repository)

d. Deploy (Pull and Run) the new built Docker Ubuntu / Squid Open Proxy Image to the Kubernetes Cluster slave nodes – the K8S Cluster was created using Rancher Enterprise Kubernetes Platform (check out Rancher.net – a bleeding edge tool for k8s quick GUI cluster creation / integration)

1. Install Docker Containerization Software Community Edition

Docker containers are similar to virtual machines, except they run as normal processes (containers), that does not use a Hypervisor of Type 1 or Type 2 and consume less resources than VMs and are easier to manage, nomatter what the OS environment is.

Docker uses cgroups and namespace to allow independent containers to run within a single Linux instance.

Docker Architecture – Picture source docker.com

Below docker install instructions are for Debian / Ubuntu Linux, the instructions for RPM package distros Fedora / CentOS / RHEL are very similar except yum or dnf tool is to be used.

a) Uninstall older versions of docker , docker-engine if present

apt-get -y remove docker docker-engine docker.io

! Previously running docker stuff such as Volumes, Images and networks will be preserved in /var/lib/docker/

b) install prerequired packages and add apt repositories for doko

apt-get update
apt-get install -y apt-transport-https ca-certificates wget software-properties-common

wget https://download.docker.com/linux/debian/gpg
apt-key add gpg
rm -f gpg

Create docker.list apt sources file

echo "deb [arch=amd64] https://download.docker.com/linux/debian $(lsb_release -cs) stable" | sudo tee -a /etc/apt/sources.list.d/docker.list

apt-get update

c) check out the docker policy (will list you a multiple installable versoins of docker).

apt-cache policy docker-ce

docker-ce:
Installed: (none)
Candidate: 17.06.0~ce-0~debian
Version table:
     17.06.0~ce-0~debian 500
        500 https://download.docker.com/linux/debian stretch/stable amd64 Packages
     17.03.2~ce-0~debian-stretch 500
        500 https://download.docker.com/linux/debian stretch/stable amd64 Packages
     17.03.1~ce-0~debian-stretch 500
        500 https://download.docker.com/linux/debian stretch/stable amd64 Packages
     17.03.0~ce-0~debian-stretch 500

d) Install and run docker

apt-get -y install docker-ce
systemctl start docker

systemctl status docker

Previously running docker stuff such as Volumes, Images and networks will be preserved in /var/lib/docker/

2. Build Docker image with Ubuntu Linux OS and Squid inside

To build a docker image all you need to do is have the Dockerfile (which is docker definitions build file), an Official image of Ubuntu Linux OS (that is provided / downloaded from dockerhub repo) and a bunch of docker commands to use apt / apt-get to install the Squid Proxy inside the Docker Virtual Machine Container

In dockerfile it is common to define for use an entrypoint.sh which is file with shell script commands definitions, that gets executed immediately after Docker fetches the OS from its remote repository on top of the newly run OS. It is pretty much like you have configured your own Linux distribution like using Linux from Scratch! to run on a bare-metal (hardware) server and part of the installation OS process you have made the Linux to run a number of scripts or commands during install not part of its regular installation process.

a) Go to https://hub.docker.com/ and create an account for free

The docker account is necessery in order to push the built docker image later on.
Creating the account creates just few minutes time.

b) Create a Dockerfile with definitions for Squid Open Proxy setup

I'll not get into details on the syntax that Dockerfile accepts, as this is well documented on Docker Enterprise Platform official website but in general gettings the basics and starting it is up to a 30 minutes to maximum 1h time.

After playing a bit to achieve the task to have my Linux distribution OS (Ubuntu Xenial) with Squid on installed on top of it with the right configuration of SQUID Cacher to serve as Open Proxy I've ended up with the following Dockerfile.

FROM ubuntu:xenial
LABEL maintainer="hipo@www.pc-freak.net"

ENV SQUID_VERSION=3.5.12-1ubuntu7 \
    SQUID_CACHE_DIR=/var/spool/squid \
    SQUID_LOG_DIR=/var/log/squid \
    SQUID_USER=proxy

RUN apt-get update \
&& apt-get upgrade && apt-get dist-upgrade && DEBIAN_FRONTEND=noninteractive apt-get install -y squid=${SQUID_VERSION}* \
&& rm -rf /var/lib/apt/lists/*

COPY entrypoint.sh /sbin/entrypoint.sh
COPY squid.conf /etc/squid/squid.conf
RUN chmod 755 /sbin/entrypoint.sh

EXPOSE 3128/tcp
ENTRYPOINT [“/sbin/entrypoint.sh”]

You can download the Dockerfile here.

c) Create entrypoint.sh and squid.conf files

Apart from that I've used the following entrypoint.sh (which creates and sets necessery caching and logging directories and launches script on container set-up) permissions for SQUID proxy file that is loaded from the Dockerfile on docker image build time.
To have the right SQUID configuration shipped up into the newly built docker container, it is necessery to prepare a template configuration file – which is pretty much a standard squid.conf file with the following SQUID Proxy configuration for Open Proxy

acl SSL_ports port 443
acl CONNECT method CONNECT

http_access deny !Safe_ports

http_access deny CONNECT !SSL_ports

http_access allow localhost manager
http_access deny manager

http_access allow localhost

http_access allow all

http_port 3128

coredump_dir /var/spool/squid

refresh_pattern ^ftp:           1440    20%     10080
refresh_pattern ^gopher:        1440    0%      1440
refresh_pattern -i (/cgi-bin/|\?) 0     0%      0
refresh_pattern (Release|Packages(.gz)*)$      0       20%     2880
refresh_pattern .               0       20%     4320

Once, I've created, the proper Dockerfile configuration, I've made a tiny shell script, that can create / re-create my docker image multiple times.
Here is the build-docker-image.sh :

#!/bin/sh
username='hipod';
password='';
docker login –username="$username" –password=""
DOCKER_ACC=hipod
#DOCKER_REPO=squid-3.5.12-1ubuntu7
DOCKER_REPO=squid-ubuntu
IMG_TAG=latest

docker build -t $DOCKER_ACC/$DOCKER_REPO:$IMG_TAG .
docker push $DOCKER_ACC/$DOCKER_REPO:$IMG_TAG

– You can download build-docker-image.sh script here

The script uses the docker login command to authenticate non-interactively to https://hub.docker.com
docker build command with properly set DOCKER_ACC (docker account – which is the username of your hub.docker.com account as I've pointed earlier in article), then DOCKER_REPO (docker repository name) – you can get it either from a browser, after you've logged in to dockerhub or assuming you know your username, it should look like:
https://hub.docker.com/u/your-username-name – for example mine is hipod with repository name squid-ubuntu, my squid-ubuntu docker image build is here, you'll also need to provide the password inside the script or if you consider it a security concern, instead type manually from command line docker login and authenticate in advance before running the script, finally the last line docker push pushes to remote docker hub the new build of Ubuntu + SQUID Proxy with a predefined TAG that in my case is latest (as this is my latest build of Squid – if you need a multiple version number of Squid repository just change the tag to the version tag line number.

d) Use the script to build Squid docker image

Next run the script to make and push into docker your new image:

sh build-docker-image.sh

Please consider that in order to work with docker hub push / pull, you will need to have a firewall that allows connection to dockerhub site repo, if for some reason the push / pull fails, check closely your firewall as it is the most likely cause for failure.

3. Run the new docker image to test Squid runs as expected

To make sure the docker image runs properly, you can test it on any machine that has docker.io installed, this is done with a simple cmd:

docker run -d –restart=always -p 3128:3128 hipod/squid-ubuntu:latest

The -d option tells docker to background process / run in detached mode
-p option tells docker to expose port (e.g. make NAT with iptables from the docker virtual container with Linux OS + SQUID listening inside the container on port 3128 to the TCP / IP 3128 server port).
You can use iptables to check the created Network Address Translation rules.

–restart=always option sets the docker restart policy (e.g. when the container is terminating it tells the container to restart the container (OS) after exit), there, you can use as a resetart policy (no, on-failure[:max_retries] , unless-stopped)

For clarity I've created one liner script run-squid.sh you can download here.

sh run-squid.sh

You can use a

ps aux|grep -i docker

root     13169 0.2 0.4 591300 77984 ?        Ssl 13:50   0:09 /usr/bin/dockerd -H fd://
root     13176 0.0 0.2 408244 34972 ?        Ssl 13:50   0:03 docker-containerd –config /var/run/docker/containerd/containerd.toml
root     20875 0.0 0.0 132348   948 pts/1    S+   14:52   0:00 grep -i docker

to see it running as well as the:

docker ps -a

CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS                      PORTS               NAMES
d2eb7ab635cf        c5b0f61227cd        "/bin/sh -c 'apt-get…"   12 minutes ago      Exited (1) 3 minutes ago                        trusting_elion
18476f546562        c5b0f61227cd        "/bin/sh -c 'apt-get…"   37 minutes ago      Exited (1) 37 minutes ago                       admiring_wilson

To connect to the running container later you can use docker attach ID_of_container

docker attach d2eb7ab635cf

command to see the new container runs as well as attach to the newly spawned container

**4. Deploying Dockerized SQUID Open Proxy Cache server to Kubernetes cluster**

My task was to deploy the newly built squid doko image to remote K8s cluster which was set as a default cluster via a context in .kube/config/ or manually set via:

kubectl config use-context my.k8s-cluster.net

I've used the following YAML file with kubectl to deploy:

—
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
labels:
    app: squid-app-33128
name: squid-app-33128
spec:
replicas: 1
template:
    metadata:
      labels:
        app: squid-app-33128
    spec:
      containers:
        – name: squid
          image: hipod/squid-ubuntu:latest
          imagePullPolicy: Always
          ports:
          – containerPort: 3128
            protocol: TCP
          volumeMounts:
            – mountPath: /var/spool/squid
              name: squid-cache
            – mountPath: /var/log/squid
              name: squid-log
          livenessProbe:
            tcpSocket:
              port: 3128
            initialDelaySeconds: 40
            timeoutSeconds: 4
      volumes:
        – name: squid-cache
          emptyDir: {}
        – name: squid-log
          emptyDir: {}

The task included to deploy two different Open Proxy squid servers on separate ports in order to add them external cluster Ingress load balancing via Amazon AWS, thus I actually used following 2 yaml files.

1. squid-app-33128.yaml
2. squid-app-33129.yaml

The actual deployment command was like that:

# deploys pods and adds services to kubernetes cluster

kubectl create -f squid-app-33129.yaml -f squid-app-33128.yaml -f squid-service-33129.yaml squid-service-33128.yaml

You can download the 2 squid-service .yaml below:

1. squid-service-33129.yaml
2. squid-service-33128.yaml

The service is externally exposed via later configured LoadBalancer to make the 2 squid servers deployed into k8s cluster accessible from the Internet by anyoneone without authorization (as a normal open proxies) via TCP/IP ports 33128 and 33129.

Conclusion

Below I explained a few easy steps to follow to;

– build docker image Ubuntu + Squid
– test the image
– deploy the image into a previously prepared k8s cluster

Though it all looks quite simplistic I should say creating the .yaml file took me long. Creating system configuration is not as simple as using the good old .conf files and getting used with the identation takes time.

Now once the LB are configured to play with k8s, you can enjoy the 2 proxy servers. If you need to do some similar task and you don't have to do it for a small fee, contact me.

Tags: applications, apt-get, brand, bunch, cause, check, cmd, command, configured, created, Creating, definitions, Docker Open Proxy, Docker Virtual Machine Container, download, file, fine, Install Docker Containerization Software Community Edition, kind, line, liner, make, metadata, NAT, necessery, need, number, properly, pull, repository, running, server port, servers, shell script, simple, Squid Cache Open Proxy, standard, system, tag, task, tcp, top, use, username, website
Posted in Cloud services, Docker, Educational | No Comments »

Adobe PDF Reader Mac OS X disable (Automatic) Update annoying prompt popping up every now and then

Wednesday, June 8th, 2016

Generally recent'y I've been more and more convinced the less you update the better. I've recently installed Adobe Reader on the Mac Book Air of Svetlana, cause I still have some illusions some PDFs are visible more clearly with Adobe. And soon I was unpleasently suprised that once installed Adobe automatically added auto-update service to the Mac and an auto-update policy that is popping up all the time with offers to update.

This could get crazy every normal person that prefer to not update software so often. There are not much people I know of that like to be puzzled on every next release about the new functionality he don't need anyways but I guess adobe's company aggressiveness escalated nowadays.

Thanksfully as I've googled quickly I found the fix on the common site for Mac problems Mac OS X Daily here

To get rid of the m'f'in Automatic Adobe Reader update

Launch Terminal (located in /Applications/Utilities/)
* At the command prompt, paste this command exactly

defaults write com.adobe.AdobeUpdater.Admin Disable.Update -bool yes

* Hit return to execute the command

You will get no output but anyways this nice command will have created for you inside your home directory (/Library/Preferences/) the file Preferences/com.adobe.AdobeUpdater.Admin.plist

You check the file's existence with command:

svetlana@localhost:~$ ls -al ~/Library/Preferences/com.adobe.AdobeUpdater.Admin.plist
-rw——- 1 svetlana staff 62 Jun 6 19:52 /Users/svetlana/Library/Preferences/com.adobe.AdobeUpdater.Admin.plist

The conent of the file is simple it sets a boolean valiable of Disable.Update to True

cat ~/Library/Preferences/com.adobe.AdobeUpdater.Admin.plist
bplist00?^Disable.Update

This will guarantee you that rebooting your Mac (which I guess was some 6 months ago or just a simple user logging out will not prompt you with the sh*tty annoying Adobe Update Manager

Tags: Adobe Update Manager, Automatic Adobe Reader, Automatic Update, command prompt, disable auto update mac, disable auto-update adobe mac command, file, Launch Terminal, Library Preferences, simple, stop annoying auto update adobe reader prompt, svetlana, tty
Posted in Curious Facts, Everyday Life, Mac OS X | 2 Comments »

Configuring server running both OpenSSHD and Apache to be accessed via HTTPS

Friday, December 18th, 2009

I wanted to make this machine accessible for both me and others also with a simple browser,
I was thinking about configuring this on pc-freak since some time now.
It took me a while until I found a program that does this for me, anyhow luckily I found it.
Itâ€™s called webshell and is working pretty well. Check out the home page of WebShell for download and more info on it. Iâ€™ve succesfully installed it on FreeBSD 7.2.
All that is needed for the program to operate is python 2.3 or higher and python openssl (this is optional),
however most people would desire to have the service running over SSL thus this is mandatory.
On my FreeBSD box I had to install:
the port/package py26-openssl and subversion (this is a prerequirement in order to download the source via svn)
It also necessery to modify the webshell.py and change the shebang's location pointing to python
in freebsd that is:

#!/usr/local/bin/python and not #!/usr/bin/python

as in linux.
Then I copied the downloaded source to /usr/local/web-shell/webshell as well as add recordto rc.local

# echo "/usr/local/web-shell/webshell.py -d" >> /etc/rc.local The last thing I did was manually start the daemon with:
# /usr/local/web-shell/webshell.py -d

Tadam, it's up and running accessing it is as simple as pointing the browser
to a domain name or ip on which the python service is running
Currently the running webshell for pc-freak can be accessed via

Tags: browser, Configuring, domain name, download, HTTPS, program, python, running, simple, succesfully
Posted in System Administration | No Comments »

Check your Server Download / Upload Internet Speed from Console on Linux / BSD / Unix howto

Tuesday, March 17th, 2015

If you've been given a new dedicated server from a New Dedicated-Server-Provider or VPS with Linux and you were told that a certain download speed to the Server is guaranteed from the server provider, in order to be sure the server's connection to the Internet told by service provider is correct it is useful to run a simple measurement console test after logging in remotely to the server via SSH.
Testing connection from Terminal is useful because as you probably know most of Linux / UNIX servers doesn't have a GUI interface and thus it is not possible to test Internet Up / Down Bandwidth through speedtest.net.

1. Testing Download Internet Speed given by ISP / Dedi-Server Provider from Linux Console

For the download speed (internet) test the historical approach was to just try downloading the Linux kernel source code from www.kernel.org with some text browser such as lynx or links count the seconds for which the download is completed and then multiple the kernel source archive size on the seconds to get an approximate bandwidth per second, however as nowdays internet connection speeds are much higher, thus it is better to try to download some Linux distribution iso file, you can still use kernel tar archive but it completed too fast to give you some good (adequate) statistics on Download bandwidth.

If its a fresh installed Linux server probably you will probably not have links / elinks and lynx text internet browers installed so install them depending on deb / rpm distro with:

If on Deb Linuz distro:

root@pcfreak:/root# apt-get install –yes links elinks lynx
…

On RPM Based Linuz distro:

[root@fedora ~]# yum install -y lynx elinks links
…

Conduct Internet Download Speed with links
root@pcfreak:/root# links https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.19.1.tar.xz

(Note that the kernel link is current latest stable Kernel source code archive in future that might change, so try with latest archive.)

You can also use non-interactive tool such as wget, curl or lftp to measure internet download speed

To test Download Internet Speed with wget without saving anything to disk set output to go to /dev/null

root@pcfreak:~# wget -O /dev/null https://www.pc-freak.net//~hipo/hirens-bootcd/HirensBootCD15/Hirens.BootCD.15.0.zip

You see the Download speed is 104 Mbit/s this is so because I'm conducting the download from my local 100Mbit network.

For the test you can use my mirrored version of Hirens BootCD

2. Testing Uplink Internet speed provided by ISP / Server Provider from Linux (SSH) Console

To test your uplink speed you will need lftp or iperf command tool.

root@pcfreak:~# apt-cache show lftp|grep -i descr -A 12
Description: Sophisticated command-line FTP/HTTP client programs
Lftp is a file retrieving tool that supports FTP, HTTP, FISH, SFTP, HTTPS
and FTPS protocols under both IPv4 and IPv6. Lftp has an amazing set of
features, while preserving its interface as simple and easy as possible.
.
The main two advantages over other ftp clients are reliability and ability
to perform tasks in background. It will reconnect and reget the file being
transferred if the connection broke. You can start a transfer in background
and continue browsing on the ftp site. It does this all in one process. When
you have started background jobs and feel you are done, you can just exit
lftp and it automatically moves to nohup mode and completes the transfers.
It has also such nice features as reput and mirror. It can also download a
file as soon as possible by using several connections at the same time.

root@pcfreak:/root# apt-cache show iperf|grep -i desc -A 2
Description: Internet Protocol bandwidth measuring tool
Iperf is a modern alternative for measuring TCP and UDP bandwidth performance,
allowing the tuning of various parameters and characteristics.

To test Upload Speed to Internet connect remotely and upload any FTP file:

root@pcfreak:/root# lftp -u hipo www.pc-freak.net -e 'put Hirens.BootCD.15.0.zip; bye'

On Debian Linux to install iperf:

root@pcfreak:/root# apt-get install –yes iperf

On latest CentOS 7 and Fedora (and other RPM based) Linux, you will need to add RPMForge repository and install with yum

[root@centos ~]# rpm -ivh rpmforge-release-0.5.3-1.el7.rf.x86_64.rpm
…
[root@centos ~]# yum -y install iperf
…

Once having iperf on the server the easiest way currently to test it is to use
serverius.net speedtest server – located at the Serverius datacenters, AS50673 and is running on a 10GE connection with 5GB cap.

root@pcfreak:/root# iperf -c speedtest.serverius.net -P 10
————————————————————
Client connecting to speedtest.serverius.net, TCP port 5001
TCP window size: 16.0 KByte (default)
————————————————————
[ 12] local 83.228.93.76 port 54258 connected with 178.21.16.76 port 5001
[ 7] local 83.228.93.76 port 54252 connected with 178.21.16.76 port 5001
[ 5] local 83.228.93.76 port 54253 connected with 178.21.16.76 port 5001
[ 9] local 83.228.93.76 port 54251 connected with 178.21.16.76 port 5001
[ 3] local 83.228.93.76 port 54249 connected with 178.21.16.76 port 5001
[ 4] local 83.228.93.76 port 54250 connected with 178.21.16.76 port 5001
[ 10] local 83.228.93.76 port 54254 connected with 178.21.16.76 port 5001
[ 11] local 83.228.93.76 port 54255 connected with 178.21.16.76 port 5001
[ 6] local 83.228.93.76 port 54256 connected with 178.21.16.76 port 5001
[ 8] local 83.228.93.76 port 54257 connected with 178.21.16.76 port 5001
[ ID] Interval Transfer Bandwidth
[ 9] 0.0-10.2 sec 4.05 MBytes 3.33 Mbits/sec
[ 10] 0.0-10.2 sec 3.39 MBytes 2.78 Mbits/sec
[ 11] 0.0-10.3 sec 3.75 MBytes 3.06 Mbits/sec
[ 4] 0.0-10.3 sec 3.43 MBytes 2.78 Mbits/sec
[ 12] 0.0-10.3 sec 3.92 MBytes 3.18 Mbits/sec
[ 3] 0.0-10.4 sec 4.45 MBytes 3.58 Mbits/sec
[ 5] 0.0-10.5 sec 4.06 MBytes 3.24 Mbits/sec
[ 6] 0.0-10.5 sec 4.30 MBytes 3.42 Mbits/sec
[ 8] 0.0-10.8 sec 3.92 MBytes 3.03 Mbits/sec
[ 7] 0.0-10.9 sec 4.03 MBytes 3.11 Mbits/sec
[SUM] 0.0-10.9 sec 39.3 MBytes 30.3 Mbits/sec

You see currently my home machine has an Uplink of 30.3 Mbit/s per second, that's pretty nice since I've ordered a 100Mbits from my ISP (Unguaranteed Bandwidth Connection Speed) and as you might know it is a standard practice for many Internet Proviers to give Uplink speed of 1/4 from the ISP provided overall bandwidth 1/4 would be 25Mbi/s, meaning my ISP (Bergon.NET) is doing pretty well providing me with even more than promised (ordered) bandwidth.

Iperf is probably the choice of most sysadmins who have to do regular bandwidth in local networks speed between 2 servers or test Internet Bandwidth speed on heterogenous network with Linux / BSDs / AIX / HP-UX (UNIXes). On HP-UX and AIX and other UNIXes for which iperf doesn't have port you have to compile it yourself.

If you don't have root /admin permissions on server and there is python language enterpreter installed you can use speedtest_cli.py script to test internet throughput connectivity
speedtest_cli uses speedtest.net to test server up / down link just in case if script is lost in future I've made ownload mirror of speedtest_cli.py is here

Quickest way to test net speed with speedtest_cli.py:

$ lynx -dump https://raw.github.com/sivel/speedtest-cli/master/speedtest_cli.py > speedtest_cli.py
$ chmod +x speedtest_cli.py
python speedtest_cli.py

speedtest_cli_pyhon_script_screenshot-on-gnu-linux-test-internet-network-speed-on-unix

Tags: command tool, connection, debian linux, dedicated server, Internet, internet connection, Interval Transfer Bandwidth, lftp, order, pcfreak, root, root admin, run, server provider, servers, simple, speed, ssh, tar archive, test, Testing Download Internet Speed, text browser, use, zip
Posted in Curious Facts, Linux, System Administration, Various, Web and CMS | No Comments »

The Lord’s prayer in Anglo Saxon (Vikings) like English and Welsh – Pater Nostra English language development 7th to 16 century

Tuesday, December 4th, 2012

The story of Christianization of the nations is one of the most fascinating stories of all times. Yet people and especially historians forgot that. Modern Anthropology is based on some mambo-jambo and some speculations on the basis of rejection of God. It is my believe that people who wish to know what has really happened in the past and have a better understanding on human history should study closely Christianity.

The Lord's prayer and studying the origin and spread among languages of the Lord's prayer is one great Anthropological standpoint. While researching on The Lord's Prayer in as many languages as I can I've stumbled on rather unique fascinating information, which gives one a glimpse on how Ancient Anglo-Saxon (Viking) English looked like?

This inspired me to come with this little article in which I want to share with other brother and sisters in Christ the blessing of hearing and listening to the Lord's prayer in Ancient languages.

In Arabic world, there are plenty of theories that the Qur'an and the ancient prayers from there contain more spiritual power than the modern translated ones. Of Course we Christians knows well the Qur'an does not possess a spiritual power in truth, so I just it just as example as many Islamic people believe that too.
The topic of which languages are more blessed than others has been in wide discussion in Christian realm, way before Islamic people had their Qu'ran written.

In IX – 7th century there was a wide spread belief that there are in practice two God inspired languages, which should be used for Christian faith (Jewish, Greek and Latin).

There were people in the ancient Church for and against the understanding of the only 3 God blessed languages in which the Holy writtings – the Holy Bible and Church Liturgical texts should be written.

God's will for humanity to use mother languages to read holy scriptures and pray as well as the existence of the Autocephalous Orthodox Churches all around the world has shown that this understanding is incorrect.

Even though this, the fact that there was a dispute on the topic had a good reason as the more ancient languages (in my view) had more spiritual deepness compared to newer developed mixed languages; besides that older languages are less mixed up, almost as a rule of thumb the more advanced a language is the more words from other languages it includes.

This is just my personal understanding and not a viewpoint of the one Holy Apostolic Church – The Orthodox Church. The reason why I think so is in the past people were more simple in their thinking and their understanding about the world. In my view simplicity and simple way of thinking allowed our ancestors to be more real and more closer to God, in a way offering them less options for evil, therefore I believe the ancient languages in which the Holy Bible and Church services are done like Church Slavonic, Jewish, Greek, Syriac, Coptic etc.

One of this ancient languages is Anglo-Saxon whose variation nowadays most of the world speaks in the face of modern English. As I felt so joyful to find and listen the Lord's prayer being recited in Anglo-Saxon and other ancient English-Germanic Languages, therein I decided to share the Lord's Prayer in Ancient English from XI-th century (from youtube) .