Posts Tagged ‘ntpd’

Add Zabbix time synchronization ntp userparameter check script to Monitor Linux servers

Tuesday, December 8th, 2020

Zabbix-logo-how-to-make-ntpd-time-server-monitoring-article

 

How to add Zabbix time synchronization ntp userparameter check script to Monitor Linux servers?

We needed to set on some servers at my work an elementary check with Zabbix monitoring to check whether servers time is correctly synchronized with ntpd time service as well report if the ntp daemon is correctly running on the machine. For that a userparameter script was developed called userparameter_ntp.conf the script is simplistic and few a lines of bash shell scripting 
stuff is based on gresping information required from ntpq and ntpstat common ntp client commands to get information about the status of time synchronization on the servers.
 

[root@linuxserver ]# ntpstat
synchronised to NTP server (10.80.200.30) at stratum 3
   time correct to within 47 ms
   polling server every 1024 s

 

[root@linuxserver ]# ntpq -c peers
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
+timeserver1 10.26.239.41     2 u  319 1024  377   15.864    1.270   0.262
+timeserver2 10.82.239.41     2 u  591 1024  377   16.287   -0.334   1.748
*timeserver3 10.82.239.43     2 u   47 1024  377   15.613   -0.553   0.251
 timeserver4 .INIT.          16 u    – 1024    0    0.000    0.000   0.000


Below is Zabbix UserParameter script that does report us 3 important values we monitor to make sure time server synchronization works as expected the zabbix keys we set are ntp.offset, ntp.sync, ntp.exact in attempt to describe what we're fetching from ntp client:

[root@linuxserver ]# cat /etc/zabbix/zabbix-agent.d/userparameter_ntp.conf

UserParameter=ntp.offset,(/usr/sbin/ntpq -pn | /usr/bin/awk 'BEGIN { offset=1000 } $1 ~ /\*/ { offset=$9 } END { print offset }')
#UserParameter=ntp.offset,(/usr/sbin/ntpq -pn | /usr/bin/awk 'FNR==4{print $9}')
UserParameter=ntp.sync,(/usr/bin/ntpstat | cut -f 1 -d " " | tr -d ' \t\n\r\f')
UserParameter=ntp.exact,(/usr/bin/ntpstat | /usr/bin/awk 'FNR==2{print $5,$6}')

In Zabbix the monitored ntpd parameters set-upped looks like this:

 

ntp_time_synchronization_check-zabbix-screenshot.

 

!Note that in above userparameter example, the commented userparameter script is a just another way to do an ntpd offset returned value which was developed before the more sophisticated with more regular expression checks from the /usr/sbin/ntpd via ntpq, perhaps if you want to extend it you can also use another script to report more verbose information to Zabbix if that is required like ouput from ntpq -c peers command:
 

UserParameter=ntp.verbose,(/usr/sbin/ntpq -c peers)

Of course to make the Zabbix fetch necessery data from monitored hosts, we need to set-up further new Zabbix Template with the respective Trigger and Items.

Below are few screenshots including the triggers used.

ntpd_server-time_synchronization_check-zabbix-screenshot-triggers

  • ntpd.trigger

{NTP:net.udp.service[ntp].last(0)}<1

  • NTP Synchronization trigger

{NTP:ntp.sync.iregexp(unsynchronised)}=1

 

 

As you can see from history we have setup our items to Store history of reported data to Zabbix from parameter script for 90 days and update our monitor check, every 30 seconds from the monitored hosts to which Tempate is applied.

Well that's all folks, time synchronization issues we'll be promptly triggering a new Alarm in Zabbix !

How to install OpenNTPD NTP server to synchronize system clock on FreeBSD for better security

Sunday, February 12th, 2012

FreeBSD, OpenBSD, NetBSD and Linux ntpd alternative server to synchronize server system time

Lately I've been researching on ntpd and wrote a two articles on how to install ntpd on CentOS, Fedora and how to install ntpd on FreeBSD and during my research on ntpd, I've come across OpenNTPD and decided to give it a go on my FreeBSD home router.
OpenBSD project is well known for it is high security standards and historically has passed the test of time for being a extraordinary secure UNIX like free operating system.
OpenBSD is developed in parallel with FreeBSD, however the development model of the two free operating systems are way different.

As a part of the OpenBSD to be independant in its basis of software from other free operating systems like GNU / Linux and FreeBSD. They develop the all around free software realm known OpenSSH. Along with OpenSSH, one interesting project developed for the main purpose of OpenBSD is OpenNTPD.

Here is how openntpd.org describes OpenNTPD:

"a FREE, easy to use implementation of the Network Time Protocol. It provides the ability to sync the local clock to remote NTP servers and can act as NTP server itself, redistributing the local clock."

OpenNTPD's accent just like OpenBSD's accent is security and hence for FreeBSD installs which targets security openntpd might be a good choice. Besides that the so popular classical ntpd has been well known for being historically "insecure", remote exploits for it has been released already at numerous times.

Another reason for someone to choose run openntpd instead of ntpd is its great simplicity. openntpd configuration is super simple.

Here are the steps I followed to have openntpd time server synchronize clock on my system using other public accessible openntpd servers on the internet.

1. Install openntpd through pkg_add -vr openntpd or via ports tree

a) For binar install with pkg_add issue:

freebsd# pkg_add -vr openntpd
...

b) if you prefer to compile it from source

freebsd# cd /usr/ports/net/openntpd
freebsd# make install clean
...

2. Enable OpenNTPD to start on system boot:

freebsd# echo 'openntpd_enable="YES"' >> /etc/rc.conf

3. Create openntpd ntpd.conf configuration file

There is a default sample ntpd.conf configuration which can be straight use as a conf basis:

freebsd# cp -rpf /usr/local/share/examples/openntpd/ntpd.conf /usr/local/etc/ntpd.conf

Default ntpd.conf works just fine without any modifications, if however there is a requirement the openntpd server to listen and accept time synchronization requests from only certain hosts add to conf something like:

listen on 192.168.1.2
listen on 192.168.1.3
listen on 2607:f0d0:3001:0009:0000:0000:0000:0001
listen on 127.0.0.1

This configuration will enable only 192.168.1.2 and 192.168.1.3 IPv4 addresses as well as the IPv6 2607:f0d0:3001:0009:0000:0000:0000:0001 IP to communicate with openntpd.

4. Start OpenNTPD service

freebsd# /usr/local/etc/rc.d/openntpd

5. Verify if openntpd is up and running

freebsd# ps axuww|grep -i ntp
root 31695 0.0 0.1 3188 1060 ?? Ss 11:26PM 0:00.00 ntpd: [priv] (ntpd)
_ntp 31696 0.0 0.1 3188 1140 ?? S 11:26PM 0:00.00 ntpd: ntp engine (ntpd)
_ntp 31697 0.0 0.1 3188 1088 ?? S 11:26PM 0:00.00 ntpd: dns engine (ntpd)
root 31700 0.0 0.1 3336 1192 p2 S+ 11:26PM 0:00.00 grep -i ntp

Its also good idea to check if openntpd has succesfully established connection with its peer remote openntpd time servers. This is necessery to make sure pf / ipfw firewall rules are not preventing connection to remote 123 UDP port:

freebsd# sockstat -4 -p 123
USER COMMAND PID FD PROTO LOCAL ADDRESS FOREIGN ADDRESS
_ntp ntpd 31696 4 udp4 83.228.93.76:54555 212.70.148.15:123
_ntp ntpd 31696 6 udp4 83.228.93.76:56666 195.69.120.36:123
_ntp ntpd 31696 8 udp4 83.228.93.76:49976 217.75.140.188:123

By default openntpd is also listening to IPv6 if IPv6 support is enabled in freebsd kernel.

6. Resolve openntpd firewall filtering issues

If there is a pf firewall blocking UDP requests to in/out port 123 within /etc/pf.conf rule like:

block in log on $EXT_NIC proto udp all

Before the blocking rule you will have to add pf rules:

# Ipv4 Open outgoing port TCP 123 (NTP)
pass out on $EXT_NIC proto tcp to any port ntp
# Ipv6 Open outgoing port TCP 123 (NTP)
pass out on $EXT_NIC inet6 proto tcp to any port ntp
# Ipv4 Open outgoing port UDP 123 (NTP)
pass out on $EXT_NIC proto udp to any port ntp
# Ipv6 Open outgoing port UDP 123 (NTP)
pass out on $EXT_NIC inet6 proto udp to any port ntp

where $EXT_NIC is defined to be equal to the external lan NIC interface, for example:
EXT_NIC="ml0"

Afterwards to load the new pf.conf rules firewall has to be flushed and reloaded:

freebsd# /sbin/pfctl -f /etc/pf.conf -d
...
freebsd# /sbin/pfctl -f /etc/pf.conf -e
...

In conclusion openntpd should be more secure than regular ntpd and in many cases is probably a better choice.
Anyhow bear in mind on FreeBSD openntpd is not part of the freebsd world and therefore security updates will not be issued directly by the freebsd dev team, but you will have to regularly update with the latest version provided from the bsd ports to make sure openntpd is 100% secure.

For anyone looking for more precise system clock synchronization and not so focused on security ntpd might be still a better choice. The OpenNTPD's official page states it is designed to reach reasonable time accuracy, but is not after the last microseconds.
 

How to configure NTP server (ntpd) to synchronize server clock over the Internet on FreeBSD

Friday, February 10th, 2012

 

FreeBSD ntpd logo / How to configure ntpd to synchronize with internet time servers on FreeBSD

On FreeBSD ntpd , ntpdc , ntpdate , ntpq doesn't need to be installed via a specific package like on GNU/Linux as they're part of the FreeBSD world (binary standardly shipped with FreeBSD basis system).

The FreeBSD handbook has a chapter explaining thoroughfully on ntp on FreeBSD ,however for the lazy ones here is a short few steps tutorial on how to install and configure ntpd on bsd :

1. Copy sample ntp.conf file to /etc/

freebsd# cp -rpf /usr/src/etc/ntp.conf /etc/ntp/

No need for any modifications if you don't want to apply some specific restrictions on whom can access the ntpd server. If you update regularly the FreeBSD system with freebsd-update or directly by rebuilding the FreeBSD kernel / world adding restrictions might be not necessery..

If you check /usr/src/etc/ntp.conf you will notice freebsd project people are running their own ntp servers , by default ntpd will use this servers to fetch timing information. The exact server hosts which as of time of writting are used can be seen in ntp.conf and are:

server 0.freebsd.pool.ntp.org iburst maxpoll 9
server 1.freebsd.pool.ntp.org iburst maxpoll 9
server 2.freebsd.pool.ntp.org iburst maxpoll 9

2. Add ntpd daemon to load on system boot via /etc/rc.conf

By default ntpd is disabled on FreeBSD, you can see if it is disabled or enabled by invoking:

freebsd# /etc/rc.d/ntpd rcvar
# ntpd
ntpd_enable=NO

To Enable ntpd to get loaded each time it boots , following 3 lines has to be added in /etc/rc.conf .

ntpdate_enable="YES"
ntpdate_flags="europe.pool.ntp.org"
ntpd_enable="YES"

Quick way to add them is to use echo :

echo 'ntpdate_enable="YES" >> /etc/rc.conf
echo 'ntpdate_flags="europe.pool.ntp.org" >> /etc/rc.conf
echo 'ntpd_enable="YES" >> /etc/rc.conf

Now as the 3 rc.conf vars are set to "YES", the ntpd can be started. Without having this variables in /etc/rc.conf , "/etc/rc.d/ntpd start" will refuse to start ntpd.

3. Start the ntpd service

freebsd# /etc/rc.d/ntpd start
...

One interesting note to make is ntpd can also operate without specifying any config file (/etc/ntp.conf), the only requirement for the server to start is to have a properly set ntpdate server, like lets say (ntpdate_flags="europe.pool.ntp.org")

4. Permit only certain host or localhost to "talk" to the ntpd server

If you want to imply some ntp server restrictions, the configuration directives are same like on Linux:

To allow only a a host inside a local network with IP 192.168.0.2 as well as localhost, to be able to fetch time information via ntpd server put inside /etc/ntp.conf:

restrict 127.0.0.1
restrict 192.168.0.1 mask 255.255.255.0 nomodify notrap

If you want to prohibit ntpd to serve as a Network Time Server, to any other host except localhost, add in /etc/ntp.conf :

restrict default ignore

Allowing and denying certain hosts can be also done on pf (packet filter) or ipfw firewall level, and in my view is easier (and less confusing), than adding restrictions through ntp.conf. Besides that using directly the server firewall to apply restrictions is more secure. If for instance a remote exploit vulnerability is discovered affecting your ntpd server. this will not affect you externally as access to the UDP port 123 will be disabled on a firewall level.
Something good to mention is NTP servers communicate between each other using the UDP source/destination (port 123). Hence if the NTPD server has to be publicly accessible and there is a firewall already implemented, access to source/dest port 123 should be included in the configured firewall …

5. Check if the ntp server is running properly / ntp server query operations

[root@pcfreak /home/hipo]# ps axuww|grep -i ntp
root 15647 0.0 0.2 4672 1848 ?? Ss 2:49PM 0:00.04 /usr/sbin/ntpd -c /etc/ntp.conf -p /var/run/ntpd.pid -f /var/db/ntpd.drift

To query the now running ntpd server as well as set various configuration options "on the fly" (e.g. without need for ntp.conf edits and init script restart), a tool called ntpdc exists. ntpdc tool could be used to connect to localhost running ntpd as well as to connect and manage remotely a ntpd server.
The most basic use of ntpdc is to check (server peers).:
freebsd# ntpdc localhost
ntpdc> peers
remote local st poll reach delay offset disp
===================================================

kgb.comnet.bg 83.228.93.76 2 64 377 0.00282 -0.050575 0.06059
*billing.easy-la 83.228.93.76 2 64 377 0.01068 -0.057400 0.06770
=ns2.novatelbg.n 83.228.93.76 2 64 377 0.01001 -0.055290 0.06058

ntpdc has also a non-interactive interface, handy if there is a need for requests to a ntpd to be scripted. To check ntpd server peers non-interactively:

freebsd# ntpdc -p localhost
===================================================
kgb.comnet.bg 83.228.93.76 2 64 377 0.00284 -0.043157 0.06184
=billing.easy-la 83.228.93.76 2 64 377 0.01059 -0.042648 0.05811
*ns2.novatelbg.n 83.228.93.76 2 64 377 0.00996 -0.041097 0.06094

ntpdc has plenty of other ntpd query options, e.g. :

ntpdc> help
ntpdc commands:
addpeer controlkey fudge keytype quit timeout
addrefclock ctlstats help listpeers readkeys timerstats
addserver debug host loopinfo requestkey traps
addtrap delay hostnames memstats reset trustedkey
authinfo delrestrict ifreload monlist reslist unconfig
broadcast disable ifstats passwd restrict unrestrict
clkbug dmpeers iostats peers showpeer untrustedkey
clockstat enable kerninfo preset sysinfo version
clrtrap exit keyid pstats sysstats

ntpdc is an advanced query tool for ntpd , servers. Another tool exists called ntpq which syntax is almost identical to ntpdc . The main difference between the two is ntpq is a monitoring tool mostly used just for monitoring purposes, where ntpdc can also change plenty of things in the server configuration.

For people who want to learn more on ntpd the man page is a great reading , containing chapters describing thoroughfully exactly how NTPD time servers operate, etc.

How to install and configure NTP Server (ntpd) to synchronize Linux server clock over the Internet on CentOS, RHEL, Fedora

Thursday, February 9th, 2012

Every now and then I have to work on servers running CentOS or Fedora Linux. Very typical problem that I observe on many servers which I have to inherit is the previous administrator did not know about the existence of NTP (Network Time Protocol) or forgot to install the ntpd server. As a consequence the many installed server services did not have a correct clock and at some specific cases this caused issues for web applications running on the server or any CMS installed etc.

The NTP Daemon is existing in GNU / linux since the early days of Linux and it served quite well so far. The NTP protocol has been used since the early days of the internet and for centuries is a standard protocol for BSD UNIX.

ntp is available in I believe all Linux distributions directly as a precompiled binary and can be installed on Fedora, CentOS with:

[root@centos ~]# yum install ntp

ntpd synchronizes the server clock with one of the /etc/ntp.conf defined RedHat NTP list

server 0.rhel.pool.ntp.org
server 1.rhel.pool.ntp.org
server 2.rhel.pool.ntp.org

To Synchronize manually the server system clock the ntp CentOS rpm package contains a tool called ntpdate :
Hence its a good practice to use ntpdate to synchronize the local server time with a internet server, the way I prefer to do this is via a government owned ntp server time.nist.gov, e.g.

[root@centos ~]# ntpdate time.nist.gov
8 Feb 14:21:03 ntpdate[9855]: adjust time server 192.43.244.18 offset -0.003770 sec

Alternatively if you prefer to use one of the redhat servers use:

[root@centos ~]# ntpdate 0.rhel.pool.ntp.org
8 Feb 14:20:41 ntpdate[9841]: adjust time server 72.26.198.240 offset 0.005671 sec

Now as the system time is set to a correct time via the ntp server, the ntp server is to be launched:

[root@centos ~]# /etc/init.d/ntpd start
...

To permanently enable the ntpd service to start up in boot time issue also:

[root@centos ~]# chkconfig ntpd on

Using chkconfig and /etc/init.d/ntpd cmds, makes the ntp server to run permanently via the ntpd daemon:

[root@centos ~]# ps ax |grep -i ntp
29861 ? SLs 0:00 ntpd -u ntp:ntp -p /var/run/ntpd.pid -g

If you prefer to synchronize periodically the system clock instead of running permanently a network server listening (for increased security), you should omit the above chkconfig ntpd on and /etc/init.d/ntpd start commands and instead set in root crontab the time to get synchronize lets say every 30 minutes, like so:

[root@centos ~]# echo '30 * * * * root /sbin/ntpd -q -u ntp:ntp' > /etc/cron.d/ntpd

The time synchronization via crontab can be also done using the ntpdate cmd. For example if you want to synchronize the server system clock with a network server every 5 minutes:

[root@centos ~]# crontab -u root -e

And paste inside:

*/5 * * * * /sbin/ntpdate time.nist.gov 2>1 > /dev/null

ntp package is equipped with ntpq Standard NTP Query Program. To get very basic stats for the running ntpd daemon use:

[root@centos ~]# ntpq -p
remote refid st t when poll reach delay offset jitter
======================================================
B1-66ER.matrix. 192.43.244.18 2 u 47 64 17 149.280 41.455 11.297
*ponderosa.piney 209.51.161.238 2 u 27 64 37 126.933 32.149 8.382
www2.bitvector. 132.163.4.103 2 u 1 64 37 202.433 12.994 13.999
LOCAL(0) .LOCL. 10 l 24 64 37 0.000 0.000 0.001

The remote field shows the servers to which currently the ntpd service is connected. This IPs are the servers which ntp uses to synchronize the local system server clock. when field shows when last the system was synchronized by the remote time server and the rest is statistical info about connection quality etc.

If the ntp server is to be run in daemon mode (ntpd to be running in the background). Its a good idea to allow ntp connections from the local network and filter incoming connections to port num 123 in /etc/sysconfig/iptables :

-A INPUT -s 192.168.1.0/24 -m state --state NEW -p udp --dport 123 -j ACCEPT
-A INPUT -s 127.0.0.1 -m state --state NEW -p udp --dport 123 -j ACCEPT
-A INPUT -s 0.0.0.0 -m state --state NEW -p udp --dport 123 -j DROP

Restrictions on which IPs can be connected to the ntp server can also be implied on a ntpd level through /etc/ntp.conf. For example if you would like to add the local network IPs range 192.168.0.1/24 to access ntpd, in ntpd.conf should be added policy:

# Hosts on local network are less restricted.
restrict 192.168.0.1 mask 255.255.255.0 nomodify notrap

To deny all access to any machine to the ntpd server add in /etc/ntp.conf:

restrict default ignore

After making any changes to ntp.conf , a server restart is required to load the new config settings, e.g.:

[root@centos ~]# /sbin/service ntpd restart

In most cases I think it is better to imply restrictions on a iptables (firewall) level instead of bothering change the default ntp.conf

Once ntpd is running as daemon, the server listens for UDP connections on udp port 123, to see it use:

[root@centos ~]# netstat -tulpn|grep -i ntp
udp 0 0 10.10.10.123:123 0.0.0.0:* 29861/ntpd
udp 0 0 80.95.28.179:123 0.0.0.0:* 29861/ntpd
udp 0 0 127.0.0.1:123 0.0.0.0:* 29861/ntpd
udp 0 0 0.0.0.0:123 0.0.0.0:* 29861/ntpd